WO2021070952A1 - 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置 - Google Patents

三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置 Download PDF

Info

Publication number
WO2021070952A1
WO2021070952A1 PCT/JP2020/038359 JP2020038359W WO2021070952A1 WO 2021070952 A1 WO2021070952 A1 WO 2021070952A1 JP 2020038359 W JP2020038359 W JP 2020038359W WO 2021070952 A1 WO2021070952 A1 WO 2021070952A1
Authority
WO
WIPO (PCT)
Prior art keywords
value
dimensional
coding
dimensional data
prediction mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/038359
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
敏康 杉尾
賀敬 井口
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Corp of America
Original Assignee
Panasonic Intellectual Property Corp of America
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Corp of America filed Critical Panasonic Intellectual Property Corp of America
Priority to CN202080066408.2A priority Critical patent/CN114467119A/zh
Priority to JP2021551729A priority patent/JP7633172B2/ja
Publication of WO2021070952A1 publication Critical patent/WO2021070952A1/ja
Priority to US17/708,615 priority patent/US12047605B2/en
Anticipated expiration legal-status Critical
Priority to US18/736,882 priority patent/US12581117B2/en
Priority to JP2025018467A priority patent/JP7812949B2/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/004—Predictors, e.g. intraframe, interframe coding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/40—Tree coding, e.g. quadtree, octree
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Definitions

  • the present disclosure relates to a three-dimensional data coding method, a three-dimensional data decoding method, a three-dimensional data coding device, and a three-dimensional data decoding device.
  • 3D data In the future, devices or services that utilize 3D data are expected to become widespread in a wide range of fields such as computer vision for autonomous operation of automobiles or robots, map information, monitoring, infrastructure inspection, or video distribution.
  • the three-dimensional data is acquired by various methods such as a distance sensor such as a range finder, a stereo camera, or a combination of a plurality of monocular cameras.
  • point cloud As one of the expression methods of three-dimensional data, there is an expression method called point cloud that expresses the shape of the three-dimensional structure by a point cloud in the three-dimensional space. In the point cloud, the position and color of the point cloud are stored. Point clouds are expected to become the mainstream method for expressing three-dimensional data, but point clouds have a very large amount of data. Therefore, in the storage or transmission of 3D data, it is essential to compress the amount of data by coding, as in the case of 2D moving images (for example, MPEG-4 AVC or HEVC standardized by MPEG). Become.
  • 2D moving images for example, MPEG-4 AVC or HEVC standardized by MPEG.
  • point cloud compression is partially supported by a public library (Point Cloud Library) that performs point cloud-related processing.
  • Point Cloud Library a public library that performs point cloud-related processing.
  • Patent Document 1 a technique for searching and displaying facilities located around a vehicle using three-dimensional map data is known (see, for example, Patent Document 1).
  • An object of the present disclosure is to provide a three-dimensional data coding method, a three-dimensional data decoding method, a three-dimensional data coding device, or a three-dimensional data decoding device that can improve the coding efficiency.
  • the three-dimensional data coding method is a three-dimensional data coding method that encodes a plurality of three-dimensional points having a hierarchical structure, and is one or more thirths around the first three-dimensional points. 2 Using the second position information of the three-dimensional point, one of two or more prediction modes for calculating the predicted value of the first position information of the first three-dimensional point is set and set. The predicted value of the predicted mode is calculated, the predicted residual which is the difference between the first position information and the calculated predicted value is calculated, and the set predicted mode and the predicted residual are included. A first bit stream is generated, and in the setting, the prediction mode is set based on the depth of the hierarchical structure of the first three-dimensional point.
  • the three-dimensional data decoding method is a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, and is the first of the plurality of encoded three-dimensional points.
  • the prediction mode of the three-dimensional point, the prediction mode value indicating the coded prediction mode by acquiring the first bit stream including the coded prediction residual, and the coded prediction residual. Is decoded, the predicted value of the predicted mode indicated by the predicted mode value obtained by decoding is calculated, and the predicted value is added to the predicted residual obtained by decoding to obtain the above.
  • the encoded prediction mode which calculates the first position information of the first three-dimensional point and is included in the first bit stream, is set based on the depth of the hierarchical structure of the first three-dimensional point. Prediction mode.
  • the present disclosure can provide a three-dimensional data coding method, a three-dimensional data decoding method, a three-dimensional data coding device, or a three-dimensional data decoding device that can improve the coding efficiency.
  • FIG. 1 is a diagram showing a configuration of a three-dimensional data coding / decoding system according to the first embodiment.
  • FIG. 2 is a diagram showing a configuration example of point cloud data according to the first embodiment.
  • FIG. 3 is a diagram showing a configuration example of a data file in which the point cloud data information according to the first embodiment is described.
  • FIG. 4 is a diagram showing the types of point cloud data according to the first embodiment.
  • FIG. 5 is a diagram showing a configuration of a first coding unit according to the first embodiment.
  • FIG. 6 is a block diagram of the first coding unit according to the first embodiment.
  • FIG. 7 is a diagram showing a configuration of a first decoding unit according to the first embodiment.
  • FIG. 1 is a diagram showing a configuration of a three-dimensional data coding / decoding system according to the first embodiment.
  • FIG. 2 is a diagram showing a configuration example of point cloud data according to the first embodiment.
  • FIG. 3 is a diagram showing a
  • FIG. 8 is a block diagram of the first decoding unit according to the first embodiment.
  • FIG. 9 is a block diagram of the three-dimensional data coding device according to the first embodiment.
  • FIG. 10 is a diagram showing an example of position information according to the first embodiment.
  • FIG. 11 is a diagram showing an example of an ocree representation of position information according to the first embodiment.
  • FIG. 12 is a block diagram of the three-dimensional data decoding device according to the first embodiment.
  • FIG. 13 is a block diagram of the attribute information coding unit according to the first embodiment.
  • FIG. 14 is a block diagram of the attribute information decoding unit according to the first embodiment.
  • FIG. 15 is a block diagram showing a configuration of an attribute information coding unit according to the first embodiment.
  • FIG. 16 is a block diagram of the attribute information coding unit according to the first embodiment.
  • FIG. 17 is a block diagram showing a configuration of an attribute information decoding unit according to the first embodiment.
  • FIG. 18 is a block diagram of the attribute information decoding unit according to the first embodiment.
  • FIG. 19 is a diagram showing a configuration of a second coding unit according to the first embodiment.
  • FIG. 20 is a block diagram of a second coding unit according to the first embodiment.
  • FIG. 21 is a diagram showing a configuration of a second decoding unit according to the first embodiment.
  • FIG. 22 is a block diagram of a second decoding unit according to the first embodiment.
  • FIG. 23 is a diagram showing a protocol stack related to PCC coded data according to the first embodiment.
  • FIG. 24 is a diagram showing a configuration of a coding unit and a multiplexing unit according to the second embodiment.
  • FIG. 25 is a diagram showing a configuration example of coded data according to the second embodiment.
  • FIG. 26 is a diagram showing a configuration example of the coded data and the NAL unit according to the second embodiment.
  • FIG. 27 is a diagram showing an example of semantics of pcc_nal_unit_type according to the second embodiment.
  • FIG. 28 is a diagram showing an example of the delivery order of the NAL unit according to the second embodiment.
  • FIG. 29 is a flowchart of processing by the three-dimensional data coding apparatus according to the second embodiment.
  • FIG. 30 is a flowchart of processing by the three-dimensional data decoding device according to the second embodiment.
  • FIG. 31 is a flowchart of the multiplexing process according to the second embodiment.
  • FIG. 32 is a flowchart of the demultiplexing process according to the second embodiment.
  • FIG. 33 is a diagram showing a configuration of a distribution system according to the third embodiment.
  • FIG. 34 is a diagram showing a configuration example of a bit stream of the coded three-dimensional map according to the third embodiment.
  • FIG. 35 is a diagram for explaining the effect of improving the coding efficiency according to the third embodiment.
  • FIG. 36 is a flowchart of processing by the server according to the third embodiment.
  • FIG. 37 is a flowchart of processing by the client according to the third embodiment.
  • FIG. 38 is a diagram showing an example of syntax of the submap according to the third embodiment.
  • FIG. 39 is a diagram schematically showing the coding type switching process according to the third embodiment.
  • FIG. 40 is a diagram showing an example of syntax of the submap according to the third embodiment.
  • FIG. 41 is a flowchart of the three-dimensional data coding process according to the third embodiment.
  • FIG. 42 is a flowchart of the three-dimensional data decoding process according to the third embodiment.
  • FIG. 43 is a diagram schematically showing the operation of a modified example of the coding type switching process according to the third embodiment.
  • FIG. 44 is a diagram schematically showing the operation of a modified example of the coding type switching process according to the third embodiment.
  • FIG. 45 is a diagram schematically showing the operation of a modified example of the coding type switching process according to the third embodiment.
  • FIG. 40 is a diagram showing an example of syntax of the submap according to the third embodiment.
  • FIG. 41 is a flowchart of the three-dimensional data coding process according to the third embodiment.
  • FIG. 42 is a flowchar
  • FIG. 46 is a diagram schematically showing the operation of a modified example of the difference value calculation process according to the third embodiment.
  • FIG. 47 is a diagram schematically showing the operation of a modified example of the difference value calculation process according to the third embodiment.
  • FIG. 48 is a diagram schematically showing the operation of a modified example of the difference value calculation process according to the third embodiment.
  • FIG. 49 is a diagram schematically showing the operation of a modified example of the difference value calculation process according to the third embodiment.
  • FIG. 50 is a diagram showing an example of volume syntax according to the third embodiment.
  • FIG. 51 is a diagram showing an example of three-dimensional points according to the fourth embodiment.
  • FIG. 52 is a diagram showing a setting example of LoD according to the fourth embodiment.
  • FIG. 53 is a diagram showing an example of a threshold value used for setting LoD according to the fourth embodiment.
  • FIG. 54 is a diagram showing an example of attribute information used for the predicted value according to the fourth embodiment.
  • FIG. 55 is a diagram showing an example of the exponential Golomb code according to the fourth embodiment.
  • FIG. 56 is a diagram showing processing for the exponential Golomb code according to the fourth embodiment.
  • FIG. 57 is a diagram showing an example of syntax of the attribute header according to the fourth embodiment.
  • FIG. 58 is a diagram showing an example of syntax of attribute data according to the fourth embodiment.
  • FIG. 59 is a flowchart of the three-dimensional data coding process according to the fourth embodiment.
  • FIG. 60 is a flowchart of the attribute information coding process according to the fourth embodiment.
  • FIG. 61 is a diagram showing processing for the exponential Golomb code according to the fourth embodiment.
  • FIG. 62 is a diagram showing an example of a reverse lookup table showing the relationship between the remaining code and the value thereof according to the fourth embodiment.
  • FIG. 63 is a flowchart of the three-dimensional data decoding process according to the fourth embodiment.
  • FIG. 64 is a flowchart of the attribute information decoding process according to the fourth embodiment.
  • FIG. 65 is a block diagram of the three-dimensional data coding device according to the fourth embodiment.
  • FIG. 66 is a block diagram of the three-dimensional data decoding device according to the fourth embodiment.
  • FIG. 67 is a diagram showing a first example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 68 is a diagram showing an example of attribute information used for the predicted value according to the fifth embodiment.
  • FIG. 69 is a diagram showing a second example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 70 is a diagram showing a third example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 71 is a diagram showing a fourth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 72 is a diagram showing a fifth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 73 is a diagram showing a sixth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 69 is a diagram showing an example of attribute information used for the predicted value according to the fifth embodiment.
  • FIG. 69 is a diagram showing a second example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 70 is
  • FIG. 74 is a diagram showing a seventh example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 75 is a diagram showing an eighth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 76 is a diagram showing a first example of a binarization table in the case where the prediction mode value according to the fifth embodiment is binarized and encoded.
  • FIG. 77 is a diagram showing a second example of a binarization table in the case where the prediction mode value according to the fifth embodiment is binarized and encoded.
  • FIG. 78 is a diagram showing a third example of a binarization table in the case where the prediction mode value according to the fifth embodiment is binarized and encoded.
  • FIG. 79 is a diagram for explaining an example of encoding the binary data of the binarization table in the case where the prediction mode according to the fifth embodiment is binarized and encoded.
  • FIG. 80 is a flowchart showing an example of coding of the prediction mode value according to the fifth embodiment.
  • FIG. 81 is a flowchart showing an example of decoding the predicted mode value according to the fifth embodiment.
  • FIG. 82 is a diagram showing another example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 83 is a diagram for explaining an example of encoding the binary data of the binarization table in the case where the prediction mode according to the fifth embodiment is binarized and encoded.
  • FIG. 80 is a flowchart showing an example of coding of the prediction mode value according to the fifth embodiment.
  • FIG. 81 is a flowchart showing an example of decoding the predicted mode value according to the fifth embodiment.
  • FIG. 82 is a diagram showing another example of a table showing predicted
  • FIG. 84 is a flowchart showing another example of coding the prediction mode value according to the fifth embodiment.
  • FIG. 85 is a flowchart showing another example of decoding the predicted mode value according to the fifth embodiment.
  • FIG. 86 is a flowchart showing an example of a process of determining whether or not to fix the prediction mode value according to the condition A at the time of coding according to the fifth embodiment.
  • FIG. 87 is a flowchart showing an example of a process of determining whether to set the prediction mode value to a fixed value or to decode according to the condition A at the time of decoding according to the fifth embodiment.
  • FIG. 88 is a diagram for explaining the maximum absolute difference value maxdiff according to the fifth embodiment.
  • FIG. 89 is a diagram showing an example of the syntax according to the fifth embodiment.
  • FIG. 90 is a diagram showing an example of the syntax according to the fifth embodiment.
  • FIG. 91 is a flowchart of the three-dimensional data coding process by the three-dimensional data coding apparatus according to the fifth embodiment.
  • FIG. 92 is a flowchart of the attribute information coding process according to the fifth embodiment.
  • FIG. 93 is a flowchart of the calculation process of the predicted value in the three-dimensional data coding apparatus according to the fifth embodiment.
  • FIG. 94 is a flowchart of the prediction mode selection process according to the fifth embodiment.
  • FIG. 95 is a flowchart of a process for selecting a prediction mode that minimizes the cost according to the fifth embodiment.
  • FIG. 91 is a flowchart of the three-dimensional data coding process by the three-dimensional data coding apparatus according to the fifth embodiment.
  • FIG. 92 is a flowchart of the attribute information coding process according to the fifth embodiment.
  • FIG. 93 is a flowchart of the calculation process of the predicted value in the
  • FIG. 96 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding apparatus according to the fifth embodiment.
  • FIG. 97 is a flowchart of the attribute information decoding process according to the fifth embodiment.
  • FIG. 98 is a flowchart of the calculation process of the predicted value in the three-dimensional data decoding apparatus according to the fifth embodiment.
  • FIG. 99 is a flowchart of the decoding process of the prediction mode according to the fifth embodiment.
  • FIG. 100 is a block diagram showing a configuration of an attribute information coding unit included in the three-dimensional data coding device according to the fifth embodiment.
  • FIG. 101 is a block diagram showing a configuration of an attribute information decoding unit included in the three-dimensional data decoding device according to the fifth embodiment.
  • FIG. 102 is a flowchart showing a process of determining a prediction mode executed by the three-dimensional data coding apparatus according to the sixth embodiment.
  • FIG. 103 is a diagram showing a syntax example of processing for determining a prediction mode executed by the three-dimensional data coding apparatus according to the sixth embodiment.
  • FIG. 104 is a flowchart showing a process of determining a prediction mode executed by the three-dimensional data decoding apparatus according to the sixth embodiment.
  • FIG. 105 is a diagram showing a syntax example of attribute data when the prediction mode fixed flag according to the sixth embodiment is provided for each three-dimensional point.
  • FIG. 106 is a diagram showing an example of syntax of attribute data when the prediction mode fixed flag according to the sixth embodiment is provided for each LoD layer.
  • FIG. 103 is a diagram showing a syntax example of processing for determining a prediction mode executed by the three-dimensional data coding apparatus according to the sixth embodiment.
  • FIG. 104 is a flowchart showing a process of determining a prediction mode
  • FIG. 107 is a flowchart showing an example of the coding process of the prediction mode of the three-dimensional data coding apparatus according to the sixth embodiment.
  • FIG. 108 is a flowchart showing an example of decoding processing in the prediction mode of the three-dimensional data decoding apparatus according to the sixth embodiment.
  • FIG. 109 is a flowchart of the three-dimensional data coding process by the three-dimensional data coding apparatus according to the sixth embodiment.
  • FIG. 110 is a flowchart of the attribute information coding process shown in FIG. 109.
  • FIG. 111 is a flowchart of the calculation process of the predicted value shown in FIG. 110.
  • FIG. 112 is a flowchart of the prediction mode selection process shown in FIG. 111.
  • FIG. 113 is a flowchart showing a specific example of the prediction mode selection process shown in FIG. 112.
  • FIG. 114 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding apparatus according to the sixth embodiment.
  • FIG. 115 is a flowchart of the calculation process of the predicted value shown in FIG. 114.
  • FIG. 116 is a flowchart showing details of the calculation process of the predicted value shown in FIG. 115.
  • FIG. 117 is a flowchart of the prediction mode and the calculation process of the quantization value shown in FIG. 116.
  • FIG. 118 is a flowchart of processing when the three-dimensional data coding apparatus according to the sixth embodiment does not fix the prediction mode.
  • FIG. 118 is a flowchart of processing when the three-dimensional data coding apparatus according to the sixth embodiment does not fix the prediction mode.
  • FIG. 119 is a flowchart of processing when the three-dimensional data decoding apparatus according to the sixth embodiment does not fix the prediction mode.
  • FIG. 120 is a diagram showing another example of the syntax of the attribute data according to the sixth embodiment.
  • FIG. 121 is a flowchart showing an example of coding processing in the prediction mode of the three-dimensional data coding apparatus according to the sixth embodiment.
  • FIG. 122 is a flowchart showing an example of decoding processing in the prediction mode of the three-dimensional data decoding apparatus according to the sixth embodiment.
  • FIG. 123 is a flowchart showing another example of the calculation process of the predicted value shown in FIG. 110.
  • FIG. 124 is a flowchart of the prediction mode selection process shown in FIG. 123.
  • FIG. 125 is a flowchart showing a specific example of the prediction mode selection process shown in FIG. 124.
  • FIG. 126 is a flowchart showing another example of the prediction mode and the calculation process of the quantization value shown in FIG. 115.
  • FIG. 127 is a diagram showing an example of attribute information used for the predicted value.
  • FIG. 128 is a diagram showing an example of a reference relationship according to the seventh embodiment.
  • FIG. 129 is a diagram showing a calculation example of QW according to the seventh embodiment.
  • FIG. 130 is a diagram showing a calculation example of the predicted residual according to the seventh embodiment.
  • FIG. 131 is a diagram showing an example of a prediction tree used in the three-dimensional data coding method according to the eighth embodiment.
  • FIG. 132 is a flowchart showing an example of the three-dimensional data coding method according to the eighth embodiment.
  • FIG. 133 is a flowchart showing an example of the three-dimensional data decoding method according to the eighth embodiment.
  • FIG. 134 is a diagram for explaining a method of generating a predicted tree according to the eighth embodiment.
  • FIG. 135 is a diagram for explaining a first example of the prediction mode according to the eighth embodiment.
  • FIG. 136 is a diagram showing a second example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 137 is a diagram showing a specific example of a second example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 138 is a diagram showing a third example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 139 is a diagram showing a fourth example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 140 is a diagram showing a fifth example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 141 is a diagram showing a sixth example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 142 is a diagram showing a seventh example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 143 is a diagram showing a first example of a binarization table in the case where the prediction mode value according to the eighth embodiment is binarized and encoded.
  • FIG. 144 is a diagram showing a second example of a binarization table in the case where the prediction mode value according to the eighth embodiment is binarized and encoded.
  • FIG. 145 is a diagram showing a third example of a binarization table in the case where the prediction mode value according to the eighth embodiment is binarized and encoded.
  • FIG. 146 is a diagram for explaining an example of encoding the binary data of the binarization table in the case where the prediction mode according to the eighth embodiment is binarized and encoded.
  • FIG. 147 is a flowchart showing an example of coding of the prediction mode value according to the eighth embodiment.
  • FIG. 148 is a flowchart showing an example of decoding the predicted mode value according to the eighth embodiment.
  • FIG. 149 is a diagram showing another example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • FIG. 150 is a diagram for explaining an example of encoding the binary data of the binarization table in the case where the prediction mode according to the eighth embodiment is binarized and encoded.
  • FIG. 151 is a flowchart showing another example of coding the prediction mode value according to the eighth embodiment.
  • FIG. 152 is a flowchart showing another example of decoding the predicted mode value according to the eighth embodiment.
  • FIG. 153 is a flowchart showing an example of a process of determining whether or not to fix the prediction mode value according to the condition A at the time of coding according to the eighth embodiment.
  • FIG. 154 is a flowchart showing an example of a process of determining whether to set the prediction mode value to a fixed value or to decode according to the condition A at the time of decoding according to the eighth embodiment.
  • FIG. 155 is a diagram showing an example of the syntax of the header of the position information according to the eighth embodiment.
  • FIG. 156 is a diagram showing an example of the syntax of the position information according to the eighth embodiment.
  • FIG. 157 is a diagram showing another example of the syntax of position information according to the eighth embodiment.
  • FIG. 158 is a diagram showing an example of a prediction tree used in the three-dimensional data coding method according to the ninth embodiment.
  • FIG. 159 is a diagram showing another example of the syntax of position information according to the ninth embodiment.
  • FIG. 160 is a diagram showing an example of the configuration of a prediction tree commonly used for coding the position information and the attribute information according to the ninth embodiment.
  • FIG. 161 is a flowchart showing an example of a three-dimensional data coding method according to a modification of the ninth embodiment.
  • FIG. 162 is a flowchart showing an example of a three-dimensional data decoding method according to a modified example of the ninth embodiment.
  • FIG. 163 is a diagram showing an example of the syntax of the header of the attribute information according to the ninth embodiment.
  • FIG. 164 is a diagram showing another example of the syntax of attribute information according to the ninth embodiment.
  • FIG. 165 is a diagram showing an example of the syntax of position information and attribute information according to the ninth embodiment.
  • FIG. 166 is a flowchart of processing of the three-dimensional data coding apparatus according to the eighth and ninth embodiments.
  • FIG. 167 is a flowchart of processing of the three-dimensional data decoding apparatus according to the eighth and ninth embodiments.
  • FIG. 168 is a block diagram of the three-dimensional data creation device according to the tenth embodiment.
  • FIG. 169 is a flowchart of the three-dimensional data creation method according to the tenth embodiment.
  • FIG. 170 is a diagram showing a configuration of the system according to the tenth embodiment.
  • FIG. 170 is a diagram showing a configuration of the system according to the tenth embodiment.
  • FIG. 171 is a block diagram of the client device according to the tenth embodiment.
  • FIG. 172 is a block diagram of the server according to the tenth embodiment.
  • FIG. 173 is a flowchart of the three-dimensional data creation process by the client device according to the tenth embodiment.
  • FIG. 174 is a flowchart of the sensor information transmission process by the client device according to the tenth embodiment.
  • FIG. 175 is a flowchart of the three-dimensional data creation process by the server according to the tenth embodiment.
  • FIG. 176 is a flowchart of the three-dimensional map transmission process by the server according to the tenth embodiment.
  • FIG. 177 is a diagram showing a configuration of a modified example of the system according to the tenth embodiment.
  • FIG. 177 is a diagram showing a configuration of a modified example of the system according to the tenth embodiment.
  • FIG. 178 is a diagram showing a configuration of a server and a client device according to the tenth embodiment.
  • FIG. 179 is a diagram showing a configuration of a server and a client device according to the tenth embodiment.
  • FIG. 180 is a flowchart of processing by the client device according to the tenth embodiment.
  • FIG. 181 is a diagram showing the configuration of the sensor information collection system according to the tenth embodiment.
  • FIG. 182 is a diagram showing an example of the system according to the tenth embodiment.
  • FIG. 183 is a diagram showing a modified example of the system according to the tenth embodiment.
  • FIG. 184 is a flowchart showing an example of application processing according to the tenth embodiment.
  • FIG. 184 is a flowchart showing an example of application processing according to the tenth embodiment.
  • FIG. 185 is a diagram showing sensor ranges of various sensors according to the tenth embodiment.
  • FIG. 186 is a diagram showing a configuration example of the automatic driving system according to the tenth embodiment.
  • FIG. 187 is a diagram showing a configuration example of a bit stream according to the tenth embodiment.
  • FIG. 188 is a flowchart of the point group selection process according to the tenth embodiment.
  • FIG. 189 is a diagram showing a screen example of the point group selection process according to the tenth embodiment.
  • FIG. 190 is a diagram showing a screen example of the point group selection process according to the tenth embodiment.
  • FIG. 191 is a diagram showing a screen example of the point group selection process according to the tenth embodiment.
  • the three-dimensional data coding method is a three-dimensional data coding method that encodes a plurality of three-dimensional points having a hierarchical structure, and is one or more thirths around the first three-dimensional points. 2 Using the second position information of the three-dimensional point, one of two or more prediction modes for calculating the predicted value of the first position information of the first three-dimensional point is set and set. The predicted value of the predicted mode is calculated, the predicted residual which is the difference between the first position information and the calculated predicted value is calculated, and the set predicted mode and the predicted residual are included. A first bit stream is generated, and in the setting, the prediction mode is set based on the depth of the hierarchical structure of the first three-dimensional point.
  • the position information can be encoded using the predicted value of one prediction mode set based on the depth of the hierarchical structure among the two or more prediction modes, so that the coding efficiency of the position information can be improved. Can be improved.
  • a prediction mode value equal to or less than the value of the depth of the hierarchical structure of the first three-dimensional point may be set, and the prediction mode value may indicate the prediction mode.
  • the first bit stream may further include a number of prediction modes indicating the number of the two or more prediction modes.
  • the predicted mode value indicating the set predicted mode is encoded using the number of predicted modes, and the first bit stream sets the encoded predicted mode value. It may be included as a prediction mode.
  • the prediction mode value may be encoded by a truncated unary code having the number of prediction modes as the maximum value.
  • each of the first position information and the second position information includes three elements, and in the setting, the predicted value of each element of the three elements included in the first position information is calculated.
  • a common prediction mode may be set for the three elements.
  • each of the first position information and the second position information includes three elements, and in the setting, the predicted value of each element of the three elements included in the first position information is calculated.
  • an independent prediction mode may be set for each of the three elements.
  • the three-dimensional data decoding device can decode each element independently.
  • each of the first position information and the second position information includes three elements, and in the setting, the predicted value of each element of the three elements included in the first position information is calculated.
  • a common prediction mode may be set for two of the three elements, and a prediction mode independent of the two elements may be set for the remaining one element.
  • the code amount of the prediction mode value for the two elements can be reduced.
  • the three-dimensional data decoding device can independently decode the remaining one element.
  • the second bit stream that does not include the prediction mode value may be generated without encoding the prediction mode value indicating the prediction mode.
  • the positive / negative information indicating whether the predicted residual is positive or negative is not encoded.
  • the third bit stream that does not include the positive / negative information may be generated.
  • the three-dimensional data decoding method is a three-dimensional data decoding method for decoding a plurality of three-dimensional points having a hierarchical structure, and is the first of the plurality of encoded three-dimensional points.
  • the prediction mode of the three-dimensional point, the prediction mode value indicating the coded prediction mode by acquiring the first bit stream including the coded prediction residual, and the coded prediction residual. Is decoded, the predicted value of the predicted mode indicated by the predicted mode value obtained by decoding is calculated, and the predicted value is added to the predicted residual obtained by decoding to obtain the above.
  • the encoded prediction mode which calculates the first position information of the first three-dimensional point and is included in the first bit stream, is set based on the depth of the hierarchical structure of the first three-dimensional point. Prediction mode.
  • the prediction mode value including the coded prediction mode included in the first bit stream may be equal to or less than the value of the depth of the hierarchical structure of the first three-dimensional point.
  • the first bit stream may include a number of prediction modes indicating the number of the two or more prediction modes.
  • the encoded prediction mode value may be decoded by a truncated unary code having the number of prediction modes as the maximum value.
  • each of the first position information and the second position information of one or more second three-dimensional points around the first three-dimensional point includes three elements, and the prediction mode is the first. It is used to calculate the predicted value of each element of the three elements included in the position information, and may be set in common for the three elements.
  • each of the first position information and the second position information of one or more second three-dimensional points around the first three-dimensional point includes three elements, and the prediction mode is the first. It is used to calculate the predicted value of each element of the three elements included in the position information, and may be set independently for each of the three elements.
  • each of the first position information and the second position information of one or more second three-dimensional points around the first three-dimensional point includes three elements, and the prediction mode is the first. It is used to calculate the predicted value of each element of the three elements included in the position information, is set in common for two of the three elements, and is set for the remaining one element. It may be set independently of one element.
  • the prediction value of a specific prediction mode may be calculated in the calculation of the prediction value.
  • the predicted residual is set to 0 in the calculation of the first position information. Alternatively, it may be treated as a positive number.
  • the three-dimensional data coding device is a three-dimensional data coding device that encodes a plurality of three-dimensional points, and includes a processor and a memory, and the processor includes the memory.
  • a predicted residual which is the difference between the first position information and the calculated predicted value, is calculated to generate a first bit stream including the set prediction mode and the predicted residual, and in the setting, the first bit stream is generated.
  • the prediction mode is set based on the depth of the hierarchical structure of the first three-dimensional point.
  • the position information can be encoded using the predicted value of one prediction mode set based on the depth of the hierarchical structure among the two or more prediction modes, so that the coding efficiency of the position information can be improved. Can be improved.
  • the three-dimensional data decoding device is a three-dimensional data decoding device that decodes a plurality of three-dimensional points, and includes a processor and a memory, and the processor uses the memory.
  • the prediction mode value indicating the encoded prediction mode and the prediction obtained by decoding the first bit stream including the predicted residual and decoding the encoded prediction residual are obtained.
  • the first position information of the first three-dimensional point is calculated by calculating the predicted value of the predicted mode indicated by the mode value and adding the predicted value and the predicted residual obtained by decoding.
  • the encoded prediction mode included in the first bit stream is a prediction mode set based on the depth of the hierarchical structure of the first three-dimensional point.
  • a recording medium such as a system, method, integrated circuit, computer program or computer-readable CD-ROM, and the system, method, integrated circuit, computer program. And any combination of recording media may be realized.
  • a three-dimensional data coding method and a three-dimensional data coding device for providing a function of transmitting and receiving necessary information according to an application in the coded data of a three-dimensional point cloud, and the code thereof.
  • a three-dimensional data decoding method and a three-dimensional data decoding device for decoding the encoded data, a three-dimensional data multiplexing method for multiplexing the encoded data, and a three-dimensional data transmission method for transmitting the encoded data will be described. To do.
  • a first coding method and a second coding method are being studied as a coding method (coding method) for point group data, but the configuration of the coded data and the coded data are used as a system.
  • the method of storing in the format is not defined, and there is a problem that the MUX processing (multiplexing), transmission or storage in the coding unit cannot be performed as it is.
  • FIG. 1 is a diagram showing a configuration example of a three-dimensional data coding / decoding system according to the present embodiment.
  • the three-dimensional data coding / decoding system includes a three-dimensional data coding system 4601, a three-dimensional data decoding system 4602, a sensor terminal 4603, and an external connection unit 4604.
  • the three-dimensional data coding system 4601 generates coded data or multiplexed data by encoding point cloud data which is three-dimensional data.
  • the three-dimensional data coding system 4601 may be a three-dimensional data coding device realized by a single device, or may be a system realized by a plurality of devices. Further, the three-dimensional data coding apparatus may include a part of a plurality of processing units included in the three-dimensional data coding system 4601.
  • the three-dimensional data coding system 4601 includes a point cloud data generation system 4611, a presentation unit 4612, a coding unit 4613, a multiplexing unit 4614, an input / output unit 4615, and a control unit 4616.
  • the point cloud data generation system 4611 includes a sensor information acquisition unit 4617 and a point cloud data generation unit 4618.
  • the sensor information acquisition unit 4617 acquires the sensor information from the sensor terminal 4603 and outputs the sensor information to the point cloud data generation unit 4618.
  • the point cloud data generation unit 4618 generates point cloud data from the sensor information and outputs the point cloud data to the coding unit 4613.
  • the presentation unit 4612 presents the sensor information or the point cloud data to the user. For example, the presentation unit 4612 displays information or an image based on sensor information or point cloud data.
  • the coding unit 4613 encodes (compresses) the point cloud data, and outputs the obtained coded data, the control information obtained in the coding process, and other additional information to the multiplexing unit 4614.
  • the additional information includes, for example, sensor information.
  • the multiplexing unit 4614 generates multiplexed data by multiplexing the coded data input from the coding unit 4613, the control information, and the additional information.
  • the format of the multiplexed data is, for example, a file format for storage or a packet format for transmission.
  • the input / output unit 4615 (for example, the communication unit or the interface) outputs the multiplexed data to the outside.
  • the multiplexed data is stored in a storage unit such as an internal memory.
  • the control unit 4616 (or application execution unit) controls each processing unit. That is, the control unit 4616 controls coding, multiplexing, and the like.
  • the sensor information may be input to the coding unit 4613 or the multiplexing unit 4614. Further, the input / output unit 4615 may output the point cloud data or the coded data as it is to the outside.
  • the transmission signal (multiplexed data) output from the three-dimensional data coding system 4601 is input to the three-dimensional data decoding system 4602 via the external connection unit 4604.
  • the three-dimensional data decoding system 4602 generates point cloud data, which is three-dimensional data, by decoding encoded data or multiplexed data.
  • the three-dimensional data decoding system 4602 may be a three-dimensional data decoding device realized by a single device, or may be a system realized by a plurality of devices. Further, the three-dimensional data decoding device may include a part of a plurality of processing units included in the three-dimensional data decoding system 4602.
  • the three-dimensional data decoding system 4602 includes a sensor information acquisition unit 4621, an input / output unit 4622, a demultiplexing unit 4623, a decoding unit 4624, a presentation unit 4625, a user interface 4626, and a control unit 4627.
  • the sensor information acquisition unit 4621 acquires sensor information from the sensor terminal 4603.
  • the input / output unit 4622 acquires the transmission signal, decodes the multiplexed data (file format or packet) from the transmitted signal, and outputs the multiplexed data to the demultiplexed unit 4623.
  • the demultiplexing unit 4623 acquires encoded data, control information and additional information from the multiplexing data, and outputs the encoded data, control information and additional information to the decoding unit 4624.
  • the decoding unit 4624 reconstructs the point cloud data by decoding the coded data.
  • the presentation unit 4625 presents the point cloud data to the user. For example, the presentation unit 4625 displays information or an image based on the point cloud data.
  • the user interface 4626 acquires instructions based on user operations.
  • the control unit 4627 (or application execution unit) controls each processing unit. That is, the control unit 4627 controls demultiplexing, decoding, presentation, and the like.
  • the input / output unit 4622 may acquire the point cloud data or the coded data as it is from the outside. Further, the presentation unit 4625 may acquire additional information such as sensor information and present information based on the additional information. In addition, the presentation unit 4625 may make a presentation based on the user's instruction acquired by the user interface 4626.
  • the sensor terminal 4603 generates sensor information, which is information obtained by the sensor.
  • the sensor terminal 4603 is a terminal equipped with a sensor or a camera, and includes, for example, a moving object such as an automobile, a flying object such as an airplane, a mobile terminal, or a camera.
  • the sensor information that can be acquired by the sensor terminal 4603 is, for example, (1) the distance between the sensor terminal 4603 and the object obtained from the LIDAR, the millimeter-wave radar, or the infrared sensor, or the reflectance of the object, and (2) a plurality.
  • the sensor information may include the attitude, orientation, gyro (angular velocity), position (GPS information or altitude), speed, acceleration, and the like of the sensor.
  • the sensor information may include temperature, atmospheric pressure, humidity, magnetism, and the like.
  • the external connection unit 4604 is realized by communication with an integrated circuit (LSI or IC), an external storage unit, a cloud server via the Internet, broadcasting, or the like.
  • LSI integrated circuit
  • IC integrated circuit
  • cloud server via the Internet, broadcasting, or the like.
  • FIG. 2 is a diagram showing the structure of point cloud data.
  • FIG. 3 is a diagram showing a configuration example of a data file in which information on point cloud data is described.
  • the point cloud data includes data of a plurality of points.
  • the data of each point includes position information (three-dimensional coordinates) and attribute information for the position information.
  • a collection of multiple points is called a point cloud.
  • a point cloud indicates a three-dimensional shape of an object.
  • Position information such as three-dimensional coordinates is sometimes called geometry.
  • the data of each point may include attribute information (attribute) of a plurality of attribute types.
  • the attribute type is, for example, color or reflectance.
  • One attribute information may be associated with one position information, or attribute information having a plurality of different attribute types may be associated with one position information. Further, a plurality of attribute information of the same attribute type may be associated with one position information.
  • the configuration example of the data file shown in FIG. 3 is an example in which the position information and the attribute information have a one-to-one correspondence, and shows the position information and the attribute information of N points constituting the point cloud data. There is.
  • the position information is, for example, information on three axes of x, y, and z.
  • the attribute information is, for example, RGB color information.
  • a typical data file is a ply file or the like.
  • FIG. 4 is a diagram showing the types of point cloud data.
  • the point cloud data includes a static object and a dynamic object.
  • the static object is 3D point cloud data at an arbitrary time (certain time).
  • a dynamic object is three-dimensional point cloud data that changes over time.
  • the three-dimensional point cloud data at a certain time is referred to as a PCC frame or a frame.
  • the object may be a point cloud whose area is limited to some extent like ordinary video data, or a large-scale point cloud whose area is not limited such as map information.
  • the sensor information is acquired by various methods such as a distance sensor such as LIDAR or a range finder, a stereo camera, or a combination of a plurality of monocular cameras.
  • the point cloud data generation unit 4618 generates point cloud data based on the sensor information obtained by the sensor information acquisition unit 4617.
  • the point cloud data generation unit 4618 generates position information as point cloud data, and adds attribute information for the position information to the position information.
  • the point cloud data generation unit 4618 may process the point cloud data when generating the position information or adding the attribute information. For example, the point cloud data generation unit 4618 may reduce the amount of data by deleting the point clouds whose positions overlap. Further, the point cloud data generation unit 4618 may convert the position information (position shift, rotation, normalization, etc.), or may render the attribute information.
  • point cloud data generation system 4611 is included in the three-dimensional data coding system 4601 in FIG. 1, it may be provided independently outside the three-dimensional data coding system 4601.
  • the coding unit 4613 generates coded data by coding the point cloud data based on a predetermined coding method.
  • a predetermined coding method There are roughly the following two types of coding methods.
  • the first is a coding method using position information, and this coding method will be hereinafter referred to as a first coding method.
  • the second is a coding method using a video codec, and this coding method will be hereinafter referred to as a second coding method.
  • the decoding unit 4624 decodes the point cloud data by decoding the coded data based on a predetermined coding method.
  • the multiplexing unit 4614 generates multiplexed data by multiplexing the encoded data using an existing multiplexing method.
  • the generated multiplexed data is transmitted or accumulated.
  • the multiplexing unit 4614 multiplexes other media such as video, audio, subtitles, applications, and files, or reference time information. Further, the multiplexing unit 4614 may further multiplex the attribute information related to the sensor information or the point cloud data.
  • the multiplexing method or file format includes ISOBMFF, ISOBMFF-based transmission method MPEG-DASH, MMT, MPEG-2 TS Systems, RMP, and the like.
  • the demultiplexing unit 4623 extracts PCC coded data, other media, time information, etc. from the multiplexing data.
  • the input / output unit 4615 transmits the multiplexed data by using a method suitable for the medium to be transmitted or the medium to be stored, such as broadcasting or communication.
  • the input / output unit 4615 may communicate with other devices via the Internet, or may communicate with a storage unit such as a cloud server.
  • http http, ftp, TCP, UDP, etc. are used.
  • a PULL type communication method may be used, or a PUSH type communication method may be used.
  • Either wired transmission or wireless transmission may be used.
  • Ethernet registered trademark
  • USB registered trademark
  • RS-232C USB-232C
  • HDMI registered trademark
  • coaxial cable or the like
  • wireless transmission a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), millimeter wave, or the like is used.
  • DVB-T2 DVB-S2, DVB-C2, ATSC3.0, ISDB-S3 or the like is used.
  • FIG. 5 is a diagram showing the configuration of the first coding unit 4630, which is an example of the coding unit 4613 that encodes the first coding method.
  • FIG. 6 is a block diagram of the first coding unit 4630.
  • the first coding unit 4630 generates coded data (coded stream) by coding the point cloud data by the first coding method.
  • the first coding unit 4630 includes a position information coding unit 4631, an attribute information coding unit 4632, an additional information coding unit 4633, and a multiplexing unit 4634.
  • the first coding unit 4630 has a feature of performing coding while being aware of the three-dimensional structure. Further, the first coding unit 4630 has a feature that the attribute information coding unit 4632 performs coding using the information obtained from the position information coding unit 4631.
  • the first coding method is also called GPCC (Geometry based PCC).
  • the point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and is position information (Position), attribute information (Attribute), and other additional information (MetaData). including.
  • the position information is input to the position information coding unit 4631, the attribute information is input to the attribute information coding unit 4632, and the additional information is input to the additional information coding unit 4633.
  • the position information coding unit 4631 generates coded position information (Compressed Geometry) which is coded data by encoding the position information.
  • the position information coding unit 4631 encodes the position information using an N-branch structure such as an octa-tree. Specifically, in an ocree, the target space is divided into eight nodes (subspaces), and 8-bit information (occupancy code) indicating whether or not each node contains a point cloud is generated. .. Further, the node including the point cloud is further divided into eight nodes, and 8-bit information indicating whether or not the point cloud is included in each of the eight nodes is generated. This process is repeated until it becomes equal to or less than the threshold value of the number of point clouds included in the predetermined hierarchy or node.
  • the attribute information coding unit 4632 generates the coded attribute information (Compressed Attribute) which is the coded data by encoding using the configuration information generated by the position information coding unit 4631. For example, the attribute information coding unit 4632 determines a reference point (reference node) to be referred to in the coding of the target point (target node) to be processed based on the ocree tree structure generated by the position information coding unit 4631. To do. For example, the attribute information coding unit 4632 refers to a node whose parent node in the octree is the same as the target node among the peripheral nodes or adjacent nodes. The method of determining the reference relationship is not limited to this.
  • the attribute information coding process may include at least one of a quantization process, a prediction process, and an arithmetic coding process.
  • the reference means that the reference node is used to calculate the predicted value of the attribute information, or the state of the reference node (for example, occupancy indicating whether or not the reference node contains a point cloud) is used to determine the encoding parameter. Information) is used.
  • the coding parameter is a quantization parameter in the quantization process, a context in arithmetic coding, or the like.
  • the additional information coding unit 4633 generates the encoded additional information (Compressed Metadata Data) which is the encoded data by encoding the compressible data among the additional information.
  • Compressed Metadata Data is the encoded data by encoding the compressible data among the additional information.
  • the multiplexing unit 4634 generates a coded stream (Compressed Stream) which is coded data by multiplexing the coded position information, the coded attribute information, the coded additional information, and other additional information.
  • the generated coded stream is output to a processing unit of a system layer (not shown).
  • FIG. 7 is a diagram showing the configuration of the first decoding unit 4640.
  • FIG. 8 is a block diagram of the first decoding unit 4640.
  • the first decoding unit 4640 generates point cloud data by decoding the coded data (coded stream) encoded by the first coding method by the first coding method.
  • the first decoding unit 4640 includes a demultiplexing unit 4461, a position information decoding unit 4642, an attribute information decoding unit 4634, and an additional information decoding unit 4644.
  • a coded stream (Compressed Stream), which is coded data, is input to the first decoding unit 4640 from a processing unit of a system layer (not shown).
  • the demultiplexing unit 4641 separates the coded position information (Compressed Geometry), the coded attribute information (Compressed Attribute), the coded additional information (Compressed Metadata), and other additional information from the coded data.
  • the position information decoding unit 4642 generates position information by decoding the coded position information. For example, the position information decoding unit 4642 restores the position information of the point cloud represented by the three-dimensional coordinates from the coded position information represented by the N-branch structure such as the ocree.
  • the attribute information decoding unit 4643 decodes the coded attribute information based on the configuration information generated by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 determines a reference point (reference node) to be referred to in decoding the target point (target node) to be processed, based on the octave tree structure obtained by the position information decoding unit 4642. For example, the attribute information decoding unit 4643 refers to a node whose parent node in the octree is the same as the target node among the peripheral nodes or adjacent nodes. The method of determining the reference relationship is not limited to this.
  • the attribute information decoding process may include at least one of an inverse quantization process, a prediction process, and an arithmetic decoding process.
  • the reference means that the reference node is used to calculate the predicted value of the attribute information, or the state of the reference node (for example, occupancy information indicating whether or not the reference node contains a point cloud) is used to determine the decoding parameter. ) Is used.
  • the decoding parameter is a quantization parameter in the inverse quantization process, a context in arithmetic decoding, or the like.
  • the additional information decoding unit 4644 generates additional information by decoding the coded additional information. Further, the first decoding unit 4640 uses the additional information necessary for the decoding process of the position information and the attribute information at the time of decoding, and outputs the additional information necessary for the application to the outside.
  • FIG. 9 is a block diagram of the position information coding unit 2700 according to the present embodiment.
  • the position information coding unit 2700 includes an octane tree generation unit 2701, a geometric information calculation unit 2702, a coding table selection unit 2703, and an entropy coding unit 2704.
  • the ocree generation unit 2701 generates, for example, an ocree from the input position information, and generates an occupancy code for each node of the ocree.
  • the geometric information calculation unit 2702 acquires information indicating whether or not the adjacent node of the target node is an occupied node. For example, the geometric information calculation unit 2702 calculates the occupancy information of the adjacent node (information indicating whether or not the adjacent node is the occupancy node) from the occupancy code of the parent node to which the target node belongs. Further, the geometric information calculation unit 2702 may save the encoded nodes in a list and search for adjacent nodes in the list. The geometric information calculation unit 2702 may switch the adjacent node according to the position in the parent node of the target node.
  • the coding table selection unit 2703 selects a coding table to be used for entropy coding of the target node using the occupancy information of the adjacent node calculated by the geometric information calculation unit 2702. For example, the coded table selection unit 2703 may generate a bit string using the occupancy information of the adjacent node, and select the coded table of the index number generated from the bit string.
  • the entropy coding unit 2704 generates coded position information and metadata by performing entropy coding on the occupancy code of the target node using the coded table of the selected index number.
  • the entropy coding unit 2704 may add information indicating the selected coding table to the coding position information.
  • the position information (position data) is converted into an octree structure (octree tree formation) and then encoded.
  • the ocree tree structure consists of nodes and leaves. Each node has eight nodes or leaves, and each leaf has voxel (VXL) information.
  • FIG. 10 is a diagram showing a structural example of position information including a plurality of voxels.
  • FIG. 11 is a diagram showing an example in which the position information shown in FIG. 10 is converted into an octane tree structure.
  • the leaves 1, 2 and 3 represent the voxels VXL1, VXL2 and VXL3 shown in FIG. 10, respectively, and represent the VXL including the point cloud (hereinafter, effective VXL).
  • the node 1 corresponds to the entire space including the position information of FIG.
  • the entire space corresponding to node 1 is divided into eight nodes, and among the eight nodes, the node containing the valid VXL is further divided into eight nodes or leaves, and this process is repeated for the hierarchy of the tree structure.
  • each node corresponds to a subspace, and has information (occupancy code) indicating at which position the next node or leaf is held after division as node information.
  • the lowest block is set in the leaf, and the number of point groups included in the leaf is held as leaf information.
  • FIG. 12 is a block diagram of the position information decoding unit 2710 according to the present embodiment.
  • the position information decoding unit 2710 includes an octane tree generation unit 2711, a geometric information calculation unit 2712, a coding table selection unit 2713, and an entropy decoding unit 2714.
  • the ocree generation unit 2711 generates an ocree in a certain space (node) by using the header information or metadata of the bitstream. For example, the ocree generation unit 2711 generates a large space (root node) using the x-axis, y-axis, and z-axis directions of a certain space added to the header information, and uses that space as the x-axis.
  • An ocree is generated by generating eight small spaces A (nodes A0 to A7) by dividing each into two in the y-axis and z-axis directions. Further, nodes A0 to A7 are set in order as target nodes.
  • the geometric information calculation unit 2712 acquires occupancy information indicating whether or not the node adjacent to the target node is an occupancy node. For example, the geometric information calculation unit 2712 calculates the occupancy information of the adjacent node from the occupancy code of the parent node to which the target node belongs. Further, the geometric information calculation unit 2712 may save the decoded nodes in a list and search for adjacent nodes from the list. The geometric information calculation unit 2712 may switch adjacent nodes according to the position of the target node in the parent node.
  • the coding table selection unit 2713 selects a coding table (decoding table) to be used for entropy decoding of the target node using the occupancy information of the adjacent node calculated by the geometric information calculation unit 2712. For example, the coded table selection unit 2713 may generate a bit string using the occupancy information of the adjacent node and select the coded table of the index number generated from the bit string.
  • a coding table decoding table
  • the entropy decoding unit 2714 generates position information by entropy decoding the occupancy code of the target node using the selected coding table.
  • the entropy decoding unit 2714 may decode and acquire the information of the selected coding table from the bit stream, and entropy decode the occupancy code of the target node using the coding table indicated by the information. ..
  • FIG. 13 is a block diagram showing a configuration example of the attribute information coding unit A100.
  • the attribute information coding unit may include a plurality of coding units that execute different coding methods. For example, the attribute information coding unit may switch between the following two methods according to the use case.
  • the attribute information coding unit A100 includes a LoD attribute information coding unit A101 and a conversion attribute information coding unit A102.
  • the LoD attribute information coding unit A101 classifies each three-dimensional point into a plurality of layers using the position information of the three-dimensional points, predicts the attribute information of the three-dimensional points belonging to each layer, and encodes the predicted residual. To become.
  • each classified layer is referred to as LoD (Level of Detail).
  • the conversion attribute information coding unit A102 encodes the attribute information using RAHT (Region Adaptive Hierarchical Transfer). Specifically, the conversion attribute information coding unit A102 generates high-frequency components and low-frequency components of each layer by applying RAHT or Har conversion to each attribute information based on the position information of the three-dimensional points. Then, those values are encoded by using quantization, entropy coding, or the like.
  • RAHT Registered Adaptive Hierarchical Transfer
  • FIG. 14 is a block diagram showing a configuration example of the attribute information decoding unit A110.
  • the attribute information decoding unit may include a plurality of decoding units that execute different decoding methods. For example, the attribute information decoding unit may switch between the following two methods based on the information included in the header and metadata for decoding.
  • the attribute information decoding unit A110 includes a LoD attribute information decoding unit A111 and a conversion attribute information decoding unit A112.
  • the LoD attribute information decoding unit A111 classifies each three-dimensional point into a plurality of layers using the position information of the three-dimensional points, and decodes the attribute value while predicting the attribute information of the three-dimensional points belonging to each layer.
  • the conversion attribute information decoding unit A112 decodes the attribute information using RAHT (Region Adaptive Hierarchical Transfer). Specifically, the conversion attribute information decoding unit A112 applies the invoke RAHT or invoke Haar conversion to the high-frequency component and the low-frequency component of each attribute value based on the position information of the three-dimensional point to obtain the attribute value. Decrypt.
  • RAHT Restion Adaptive Hierarchical Transfer
  • FIG. 15 is a block diagram showing the configuration of the attribute information coding unit 3140, which is an example of the LoD attribute information coding unit A101.
  • the attribute information coding unit 3140 includes a LoD generation unit 3141, a surrounding search unit 3142, a prediction unit 3143, a prediction residual calculation unit 3144, a quantization unit 3145, an arithmetic coding unit 3146, and an inverse quantization unit. It includes 3147, a decoding value generation unit 3148, and a memory 3149.
  • LoD generation unit 3141 generates LoD using the position information of the three-dimensional point.
  • the surrounding search unit 3142 searches for neighboring three-dimensional points adjacent to each three-dimensional point by using the LoD generation result by the LoD generation unit 3141 and the distance information indicating the distance between each three-dimensional point.
  • the prediction unit 3143 generates a prediction value of the attribute information of the target three-dimensional point to be encoded.
  • the predicted residual calculation unit 3144 calculates (generates) the predicted residual of the predicted value of the attribute information generated by the prediction unit 3143.
  • the quantization unit 3145 quantizes the predicted residual of the attribute information calculated by the predicted residual calculation unit 3144.
  • the arithmetic coding unit 3146 arithmetically encodes the predicted residual after being quantized by the quantization unit 3145.
  • the arithmetic coding unit 3146 outputs a bit stream including the arithmetically coded predicted residual to, for example, a three-dimensional data decoding device.
  • the predicted residual may be binarized by, for example, the quantization unit 3145 before being arithmetically coded by the arithmetic coding unit 3146.
  • the arithmetic coding unit 3146 may initialize the coding table used for arithmetic coding before arithmetic coding.
  • the arithmetic coding unit 3146 may initialize the coding table used for arithmetic coding for each layer.
  • the arithmetic coding unit 3146 may include information indicating the position of the layer in which the coding table is initialized in the bit stream and output the information.
  • the inverse quantization unit 3147 dequantizes the predicted residual after being quantized by the quantization unit 3145.
  • the decoded value generation unit 3148 generates a decoded value by adding the predicted value of the attribute information generated by the prediction unit 3143 and the predicted residual after dequantization by the inverse quantization unit 3147.
  • the memory 3149 is a memory that stores the decoded value of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3148. For example, when generating a predicted value of a three-dimensional point that has not yet been encoded, the prediction unit 3143 generates the predicted value by using the decoded value of the attribute information of each three-dimensional point stored in the memory 3149. ..
  • FIG. 16 is a block diagram of the attribute information coding unit 6600, which is an example of the conversion attribute information coding unit A102.
  • the attribute information coding unit 6600 includes a sorting unit 6601, a Har conversion unit 6602, a quantization unit 6603, an inverse quantization unit 6604, an inverse Haar conversion unit 6605, a memory 6606, and an arithmetic coding unit 6607. Be prepared.
  • the sort unit 6601 generates a Morton code using the position information of the three-dimensional points, and sorts a plurality of three-dimensional points in the order of the Morton code.
  • the Har conversion unit 6602 generates a coding coefficient by applying the Har conversion to the attribute information.
  • the quantization unit 6603 quantizes the coding coefficient of the attribute information.
  • the dequantization unit 6604 dequantizes the coding coefficient after quantization.
  • the inverse Har conversion unit 6605 applies the inverse Har conversion to the coding coefficient.
  • the memory 6606 stores the values of the attribute information of the plurality of decoded three-dimensional points. For example, the attribute information of the decoded three-dimensional point stored in the memory 6606 may be used for the prediction of the unencoded three-dimensional point and the like.
  • the arithmetic coding unit 6607 calculates ZeroCnt from the coding coefficient after quantization, and arithmetically encodes ZeroCnt. In addition, the arithmetic coding unit 6607 arithmetically encodes the non-zero coding coefficient after quantization. The arithmetic coding unit 6607 may binarize the coding coefficient before arithmetic coding. Further, the arithmetic coding unit 6607 may generate and encode various header information.
  • FIG. 17 is a block diagram showing the configuration of the attribute information decoding unit 3150, which is an example of the LoD attribute information decoding unit A111.
  • the attribute information decoding unit 3150 includes a LoD generation unit 3151, a surrounding search unit 3152, a prediction unit 3153, an arithmetic decoding unit 3154, an inverse quantization unit 3155, a decoding value generation unit 3156, and a memory 3157. ..
  • the LoD generation unit 3151 generates LoD using the position information of the three-dimensional point decoded by the position information decoding unit (not shown in FIG. 17).
  • the surrounding search unit 3152 searches for neighboring three-dimensional points adjacent to each three-dimensional point by using the LoD generation result by the LoD generation unit 3151 and the distance information indicating the distance between each three-dimensional point.
  • the prediction unit 3153 generates a predicted value of the attribute information of the target three-dimensional point to be decoded.
  • the arithmetic decoding unit 3154 arithmetically decodes the predicted residual in the bit stream acquired from the attribute information coding unit 3140 shown in FIG.
  • the arithmetic decoding unit 3154 may initialize the decoding table used for arithmetic decoding.
  • the arithmetic decoding unit 3154 initializes the decoding table used for arithmetic decoding for the layer in which the arithmetic coding unit 3146 shown in FIG. 15 has been encoded.
  • the arithmetic decoding unit 3154 may initialize the decoding table used for arithmetic decoding for each layer. Further, the arithmetic decoding unit 3154 may initialize the decoding table based on the information included in the bit stream indicating the position of the layer in which the coding table is initialized.
  • the inverse quantization unit 3155 dequantizes the predicted residuals arithmetically decoded by the arithmetic decoding unit 3154.
  • the decoding value generation unit 3156 adds the prediction value generated by the prediction unit 3153 and the prediction residual after dequantization by the inverse quantization unit 3155 to generate a decoding value.
  • the decoded value generation unit 3156 outputs the decoded attribute information data to another device.
  • the memory 3157 is a memory that stores the decoded value of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3156. For example, when the prediction unit 3153 generates a predicted value of a three-dimensional point that has not been decoded yet, the prediction unit 3153 generates the predicted value by using the decoded value of the attribute information of each three-dimensional point stored in the memory 3157. ..
  • FIG. 18 is a block diagram of the attribute information decoding unit 6610, which is an example of the conversion attribute information decoding unit A112.
  • the attribute information decoding unit 6610 includes an arithmetic decoding unit 6611, an inverse quantization unit 6612, an inverse Har conversion unit 6613, and a memory 6614.
  • the arithmetic decoding unit 6611 arithmetically decodes ZeroCnt and the coding coefficient included in the bit stream.
  • the arithmetic decoding unit 6611 may decode various header information.
  • the dequantization unit 6612 dequantizes the arithmetically decoded coding coefficient.
  • the inverse Har conversion unit 6613 applies the inverse Har transformation to the coding coefficient after the inverse quantization.
  • the memory 6614 stores the values of the attribute information of the plurality of decoded three-dimensional points. For example, the attribute information of the decoded three-dimensional point stored in the memory 6614 may be used for predicting the undecoded three-dimensional point.
  • FIG. 19 is a diagram showing the configuration of the second coding unit 4650.
  • FIG. 20 is a block diagram of the second coding unit 4650.
  • the second coding unit 4650 generates coded data (coded stream) by coding the point cloud data by the second coding method.
  • the second coding unit 4650 includes an additional information generation unit 4651, a position image generation unit 4652, an attribute image generation unit 4653, a video coding unit 4654, an additional information coding unit 4655, and a multiplexing unit 4656. And include.
  • the second coding unit 4650 generates a position image and an attribute image by projecting the three-dimensional structure onto the two-dimensional image, and encodes the generated position image and the attribute image using an existing video coding method. It has the feature.
  • the second coding method is also called VPCC (Video based PCC).
  • the point cloud data is PCC point cloud data such as a PLY file, or PCC point cloud data generated from sensor information, and provides position information (Position), attribute information (Attribute), and other additional information (MetaData). Including.
  • the additional information generation unit 4651 generates map information of a plurality of two-dimensional images by projecting the three-dimensional structure onto the two-dimensional image.
  • the position image generation unit 4652 generates a position image (Geometry Image) based on the position information and the map information generated by the additional information generation unit 4651.
  • This position image is, for example, a distance image in which a distance (Dept) is shown as a pixel value.
  • this distance image may be an image in which a plurality of point clouds are viewed from one viewpoint (an image in which a plurality of point clouds are projected on one two-dimensional plane), or a plurality of point clouds from a plurality of viewpoints. It may be a plurality of images viewed, or it may be a single image in which these plurality of images are integrated.
  • the attribute image generation unit 4653 generates an attribute image based on the attribute information and the map information generated by the additional information generation unit 4651.
  • This attribute image is, for example, an image in which attribute information (for example, color (RGB)) is shown as a pixel value.
  • RGB color
  • this image may be an image in which a plurality of point clouds are viewed from one viewpoint (an image in which a plurality of point clouds are projected on one two-dimensional plane), or a plurality of point clouds may be viewed from a plurality of viewpoints. It may be a plurality of images viewed, or it may be a single image in which these plurality of images are integrated.
  • the video coding unit 4654 encodes the position image and the attribute image by using the video coding method, so that the coded position image (Compressed Geometry Image) and the coded attribute image (Compressed Attribute Image) which are the coded data are encoded. ) Is generated.
  • the video coding method any known coding method may be used.
  • the video coding method is AVC, HEVC, or the like.
  • the additional information coding unit 4655 generates the encoded additional information (Compressed Metadata Data) by encoding the additional information included in the point cloud data, the map information, and the like.
  • the multiplexing unit 4656 generates a coded stream (Compressed Stream) which is coded data by multiplexing the coded position image, the coded attribute image, the coded additional information, and other additional information.
  • the generated coded stream is output to a processing unit of a system layer (not shown).
  • FIG. 21 is a diagram showing a configuration of a second decoding unit 4660.
  • FIG. 22 is a block diagram of the second decoding unit 4660.
  • the second decoding unit 4660 generates point cloud data by decoding the coded data (coded stream) encoded by the second coding method by the second coding method.
  • the second decoding unit 4660 includes a demultiplexing unit 4661, a video decoding unit 4662, an additional information decoding unit 4663, a position information generation unit 4664, and an attribute information generation unit 4665.
  • a coded stream (Compressed Stream), which is coded data, is input to the second decoding unit 4660 from a processing unit of a system layer (not shown).
  • the demultiplexing unit 4661 separates the coded position image (Compressed Geometry Image), the coded attribute image (Compressed Attribute Image), the coded additional information (Compressed Metadata Data), and other additional information from the coded data. ..
  • the video decoding unit 4662 generates a position image and an attribute image by decoding the coded position image and the coded attribute image using a video coding method.
  • a video coding method any known coding method may be used.
  • the video coding method is AVC, HEVC, or the like.
  • the additional information decoding unit 4663 generates additional information including map information and the like by decoding the coded additional information.
  • the position information generation unit 4664 generates position information using the position image and the map information.
  • the attribute information generation unit 4665 generates attribute information using the attribute image and the map information.
  • the second decoding unit 4660 uses the additional information necessary for decoding at the time of decoding, and outputs the additional information necessary for the application to the outside.
  • FIG. 23 is a diagram showing a protocol stack related to PCC coded data.
  • FIG. 23 shows an example in which data of another medium such as video (for example, HEVC) or audio is multiplexed, transmitted or stored in PCC coded data.
  • video for example, HEVC
  • audio is multiplexed, transmitted or stored in PCC coded data.
  • the multiplexing method and file format have a function for multiplexing, transmitting or accumulating various coded data.
  • the coded data In order to transmit or store the coded data, the coded data must be converted to a multiplexing format.
  • HEVC defines a technique for storing coded data in a data structure called a NAL unit and storing the NAL unit in ISOBMFF.
  • a first coding method (Codec1) and a second coding method (Codec2) are being studied as a method for coding point group data.
  • the method of storing in the system format is not defined, and there is a problem that MUX processing (multiplexing), transmission and storage in the coding unit cannot be performed as it is.
  • the coded data (position information (Geometry), attribute information (Attribute), additional information (Metadata)) generated by the first coding unit 4630 or the second coding unit 4650 described above).
  • the type of the data, the method of generating additional information (metadata), and the multiplexing process in the multiplexing unit will be described.
  • the additional information (metadata) may be referred to as a parameter set or control information.
  • the dynamic object (three-dimensional point cloud data that changes with time) described in FIG. 4 will be described as an example, but even in the case of a static object (three-dimensional point cloud data at an arbitrary time). A similar method may be used.
  • FIG. 24 is a diagram showing a configuration of a coding unit 4801 and a multiplexing unit 4802 included in the three-dimensional data coding apparatus according to the present embodiment.
  • the coding unit 4801 corresponds to, for example, the first coding unit 4630 or the second coding unit 4650 described above.
  • the multiplexing unit 4802 corresponds to the multiplexing unit 4634 or 4656 described above.
  • the coding unit 4801 encodes the point cloud data of a plurality of PCC (Point Cloud Compression) frames and generates coded data (Multiple Compressed Data) of a plurality of position information, attribute information and additional information.
  • PCC Point Cloud Compression
  • the multiplexing unit 4802 converts the data of a plurality of data types (position information, attribute information, and additional information) into a NAL unit, thereby converting the data into a data structure in consideration of data access in the decoding device.
  • FIG. 25 is a diagram showing a configuration example of coded data generated by the coding unit 4801.
  • the arrow in the figure shows the dependency related to the decoding of the coded data, and the source of the arrow depends on the data at the tip of the arrow. That is, the decoding device decodes the data at the tip of the arrow, and uses the decoded data to decode the original data of the arrow.
  • "dependence" means that the dependent data is referenced (used) in the processing (encoding or decoding, etc.) of the dependent data.
  • the coding unit 4801 encodes the position information of each frame to generate coded position data (Compressed Geometry Data) for each frame.
  • the coded position data is represented by G (i). i indicates a frame number, a frame time, or the like.
  • the coding unit 4801 generates a position parameter set (GPS (i)) corresponding to each frame.
  • the position parameter set contains parameters that can be used to decode the coded position data. Also, the coded position data for each frame depends on the corresponding position parameter set.
  • the coded position data composed of a plurality of frames is defined as a position sequence (Geometric Sequence).
  • the coding unit 4801 generates a position sequence parameter set (also referred to as Geometric Sequence PS: position SPS) that stores parameters commonly used for decoding processing for a plurality of frames in the position sequence.
  • the position sequence depends on the position SPS.
  • the coding unit 4801 encodes the attribute information of each frame to generate the coded attribute data (Compressed Attribute Data) for each frame.
  • the coded attribute data is represented by A (i).
  • FIG. 25 shows an example in which the attribute X and the attribute Y exist, the coded attribute data of the attribute X is represented by AX (i), and the coded attribute data of the attribute Y is represented by AY (i). ..
  • the coding unit 4801 generates an attribute parameter set (APS (i)) corresponding to each frame.
  • the attribute parameter set of the attribute X is represented by AXPS (i)
  • the attribute parameter set of the attribute Y is represented by AYPS (i).
  • the attribute parameter set contains parameters that can be used to decode the coded attribute information.
  • the coded attribute data depends on the corresponding set of attribute parameters.
  • the coded attribute data consisting of a plurality of frames is defined as an attribute sequence (Attribute Sequence).
  • the coding unit 4801 generates an attribute sequence parameter set (Attribute Sequence PS: also referred to as attribute SPS) that stores parameters commonly used for decoding processing for a plurality of frames in the attribute sequence.
  • attribute sequence PS also referred to as attribute SPS
  • the attribute sequence depends on the attribute SPS.
  • the coding attribute data depends on the coding position data.
  • FIG. 25 shows an example in which two types of attribute information (attribute X and attribute Y) exist.
  • attribute information for example, the respective data and metadata are generated by the two coding units.
  • an attribute sequence is defined for each type of attribute information, and an attribute SPS is generated for each type of attribute information.
  • FIG. 25 shows an example in which the position information is one type and the attribute information is two types, but the present invention is not limited to this, and the attribute information may be one type or three or more types. Good.
  • the coded data can be generated by the same method.
  • the attribute information may not be present. In that case, the coding unit 4801 does not have to generate the parameter set related to the attribute information.
  • the coding unit 4801 generates a PCC stream PS (PCC Stream PS: also referred to as a stream PS), which is a parameter set for the entire PCC stream.
  • the coding unit 4801 stores in the stream PS parameters that can be commonly used in the decoding process for one or more position sequences and one or more attribute sequences.
  • the stream PS includes identification information indicating the codec of the point cloud data, information indicating the algorithm used for encoding, and the like.
  • the position sequence and attribute sequence depend on the stream PS.
  • An access unit is a basic unit for accessing data at the time of decryption, and is composed of one or more data and one or more metadata.
  • the access unit is composed of position information at the same time and one or more attribute information.
  • a GOF is a random access unit and is composed of one or more access units.
  • the coding unit 4801 generates an access unit header (AU Header) as identification information indicating the head of the access unit.
  • the coding unit 4801 stores the parameters related to the access unit in the access unit header.
  • the access unit header contains the structure or information of the coded data contained in the access unit.
  • the access unit header includes parameters commonly used for data included in the access unit, for example, parameters related to decoding of coded data.
  • the coding unit 4801 may generate an access unit delimiter that does not include parameters related to the access unit instead of the access unit header.
  • This access unit delimiter is used as identification information indicating the head of the access unit.
  • the decoding device identifies the head of the access unit by detecting the access unit header or the access unit delimiter.
  • the coding unit 4801 generates a GOF header (GOF Header) as identification information indicating the beginning of the GOF.
  • the coding unit 4801 stores the parameters related to the GOF in the GOF header.
  • the GOF header contains the structure or information of the coded data contained in the GOF.
  • the GOF header includes parameters commonly used for the data included in the GOF, for example, parameters related to decoding of the coded data.
  • the coding unit 4801 may generate a GOF delimiter that does not include the parameters related to the GOF instead of the GOF header.
  • This GOF delimiter is used as identification information indicating the beginning of the GOF.
  • the decoding device identifies the head of the GOF by detecting the GOF header or the GOF delimiter.
  • the access unit is defined as a PCC frame unit.
  • the decoding device accesses the PCC frame based on the identification information at the head of the access unit.
  • GOF is defined as one random access unit.
  • the decoding device accesses the random access unit based on the identification information at the head of the GOF. For example, if the PCC frames do not depend on each other and can be decoded independently, the PCC frame may be defined as a random access unit.
  • one access unit may be assigned two or more PCC frames, or one GOF may be assigned a plurality of random access units.
  • the coding unit 4801 may define and generate a parameter set or metadata other than the above.
  • the coding unit 4801 may generate SEI (Supplemental Information Information) that stores parameters (optional parameters) that may not necessarily be used at the time of decoding.
  • SEI Supplemental Information Information
  • FIG. 26 is a diagram showing an example of coded data and a NAL unit.
  • the coded data includes a header and a payload.
  • the coded data may include length information indicating the length (data amount) of the coded data, the header or the payload. Further, the coded data does not have to include a header.
  • the header contains, for example, identification information for identifying data.
  • This identification information indicates, for example, a data type or a frame number.
  • the header contains, for example, identification information indicating a reference relationship. This identification information is stored in the header when there is a dependency between the data, for example, and is information for referencing the reference destination from the reference source.
  • the referenced header contains identification information for identifying the data.
  • the header of the reference source includes identification information indicating the reference destination.
  • the identification information for specifying the data or the identification information indicating the reference relationship may be omitted.
  • the multiplexing unit 4802 stores the coded data in the payload of the NAL unit.
  • the NAL unit header includes pcc_nal_unit_type, which is identification information of the coded data.
  • FIG. 27 is a diagram showing an example of the semantics of pcc_nal_unit_type.
  • pcc_codec_type is codec 1 (Codec1: first coding method)
  • the values 0 to 10 of pcc_nar_unit_type are the coded position data (Geometry) and the coding attribute X data in codec 1.
  • HeaderX Codec Attribute Y Data (HeaderY), Position PS (Geom.PS), Attribute XPS (AttrX.PS), Attribute YPS (AttrX.PS), Position SPS (Geometry Sequence PS), Attribute XSPS (HeaderX) It is assigned to PS), attribute YSPS (AttributeY Position PS), AU header (AU Header), and GOF header (GOF Header). Further, the value 11 or later is assigned to the reserve of the codec 1.
  • pcc_codec_type is codec 2 (Codec 2: second coding method)
  • the values 0 to 2 of pcc_nal_unit_type are assigned to codec data A (DataA), metadata A (MetaDataA), and metadata B (MetaDataB). .. Further, the value 3 or later is assigned to the reserve of the codec 2.
  • the multiplexing unit 4802 collectively sends out the NAL unit in units of GOF or AU.
  • the multiplexing unit 4802 arranges the GOF header at the head of the GOF and the AU header at the head of the AU.
  • the multiplexing unit 4802 may arrange a sequence parameter set (SPS) for each AU so that the decoding device can decode from the next AU.
  • SPS sequence parameter set
  • the decoding device decodes the referenced data and then decodes the reference source data. In order to enable the decoding device to decode the data in the order in which they are received without rearranging the data, the multiplexing unit 4802 sends the referenced data first.
  • FIG. 28 is a diagram showing an example of the sending order of the NAL unit.
  • FIG. 28 shows three examples of location information priority, parameter priority, and data integration.
  • the location information priority transmission order is an example of transmitting information related to position information and information related to attribute information together. In the case of this transmission order, the transmission of the information regarding the position information is completed earlier than the transmission of the information regarding the attribute information.
  • a decoding device that does not decode the attribute information may be able to set a time for not processing by ignoring the decoding of the attribute information. Further, for example, in the case of a decoding device that wants to decode the position information quickly, there is a possibility that the position information can be decoded faster by obtaining the coded data of the position information earlier.
  • the attribute XSPS and the attribute YSPS are integrated and described as the attribute SPS, but the attribute XSPS and the attribute YSPS may be arranged separately.
  • the parameter set is transmitted first and the data is transmitted later.
  • the multiplexing unit 4802 may transmit the NAL units in any order.
  • the order identification information is defined, and the multiplexing unit 4802 may have a function of sending out NAL units in the order of a plurality of patterns.
  • the order identification information of the NAL unit is stored in the stream PS.
  • the three-dimensional data decoding device may perform decoding based on the order identification information.
  • a desired transmission order is instructed by the three-dimensional data decoding device to the three-dimensional data coding device, and the three-dimensional data coding device (multiplexing unit 4802) may control the transmission order according to the instructed transmission order.
  • the multiplexing unit 4802 may generate coded data in which a plurality of functions are merged as long as the transmission order is restricted, such as the transmission order of data integration.
  • the GOF header and the AU header may be integrated, or the AXIS and the AYPS may be integrated.
  • an identifier indicating that the data has a plurality of functions is defined in pcc_nal_unit_type.
  • the parameter storage method is The following method may be used.
  • the default PS value is indicated by the higher PS.
  • the value of PS is indicated by the lower PS.
  • the PS value is not described in the upper PS, and the PS value is described in the lower PS.
  • the information on whether the value of PS is indicated by the lower PS, the upper PS, or both is indicated in one or both of the lower PS and the upper PS.
  • the lower PS may be merged with the upper PS.
  • the multiplexing unit 4802 may omit the transmission of either one.
  • the coding unit 4801 or the multiplexing unit 4802 may divide the data into slices, tiles, or the like, and send out the divided data.
  • the divided data contains information for identifying the divided data, and the parameters used for decoding the divided data are included in the parameter set.
  • pcc_nal_unit_type an identifier indicating that the data or parameters related to the tile or slice are stored is defined.
  • FIG. 29 is a flowchart of processing by the three-dimensional data coding apparatus (encoding unit 4801 and multiplexing unit 4802) relating to the transmission order of the NAL unit.
  • the three-dimensional data encoding device determines the transmission order (position information priority or parameter set priority) of the NAL unit (S4801). For example, the three-dimensional data coding device determines the transmission order based on the designation from the user or an external device (for example, a three-dimensional data decoding device).
  • the three-dimensional data encoding device sets the order identification information included in the stream PS to position information priority (S4803). That is, in this case, the order identification information indicates that the NAL units are sent in the order in which the position information is prioritized. Then, the three-dimensional data coding device sends out the NAL units in the order of prioritizing the position information (S4804).
  • the three-dimensional data encoding device sets the order identification information included in the stream PS to parameter set priority (S4805). That is, in this case, the order identification information indicates that the NAL units are sent in the order in which the parameter set is prioritized. Then, the three-dimensional data coding apparatus sends out NAL units in the order of parameter set parameter set priority (S4806).
  • FIG. 30 is a flowchart of processing by the three-dimensional data decoding device related to the transmission order of the NAL unit.
  • the three-dimensional data decoding device analyzes the order identification information included in the stream PS (S4811).
  • the three-dimensional data decoding device decodes the NAL unit assuming that the transmission order of the NAL unit is position information priority (S4812). S4813).
  • the three-dimensional data decoding device decodes the NAL unit assuming that the transmission order of the NAL unit has parameter set priority. (S4814).
  • step S4813 the NAL unit related to the position information is acquired without acquiring all the NAL units, and the position information is decoded from the acquired NAL unit. May be good.
  • FIG. 31 is a flowchart of processing by a three-dimensional data coding device (multiplexing unit 4802) related to AU and GOF generation in multiplexing of NAL units.
  • the three-dimensional data coding device determines the type of coded data (S4821). Specifically, the three-dimensional data coding apparatus determines whether the coded data to be processed is the data at the head of the AU, the data at the head of the GOF, or other data.
  • the three-dimensional data coding device arranges the GOF header and the AU header at the head of the coded data belonging to the GOF to generate a NAL unit ( S4823).
  • the three-dimensional data coding device arranges the AU header at the head of the coded data belonging to the AU to generate the NAL unit (S4824).
  • the three-dimensional data coding device places the coded data after the AU header of the AU to which the coded data belongs. To generate a NAL unit (S4825).
  • FIG. 32 is a flowchart of the processing of the three-dimensional data decoding device related to the access of the AU and GOF in the demultiplexing of the NAL unit.
  • the three-dimensional data decoding device determines the type of coded data included in the NAL unit by analyzing the nal_unit_type included in the NAL unit (S4831). Specifically, the three-dimensional data decoding device determines whether the coded data included in the NAL unit is the data at the head of the AU, the data at the head of the GOF, or other data.
  • the three-dimensional data decoding device determines that the NAL unit is the start position of random access and accesses the NAL unit. Then, the decoding process is started (S4833).
  • the three-dimensional data decoding device determines that the NAL unit is the AU head, and the data included in the NAL unit. To decode the AU (S4834).
  • the three-dimensional data decoding device does not process the NAL unit.
  • FIG. 33 is a block diagram showing a configuration of a three-dimensional data distribution system according to the present embodiment.
  • the distribution system shown in FIG. 33 includes a server 1501 and a plurality of clients 1502.
  • the server 1501 includes a storage unit 1511 and a control unit 1512.
  • the storage unit 1511 stores a coded three-dimensional map 1513 which is coded three-dimensional data.
  • FIG. 34 is a diagram showing a configuration example of a bit stream of the coded three-dimensional map 1513.
  • the three-dimensional map is divided into a plurality of submaps (sub-maps), and each submap is encoded.
  • a random access header (RA) containing sub-coordinate information is added to each sub-map.
  • the sub-coordinate information is used to improve the coding efficiency of the sub-map.
  • This sub-coordinate information indicates the sub-coordinate (sub-coordinate) of the sub-map.
  • the sub-coordinates are the coordinates of the sub-map with reference to the reference coordinates.
  • a three-dimensional map including a plurality of submaps is called an overall map. Further, the reference coordinates (for example, the origin) in the entire map are called reference coordinates.
  • the sub-coordinates are the coordinates of the sub-map in the coordinate system of the whole map.
  • the sub-coordinates indicate the offset between the coordinate system of the entire map and the coordinate system of the sub-map.
  • the coordinates in the coordinate system of the whole map based on the reference coordinates are called the whole coordinates.
  • the coordinates in the coordinate system of the submap based on the subcoordinates are called the difference coordinates.
  • Client 1502 sends a message to server 1501.
  • This message includes the location information of client 1502.
  • the control unit 1512 included in the server 1501 acquires a bitstream of the submap of the position closest to the position of the client 1502 based on the position information included in the received message.
  • the bitstream of the submap contains the subcoordinate information and is transmitted to the client 1502.
  • the decoder 1521 included in the client 1502 uses this sub-coordinate information to obtain the overall coordinates of the sub-map with reference to the reference coordinates.
  • the application 1522 included in the client 1502 executes the application related to the self-position using the whole coordinates of the obtained submap.
  • the submap shows a part of the whole map.
  • the sub-coordinates are the coordinates where the sub-map is located in the reference coordinate space of the entire map.
  • a submap A of AA and a submap B of AB exist in the entire map of A.
  • the vehicle wants to refer to the map of AA, it starts decoding from submap A, and when it wants to refer to the map of AB, it starts decoding from submap B.
  • the submap is a random access point.
  • A is Osaka Prefecture
  • AA Osaka City
  • AB Takatsuki City.
  • Each submap is sent to the client along with the subcoordinate information.
  • the sub-coordinate information is included in the header information of each sub-map, the transmission packet, and the like.
  • the reference coordinates which are the reference coordinates of the sub-coordinate information of each sub-map, may be added to the header information of the space higher than the sub-map, such as the header information of the entire map.
  • the submap may consist of one space (SPC). Further, the submap may be composed of a plurality of SPCs.
  • the submap may include GOS (Group of Space).
  • the submap may be composed of worlds. For example, when there are a plurality of objects in a submap, if the plurality of objects are assigned to different SPCs, the submap is composed of a plurality of SPCs. If a plurality of objects are assigned to one SPC, the submap is composed of one SPC.
  • FIG. 35 is a diagram for explaining this effect.
  • the coding efficiency can be improved by coding the coordinates of the three-dimensional point A with reference to the sub-coordinates, as compared with the case of coding the coordinates of the three-dimensional point A with reference to the reference coordinates.
  • the bitstream of the submap contains subcoordinate information. By sending the bitstream of the submap and the reference coordinates to the decoding side (client), the decoding side can restore the entire coordinates of the submap.
  • FIG. 36 is a flowchart of processing by the server 1501 which is the transmitting side of the submap.
  • the server 1501 receives a message including the location information of the client 1502 from the client 1502 (S1501).
  • the control unit 1512 acquires a coded bit stream of the submap based on the position information of the client from the storage unit 1511 (S1502).
  • the server 1501 transmits the coded bit stream of the submap and the reference coordinates to the client 1502 (S1503).
  • FIG. 37 is a flowchart of processing by the client 1502, which is the receiving side of the submap.
  • the client 1502 receives the coded bit stream of the submap transmitted from the server 1501 and the reference coordinates (S1511). Next, the client 1502 acquires the submap and the subcoordinate information by decoding the coded bit stream (S1512). Next, the client 1502 restores the difference coordinates in the submap to the overall coordinates using the reference coordinates and the subcoordinates (S1513).
  • the three-dimensional data coding device calculates the difference coordinates by subtracting the sub-coordinates from the coordinates of each point cloud (three-dimensional point). Then, the three-dimensional data encoding device encodes the difference coordinates into a bit stream as the value of each point cloud. Further, the coding device encodes the sub-coordinate information indicating the sub-coordinate as the header information of the bit stream. As a result, the three-dimensional data decoding device can obtain the overall coordinates of each point cloud. For example, the 3D data encoding device is included in the server 1501 and the 3D data decoding device is included in the client 1502.
  • FIG. 38 is a diagram showing an example of syntax of the submap.
  • NuMofPoint shown in FIG. 38 indicates the number of point clouds included in the submap.
  • sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are sub-coordinate information.
  • sub_coordinate_x indicates the x-coordinate of the sub-coordinate.
  • sub_coordinate_y indicates the y-coordinate of the sub-coordinate.
  • sub_coordinate_z indicates the z-coordinate of the sub-coordinate.
  • diff_x [i], diff_y [i], and diff_z [i] are the difference coordinates of the i-th point cloud in the submap.
  • diff_x [i] indicates the difference value between the x-coordinate of the i-th point cloud in the submap and the x-coordinate of the sub-coordinate.
  • diff_y [i] indicates the difference value between the y-coordinate of the i-th point cloud in the sub-map and the y-coordinate of the sub-coordinate.
  • diff_z [i] indicates the difference value between the z-coordinate of the i-th point cloud in the submap and the z-coordinate of the sub-coordinate.
  • the three-dimensional data decoding device decodes point_clud [i] _x, point_clud [i] _y, and point_clud [i] _z, which are the overall coordinates of the i-th point cloud, using the following equations.
  • point_cloud [i] _x is the x-coordinate of the entire coordinate of the i-th point cloud.
  • point_cloud [i] _y is the y-coordinate of the entire coordinate of the i-th point cloud.
  • point_cloud [i] _z is the z-coordinate of the entire coordinate of the i-th point cloud.
  • the three-dimensional data coding device uses or encodes each point cloud in an ocree representation (hereinafter referred to as ocree coding) at the time of submap coding, or from subcoordinates. Select whether to use encoding of the difference value (hereinafter referred to as non-octree coding).
  • FIG. 39 is a diagram schematically showing this operation.
  • a three-dimensional data coding device applies ocree coding to a submap when the number of point clouds in the submap is greater than or equal to a predetermined threshold.
  • the three-dimensional data coding device applies non-octree coding to the submap when the number of point clouds in the submap is smaller than the above threshold.
  • the 3D data coding device can appropriately select whether to use octree coding or non-octree coding according to the shape and density of the objects contained in the submap. The coding efficiency can be improved.
  • the three-dimensional data coding device provides information indicating whether the octree coding or the non-octree coding is applied to the submap (hereinafter, referred to as octree coding application information). Add to the header of the submap.
  • the three-dimensional data decoding device is obtained by either the bitstream is a bitstream obtained by octree-encoding the submap, or the submap is non-octree-encoded. It can be determined whether it is a bitstream.
  • the three-dimensional data coding device calculates the coding efficiency when each of the octree coding and the non-octree coding is applied to the same point cloud, and is a coding method having good coding efficiency. May be applied to the submap.
  • FIG. 40 is a diagram showing an example of syntax of the submap when this switching is performed.
  • the coding_type shown in FIG. 40 is information indicating the coding type, and is the above-mentioned octane coding application information.
  • the submap includes NuOfPoint and sub-coordinate information (sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z).
  • the submap includes octree_info.
  • the octree_info is information necessary for octane coding, and includes, for example, depth information.
  • the submap includes the difference coordinates (diff_x [i], diff_y [i], and diff_z [i]).
  • the submap includes octree_data which is the coding data related to ocree coding.
  • a polar coordinate system may be used.
  • FIG. 41 is a flowchart of the three-dimensional data coding process by the three-dimensional data coding device.
  • the three-dimensional data coding device calculates the number of point clouds in the target submap, which is the submap to be processed (S1521).
  • the three-dimensional data coding device determines whether or not the calculated number of point clouds is equal to or greater than a predetermined threshold value (S1522).
  • the three-dimensional data coding device applies octree coding to the target submap (S1523). Further, the three-dimensional point data coding apparatus adds the octree coding application information indicating that the octree coding has been applied to the target submap to the header of the bitstream (S1525).
  • the three-dimensional data coding device applies non-octree coding to the target submap (S1524).
  • the three-dimensional point data coding apparatus adds octree coding application information indicating that non-octree coding has been applied to the target submap to the header of the bitstream (S1525).
  • FIG. 42 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device.
  • the three-dimensional data decoding device decodes the octane coding application information from the header of the bit stream (S1531).
  • the three-dimensional data decoding device determines whether or not the coding type applied to the target submap is octree coding based on the decoded octree coding application information (S1532).
  • the three-dimensional data decoding device decodes the target submap by ocree decoding (S1533).
  • the coding type indicated by the octree coding application information is non-octree coding (No in S1532)
  • the three-dimensional data decoding device decodes the target submap by non-octree decoding. (S1534).
  • 43 to 45 are diagrams schematically showing the operation of a modified example of the coding type switching process.
  • the three-dimensional data coding apparatus may select whether to apply octree coding or non-octree coding for each space.
  • the three-dimensional data coding device adds the octane coding application information to the header of the space.
  • the three-dimensional data decoding device can determine for each space whether or not the octree coding has been applied.
  • the three-dimensional data encoding device sets the sub-coordinates for each space and encodes the difference value obtained by subtracting the sub-coordinate value from the coordinates of each point cloud in the space.
  • the three-dimensional data encoding device can appropriately switch whether or not to apply the octree coding according to the shape of the object in the space or the number of point clouds, so that the coding efficiency can be improved. ..
  • the three-dimensional data coding apparatus may select whether to apply octree coding or non-octree coding for each volume.
  • the three-dimensional data coding device adds the ocree coding application information to the header of the volume.
  • the three-dimensional data decoding device can determine for each volume whether or not the octree coding has been applied.
  • the three-dimensional data coding device sets the sub-coordinates for each volume and encodes the difference value obtained by subtracting the sub-coordinate value from the coordinates of each point cloud in the volume.
  • the three-dimensional data coding device can appropriately switch whether or not to apply the octree coding according to the shape of the object in the volume or the number of point clouds, so that the coding efficiency can be improved. ..
  • non-octree coding an example of coding the difference obtained by subtracting the sub-coordinates from the coordinates of each point cloud is shown, but the present invention is not limited to this, and any device other than the octa-tree coding is used. It may be encoded by such an encoding method.
  • the 3D data coding device encodes the value of the point cloud itself in the submap, space, or volume as non-octree coding, not the difference from the subcoordinates. (Hereinafter, referred to as original coordinate coding) may be used.
  • the 3D data coding device stores in the header information indicating that the original coordinate coding has been applied to the target space (submap, space, or volume). As a result, the three-dimensional data decoding device can determine whether or not the original coordinate coding has been applied to the target space.
  • the three-dimensional data coding device may perform the coding without applying the quantization and arithmetic coding to the original coordinates. Further, the three-dimensional data coding apparatus may encode the original coordinates with a predetermined fixed bit length. As a result, the three-dimensional data coding device can generate a stream having a constant bit length at a certain timing.
  • non-octree coding an example of coding the difference obtained by subtracting the sub-coordinates from the coordinates of each point cloud is shown, but this is not always the case.
  • the three-dimensional data coding device may encode the difference values between the coordinates of each point cloud in order.
  • FIG. 46 is a diagram for explaining the operation in this case.
  • the three-dimensional data encoding device uses the sub-coordinates as the predicted coordinates when encoding the point cloud PA, and encodes the difference value between the coordinates of the point cloud PA and the predicted coordinates. To do. Further, the three-dimensional data encoding device uses the coordinates of the point cloud PA as the predicted coordinates when encoding the point cloud PB, and encodes the difference value between the point cloud PB and the predicted coordinates.
  • the three-dimensional data encoding device uses the point cloud PB as the predicted coordinates when encoding the point cloud PC, and encodes the difference value between the point cloud PB and the predicted coordinates. In this way, the three-dimensional data encoding device sets the scan order for a plurality of point clouds, and the coordinates of the target point cloud to be processed and the coordinates of the point cloud immediately before in the scan order with respect to the target point cloud are set.
  • the difference value may be encoded.
  • the sub-coordinates are the coordinates of the lower left front corner of the sub-map, but the position of the sub-coordinates is not limited to this.
  • 47 to 49 are diagrams showing another example of the position of the sub coordinate.
  • the setting position of the sub-coordinate may be set to any coordinate in the target space (sub-map, space, or volume). That is, as described above, the sub-coordinates may be the coordinates of the lower left front corner of the target space. As shown in FIG. 47, the sub-coordinates may be the coordinates of the center of the target space. As shown in FIG. 48, the sub-coordinates may be the coordinates of the upper right corner of the target space. Further, the sub-coordinates are not limited to the coordinates of the lower left front corner or the upper right back corner of the target space, and may be the coordinates of any corner of the target space.
  • the setting position of the sub-coordinates may be the same as the coordinates of a certain point cloud in the target space (sub-map, space, or volume).
  • the coordinates of the sub-coordinates match the coordinates of the point cloud PD.
  • the three-dimensional data coding apparatus may switch between applying another tree structure other than the octree tree and applying a non-tree structure other than the tree structure.
  • another tree structure is a kd tree that divides using a plane perpendicular to one of the coordinate axes. Any method may be used as another tree structure.
  • the three-dimensional data coding apparatus may encode, for example, color information, three-dimensional feature amounts, visible light feature amounts, and the like in the same manner as coordinate information.
  • the three-dimensional data encoding device sets the average value of the color information of each point cloud in the submap to the sub color information (sub-color), and the difference between the color information of each point cloud and the sub color information. May be encoded.
  • a server-side 3D data encoder is encoded by a point cloud bitstream encoded by octree coding, a point cloud bitstream encoded by non-octree coding, or both. It is possible to hold the converted point cloud bit stream and switch the bit stream to be transmitted to the three-dimensional data decoding device according to the communication environment or the processing capacity of the three-dimensional data decoding device.
  • FIG. 50 is a diagram showing an example of volume syntax when switching the application of ocree coding.
  • the syntax shown in FIG. 50 is basically the same as the syntax shown in FIG. 40, except that each piece of information is volume unit information.
  • NuMofPoint indicates the number of point clouds included in the volume.
  • sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are sub-coordinate information of the volume.
  • diff_x [i], diff_y [i], and diff_z [i] are the difference coordinates of the i-th point cloud in the volume.
  • diff_x [i] indicates the difference value between the x-coordinate of the i-th point cloud in the volume and the x-coordinate of the sub-coordinate.
  • diff_y [i] indicates the difference value between the y-coordinate of the i-th point cloud in the volume and the y-coordinate of the sub-coordinate.
  • diff_z [i] indicates the difference value between the z-coordinate of the i-th point cloud in the volume and the z-coordinate of the sub-coordinate.
  • the three-dimensional data encoding device does not have to include the sub-coordinate information in the header of the volume. That is, the three-dimensional data encoding device may calculate the relative position of the volume in the space without including the sub-coordinate information in the header, and use the calculated position as the sub-coordinate of each volume.
  • the three-dimensional data coding apparatus has an octree structure for the target spatial unit among a plurality of spatial units (for example, submap, space or volume) included in the three-dimensional data. It is determined whether or not to encode (for example, S1522 in FIG. 41). For example, when the number of three-dimensional points included in the target space unit is larger than a predetermined threshold value, the three-dimensional data coding device determines that the target space unit is encoded by an ocree tree structure. Further, when the number of three-dimensional points included in the target space unit is equal to or less than the above threshold value, the three-dimensional data coding device determines that the target space unit is not encoded by the octree structure.
  • the three-dimensional data encoding device encodes the target space unit by using the octane tree structure (S1523). Further, when it is determined that the target space unit is not encoded by the octa-tree structure (No in S1522), the three-dimensional data encoding device encodes the target space unit by a method different from that of the octa-tree structure (No). S1524). For example, a three-dimensional data coding device encodes the coordinates of a three-dimensional point included in a target space unit in a different method. Specifically, the three-dimensional data encoding device encodes the difference between the reference coordinates of the target space unit and the coordinates of the three-dimensional points included in the target space unit in different methods.
  • the three-dimensional data encoding device adds information indicating whether or not the target space unit is encoded by the octree structure to the bit stream (S1525).
  • the three-dimensional data coding device can reduce the amount of data of the coded signal, so that the coding efficiency can be improved.
  • the three-dimensional data encoding device includes a processor and a memory, and the processor uses the memory to perform the above processing.
  • the three-dimensional data decoding apparatus decodes the target space unit among a plurality of target space units (for example, submap, space or volume) included in the three-dimensional data in an octree structure. Decoding the information indicating whether or not from the bit stream (for example, S1531 in FIG. 42). When it is indicated by the above information that the target space unit is decoded by the octree structure (Yes in S1532), the three-dimensional data decoding apparatus decodes the target space unit by using the octree structure (S1533).
  • the three-dimensional data decoding apparatus decodes the target space unit in a method different from that in the octaree structure (S1534). ..
  • a three-dimensional data decoding device decodes the coordinates of a three-dimensional point included in a target space unit in a different method.
  • the three-dimensional data decoding device decodes the difference between the reference coordinates of the target space unit and the coordinates of the three-dimensional points included in the target space unit in different methods.
  • the three-dimensional data decoding device can reduce the amount of data of the coded signal, so that the coding efficiency can be improved.
  • the three-dimensional data decoding device includes a processor and a memory, and the processor uses the memory to perform the above processing.
  • the information of the three-dimensional point cloud includes position information (geometry) and attribute information (attribute).
  • the position information includes coordinates (x-coordinate, y-coordinate, z-coordinate) with respect to a certain point.
  • the attribute information includes information indicating the color information (RGB, YUV, etc.), reflectance, normal vector, etc. of each three-dimensional point.
  • the three-dimensional data coding device can encode the attribute information by using a coding method different from the position information.
  • an integer value will be used as the value of the attribute information.
  • each color component of the color information RGB or YUV has an 8-bit accuracy
  • each color component takes an integer value from 0 to 255.
  • the reflectance value has a precision of 10 bits
  • the reflectance value takes an integer value from 0 to 1023.
  • the three-dimensional data encoding device may multiply the value by a scale value and then round it to an integer value so that the value of the attribute information becomes an integer value. ..
  • the three-dimensional data coding device may add this scale value to the header of the bit stream or the like.
  • the code amount can be reduced by entropy-coding the difference absolute value Diffp using a coding table in which the number of generated bits decreases as the value becomes smaller.
  • the reference three-dimensional point is a three-dimensional point within a predetermined distance range from the target three-dimensional point.
  • the three-dimensional data encoding device determines that the position of the three-dimensional point q is close to the position of the target three-dimensional point p, and determines that the target three-dimensional point p is close to the position. It is determined that the value of the attribute information of the three-dimensional point q is used to generate the predicted value of the attribute information of the point p.
  • the distance calculation method may be another method, for example, Mahalanobis distance or the like may be used. Further, the three-dimensional data coding apparatus may determine that the three-dimensional point outside the predetermined distance range from the target three-dimensional point is not used for the prediction process.
  • the three-dimensional data encoding device predicts the three-dimensional point r. It may be determined that it is not used for.
  • the three-dimensional data encoding device may add information indicating the threshold value THd to the header of the bit stream or the like.
  • FIG. 51 is a diagram showing an example of a three-dimensional point.
  • the distance d (p, q) between the target three-dimensional point p and the three-dimensional point q is smaller than the threshold THd. Therefore, the three-dimensional data encoding device determines that the three-dimensional point q is the reference three-dimensional point of the target three-dimensional point p, and the attribute of the three-dimensional point q is used to generate the predicted value Pp of the attribute information Ap of the target three-dimensional p. It is determined that the value of the information Aq is used.
  • the three-dimensional data encoding device determines that the three-dimensional point r is not the reference three-dimensional point of the target three-dimensional point p, and generates the predicted value Pp of the attribute information Ap of the target three-dimensional point p. It is determined that the value of the attribute information Ar of is not used.
  • the three-dimensional data coding device encodes the attribute information of the target three-dimensional point using the predicted value
  • the three-dimensional point whose attribute information has already been encoded and decoded is used as the reference three-dimensional point.
  • the three-dimensional data decoding device uses the three-dimensional point whose attribute information has already been decoded as the reference three-dimensional point.
  • each classified layer is referred to as LoD (Level of Detail).
  • LoD Level of Detail
  • the three-dimensional data coding device selects the initial point a0 and assigns it to LoD0.
  • the three-dimensional data coding device extracts the point a1 whose distance from the point a0 is larger than the threshold Thres_LoD [0] of LoD0 and assigns it to LoD0.
  • the three-dimensional data encoding device extracts the point a2 whose distance from the point a1 is larger than the threshold Thres_LoD [0] of LoD0 and assigns it to LoD0.
  • the three-dimensional data coding device configures LoD0 so that the distance between each point in LoD0 is larger than the threshold value Thres_LoD [0].
  • the three-dimensional data coding device selects the point b0 to which LoD has not yet been assigned and assigns it to LoD1.
  • the three-dimensional data coding device extracts the unassigned point b1 whose distance from the point b0 is larger than the threshold Thres_LoD [1] of LoD1 and assigns it to LoD1.
  • the three-dimensional data coding device extracts the unassigned point b2 whose distance from the point b1 is larger than the threshold Thres_LoD [1] of LoD1 and assigns it to LoD1. In this way, the three-dimensional data coding device configures LoD1 so that the distance between each point in LoD1 is larger than the threshold value Thres_LoD [1].
  • the three-dimensional data coding device selects the point c0 to which LoD has not yet been assigned and assigns it to LoD2.
  • the three-dimensional data coding device extracts the unassigned point c1 whose distance from the point c0 is larger than the threshold Thres_LoD [2] of LoD2 and assigns it to LoD2.
  • the three-dimensional data coding device extracts the unassigned point c2 whose distance from the point c1 is larger than the threshold Thres_LoD [2] of LoD2 and assigns it to LoD2.
  • the three-dimensional data coding device configures LoD2 so that the distance between each point in LoD2 is larger than the threshold value Thres_LoD [2].
  • the thresholds Thres_LoD [0], Thres_LoD [1], and Thres_LoD [2] of each LoD are set.
  • the three-dimensional data coding device may add information indicating the threshold value of each LoD to the header of the bit stream or the like. For example, in the case of the example shown in FIG. 53, the three-dimensional data coding apparatus may add thresholds Thres_LoD [0], Thres_LoD [1], and Thres_LoD [2] to the header.
  • the three-dimensional data coding device may allocate all three-dimensional points to which LoD is not assigned to the lowest layer of LoD. In this case, the three-dimensional data coding apparatus can reduce the code amount of the header by not adding the threshold value of the lowest layer of LoD to the header. For example, in the case of the example shown in FIG. 53, the three-dimensional data coding apparatus adds the thresholds Thres_LoD [0] and Thres_LoD [1] to the header, and does not add Thres_LoD [2] to the header. In this case, the three-dimensional data decoding device may estimate the value of Thres_LoD [2] to be 0. Further, the three-dimensional data coding device may add the number of layers of LoD to the header. As a result, the three-dimensional data decoding device can determine the LoD of the lowest layer using the number of layers of LoD.
  • the higher layer (the layer closer to LoD0) becomes a sparse point cloud (space) in which the distance between the three-dimensional points is separated.
  • the lower layer is a dense point cloud (dense) in which the distance between three-dimensional points is closer.
  • LoD0 is the uppermost layer.
  • the method of selecting the initial three-dimensional points when setting each LoD may depend on the coding order at the time of position information coding. For example, the three-dimensional data coding device selects the first encoded three-dimensional point at the time of position information coding as the initial point a0 of LoD0, and selects the points a1 and a2 with the initial point a0 as the base point. LoD0 is configured. Then, the three-dimensional data coding apparatus may select the earliest three-dimensional point whose position information is encoded among the three-dimensional points that do not belong to LoD0 as the initial point b0 of LoD1.
  • the three-dimensional data coding device sets the initial point n0 of LoDn as the earliest three-dimensional point in which the position information is encoded among the three-dimensional points that do not belong to the upper layer (LoD0 to LoDn-1) of LoDn. May be selected.
  • the three-dimensional data decoding device can configure the same LoD as at the time of coding by using the same initial point selection method at the time of decoding, so that the bit stream can be appropriately decoded.
  • the three-dimensional data decoding apparatus selects, as the initial point n0 of LoDn, the three-dimensional point whose position information is decoded earliest among the three-dimensional points that do not belong to the upper layer of LoDn.
  • the three-dimensional data encoding device encodes the three-dimensional points included in LoD0 in order
  • the target three-dimensional points included in LoD1 are encoded and decoded included in LoD0 and LoD1 (hereinafter, simply "" It is generated using the attribute information of "encoded”).
  • the three-dimensional data encoding device sets the predicted value of the attribute information of the three-dimensional point to N or less of the encoded three-dimensional points around the target three-dimensional point to be encoded. Generated by calculating the average of attribute values. Further, the three-dimensional data coding device may add a value of N to the header of the bit stream or the like. The three-dimensional data coding apparatus may change the value of N for each three-dimensional point and add the value of N for each three-dimensional point. As a result, an appropriate N can be selected for each three-dimensional point, so that the accuracy of the predicted value can be improved. Therefore, the predicted residual can be reduced.
  • the three-dimensional data encoding device may add the value of N to the header of the bitstream and fix the value of N in the bitstream. This eliminates the need to encode or decode the value of N for each three-dimensional point, so that the amount of processing can be reduced. Further, the three-dimensional data coding apparatus may separately encode the value of N for each LoD. As a result, the coding efficiency can be improved by selecting an appropriate N for each LoD.
  • the three-dimensional data encoding device may calculate the predicted value of the attribute information of the three-dimensional points by the weighted average value of the attribute information of the surrounding N coded three-dimensional points. For example, the three-dimensional data coding device calculates the weight using the distance information of the target three-dimensional point and the surrounding N three-dimensional points.
  • the higher layer of LoD is set to have a larger N value
  • the lower layer is set to have a smaller N value. Since the distance between the three-dimensional points to which they belong is large in the upper layer of LoD, there is a possibility that the prediction accuracy can be improved by setting a large value of N and selecting and averaging a plurality of surrounding three-dimensional points. Further, since the distance between the three-dimensional points to which the LoD lower layer belongs is short, it is possible to make an efficient prediction while suppressing the amount of averaging processing by setting the value of N small.
  • FIG. 54 is a diagram showing an example of attribute information used for the predicted value.
  • the peripheral point P' is selected based on the distance from the point P.
  • the predicted value of the attribute information of the point b2 shown in FIG. 54 is generated by using the attribute information of the points a0, a1, a2, b0, and b1.
  • the predicted value is calculated by a distance-dependent weighted average.
  • the predicted value a2p of the point a2 is calculated by the weighted average of the attribute information of the points a0 and a1 as shown in (Equation A2) and (Equation A3).
  • a i is a value of the attribute information of the point ai.
  • the predicted value b2p of the point b2 is calculated by the weighted average of the attribute information of the points a0, a1, a2, b0, and b1 as shown in (Equation A4) to (Equation A6).
  • B i is the value of the attribute information of the point bi.
  • the three-dimensional data encoding device calculates the difference value (predicted residual) between the value of the attribute information of the three-dimensional point and the predicted value generated from the surrounding points, and even if the calculated predicted residual is quantized.
  • a three-dimensional data coding device performs quantization by dividing the predicted residual by a quantization scale (also called a quantization step).
  • a quantization scale also called a quantization step.
  • the smaller the quantization scale the smaller the error (quantization error) that can occur due to quantization.
  • the larger the quantization scale the larger the quantization error.
  • the three-dimensional data coding device may change the quantization scale used for each LoD. For example, in a three-dimensional data coding apparatus, the higher layer has a smaller quantization scale, and the lower layer has a larger quantization scale. Since the value of the attribute information of the three-dimensional point belonging to the upper layer may be used as the predicted value of the attribute information of the three-dimensional point belonging to the lower layer, the quantization scale of the upper layer is reduced in the upper layer. Coding efficiency can be improved by suppressing possible quantization errors and improving the accuracy of predicted values.
  • the three-dimensional data coding device may add a quantization scale used for each LoD to the header or the like. As a result, the three-dimensional data decoding device can correctly decode the quantization scale, so that the bit stream can be appropriately decoded.
  • the three-dimensional data coding apparatus may convert a signed integer value (signed quantization value), which is a predicted residual after quantization, into an unsigned integer value (unsigned quantization value). This eliminates the need to consider the occurrence of negative integers when entropy encoding the predicted residuals.
  • the three-dimensional data coding apparatus does not necessarily have to convert the signed integer value into an unsigned integer value, and for example, the sign bit may be separately entropy-encoded.
  • the predicted residual is calculated by subtracting the predicted value from the original value.
  • the prediction residual a2r of the point a2 as shown in (Equation A7), from the values A 2 of the attribute information of the point a2, is calculated by subtracting the prediction value a2p the point a2.
  • a2r A 2 -a2p ⁇ (formula A7)
  • b2r B 2 -b2p ⁇ (formula A8)
  • the predicted residual is quantized by dividing by QS (Quantization Step).
  • the quantized value a2q at point a2 is calculated by (Equation A9).
  • the quantized value b2q at point b2 is calculated by (Equation A10).
  • QS_LoD0 is a QS for LoD0
  • QS_LoD1 is a QS for LoD1. That is, the QS may be changed according to LoD.
  • the three-dimensional data encoding device converts the signed integer value, which is the above-mentioned quantization value, into an unsigned integer value as follows.
  • the signed integer value a2q is smaller than 0, the three-dimensional data coding apparatus sets the unsigned integer value a2u to -1- (2 ⁇ a2q).
  • the signed integer value a2q is 0 or more, the three-dimensional data coding apparatus sets the unsigned integer value a2u to 2 ⁇ a2q.
  • the three-dimensional data encoding device sets the unsigned integer value b2u to -1- (2 ⁇ b2q) when the signed integer value b2q is smaller than 0.
  • the three-dimensional data coding apparatus sets the unsigned integer value b2u to 2 ⁇ b2q.
  • the three-dimensional data coding apparatus may encode the predicted residual (unsigned integer value) after quantization by entropy coding. For example, after binarizing an unsigned integer value, binary arithmetic coding may be applied.
  • the three-dimensional data coding device may switch the binarization method according to the value of the predicted residual. For example, when the predicted residual pu is smaller than the threshold value R_TH, the three-dimensional data encoding device binarizes the predicted residual pu with a fixed number of bits required to express the threshold value R_TH. Further, when the predicted residual pu is equal to or higher than the threshold value R_TH, the three-dimensional data encoding device uses the binarized data of the threshold value R_TH and the value of (pu-R_TH) by using an exponential-Golomb or the like. Threshold.
  • the three-dimensional data encoding device when the threshold value R_TH is 63 and the predicted residual pu is smaller than 63, the predicted residual pu is binarized with 6 bits. Further, when the predicted residual pu is 63 or more, the three-dimensional data encoding device binarizes the binary data (111111) and (pu-63) of the threshold value R_TH by using the exponential Golomb. Arithmetic coding is performed.
  • the three-dimensional data coding device generates 6-bit binary data (100,000) when the predicted residual pu is 32, and arithmetically encodes this bit string. Further, when the predicted residual pu is 66, the three-dimensional data encoding device generates binary data (111111) of the threshold value R_TH and a bit string (00100) representing the value 3 (66-63) in exponential Golomb. , This bit string (111111 + 00100) is arithmetically coded.
  • the three-dimensional data encoding device switches the binarization method according to the magnitude of the predicted residual, so that the number of binarized bits increases sharply when the predicted residual becomes large. It is possible to encode while suppressing.
  • the three-dimensional data coding device may add the threshold value R_TH to the header of the bit stream or the like.
  • the three-dimensional data coding device sets the threshold value R_TH to a large value.
  • the possibility of encoding the binarized data of the threshold value R_TH is reduced, and the coding efficiency is improved.
  • the three-dimensional data coding device sets the threshold value R_TH to be small. As a result, it is possible to prevent a sudden increase in the bit length of the binarized data.
  • the three-dimensional data coding device may switch the threshold value R_TH for each LoD and add the threshold value R_TH for each LoD to the header or the like. That is, the three-dimensional data coding apparatus may switch the binarization method for each LoD. For example, in the upper layer, the distance between the three-dimensional points is long, so that the prediction accuracy is poor and the prediction residual may increase as a result. Therefore, the three-dimensional data coding device prevents a sudden increase in the bit length of the binarized data by setting the threshold value R_TH small for the upper layer. In addition, since the distance between the three-dimensional points is short in the lower layer, the prediction accuracy may be high and the prediction residual may be small as a result. Therefore, the three-dimensional data coding apparatus improves the coding efficiency by setting the threshold value R_TH large for the hierarchy.
  • FIG. 55 is a diagram showing an example of the exponential Golomb code, and is a diagram showing the relationship between the value before binarization (multi-value) and the bit (code) after binarization. In addition, 0 and 1 shown in FIG. 55 may be inverted.
  • the three-dimensional data coding device applies arithmetic coding to the binarized data of the predicted residuals.
  • arithmetic coding is applied, the n-bit code (n-bit code), which is the part of the binarized data binarized with n bits, and the rest, which is the part binarized using the exponential gorom.
  • the tendency of the appearance probability of 0 and 1 of each bit may be different depending on the code (reminding code). Therefore, the three-dimensional data coding apparatus may switch the application method of arithmetic coding between the n-bit code and the remaining code.
  • a three-dimensional data coding device performs arithmetic coding on an n-bit code using a different coding table (probability table) for each bit.
  • the three-dimensional data coding apparatus may change the number of coding tables used for each bit. For example, the three-dimensional data coding apparatus performs arithmetic coding using one coding table for the first bit b0 of the n-bit code. Further, the three-dimensional data coding device uses two coding tables for the next bit b1. Further, the three-dimensional data coding apparatus switches the coding table used for arithmetic coding of bit b1 according to the value (0 or 1) of b0.
  • the three-dimensional data coding device further uses four coding tables for the next bit b2. Further, the three-dimensional data coding apparatus switches the coding table used for arithmetic coding of bit b2 according to the values (0 to 3) of b0 and b1.
  • the three-dimensional data coding apparatus uses 2 n-1 coding tables when arithmetically coding each bit bn-1 of the n-bit code. Further, the three-dimensional data coding apparatus switches the coding table to be used according to the value (generation pattern) of the bits before bn-1. As a result, the three-dimensional data coding apparatus can use an appropriate coding table for each bit, so that the coding efficiency can be improved.
  • the three-dimensional data coding device may reduce the number of coding tables used in each bit. For example, in a three-dimensional data coding device, when each bit bn-1 is arithmetically coded, 2 m pieces are used according to the value (generation pattern) of m bits (m ⁇ n-1) before bn-1. The coding table of may be switched. As a result, the coding efficiency can be improved while reducing the number of coding tables used in each bit.
  • the three-dimensional data coding apparatus may update the probability of occurrence of 0 and 1 in each coding table according to the value of the binarized data actually generated. Further, the three-dimensional data coding apparatus may fix the occurrence probabilities of 0 and 1 in the coding table of some bits. As a result, the number of times the occurrence probability is updated can be suppressed, so that the amount of processing can be reduced.
  • the number of coding tables for b0 is one (CTb0).
  • CTb10, CTb11 There are two coding tables for b1 (CTb10, CTb11). Further, the coding table to be used is switched according to the value of b0 (0 to 1).
  • CTb20, CTb21, CTb22, CTb23 There are four coding tables for b2 (CTb20, CTb21, CTb22, CTb23). Further, the coding table to be used is switched according to the values of b0 and b1 (0 to 3).
  • CTbn0, CTbn1, ..., CTbn (2 n-1 -1) There are 2 n-1 coding tables for bn-1 (CTbn0, CTbn1, ..., CTbn (2 n-1 -1)). Further, the coding table to be used can be switched according to the value of b0b1 ... bn-2 (0 to 2 n-1 -1).
  • FIG. 56 is a diagram for explaining processing when, for example, the remaining code is an exponential Golomb code.
  • the remaining code which is a portion binarized using the exponential Golomb, includes a prefix part and a suffix part as shown in FIG. 56.
  • the three-dimensional data coding apparatus switches the coding table between the prefix unit and the suffix unit. That is, the three-dimensional data coding device arithmetically encodes each bit included in the prefix part using the encoding table for the prefix, and uses the encoding table for the suffix for each bit included in the suffix part. Arithmetic code.
  • the three-dimensional data coding device may update the probability of occurrence of 0 and 1 in each coding table according to the value of the binarized data actually generated.
  • the three-dimensional data coding device may fix the probability of occurrence of 0 and 1 in either coding table.
  • the number of times the occurrence probability is updated can be suppressed, so that the amount of processing can be reduced.
  • the three-dimensional data coding apparatus may update the occurrence probability with respect to the prefix unit and fix the occurrence probability with respect to the suffix unit.
  • the three-dimensional data coding device decodes the predicted residual after quantization by inverse quantization and reconstruction, and decodes the decoded value, which is the decoded predicted residual, after the three-dimensional point to be encoded. Used for prediction. Specifically, the three-dimensional data encoding device calculates the inverse quantization value by multiplying the predicted residual (quantization value) after quantization by the quantization scale, and obtains the inverse quantization value and the predicted value. To obtain the decoding value (reconstruction value).
  • the inverse quantization value a2iq of the point a2 is calculated by (Equation A11) using the quantization value a2q of the point a2.
  • the inverse quantization value b2iq of the point b2 is calculated by (Equation A12) using the quantization value b2q of the point b2.
  • QS_LoD0 is a QS for LoD0
  • QS_LoD1 is a QS for LoD1. That is, the QS may be changed according to LoD.
  • the decoding value a2rec of the point a2 is calculated by adding the predicted value a2p of the point a2 to the inverse quantization value a2iq of the point a2 as shown in (Equation A13).
  • the decoding value b2rec of the point b2 is calculated by adding the predicted value b2p of the point b2 to the inverse quantization value b2iq of the point b2.
  • FIG. 57 is a diagram showing an example of syntax of the attribute header (attribute_header) according to the present embodiment.
  • the attribute header is the header information of the attribute information.
  • the attribute header includes layer number information (NumLoD), three-dimensional point information (NumOfPoint [i]), layer threshold value (Thres_Lod [i]), and peripheral point information (NumNeightPoint [i]).
  • QS [i] quantization scale
  • R_TH [i] binarization threshold
  • the number of layers information indicates the number of layers of LoD used.
  • the three-dimensional point information indicates the number of three-dimensional points belonging to the layer i.
  • the three-dimensional data coding apparatus may add three-dimensional point total number information (AllNumOfPoint) indicating the total number of three-dimensional points to another header.
  • the three-dimensional data coding device does not have to add NuOfPoint [NumMoD-1] indicating the number of three-dimensional points belonging to the lowest layer to the header.
  • the three-dimensional data decoding device can calculate NuOfPoint [NumMoD-1] by (Equation A15). As a result, the amount of code in the header can be reduced.
  • the hierarchy threshold (Thres_Lod [i]) is a threshold used for setting the hierarchy i.
  • the three-dimensional data coding device and the three-dimensional data decoding device configure LoDi so that the distance between each point in LoDi is larger than the threshold value Thres_LoD [i]. Further, the three-dimensional data coding device does not have to add the value of Thres_Lod [NumMoD-1] (bottom layer) to the header. In this case, the three-dimensional data decoding device estimates the value of Thres_Lod [NumMoD-1] to be 0. As a result, the amount of code in the header can be reduced.
  • Peripheral point information indicates the upper limit of the peripheral points used to generate the predicted value of the three-dimensional points belonging to the layer i.
  • the three-dimensional data coding apparatus may calculate a predicted value using the points around M when the peripheral points M are less than the number of surrounding points [i] (M ⁇ NumNeigorPoint [i]). Further, when it is not necessary for the three-dimensional data coding device to separate the value of NuNeightPoint [i] in each LoD, one peripheral point information (NumNeigoPoint) used in all LoDs may be added to the header. Good.
  • the prediction threshold (THd [i]) indicates the upper limit of the distance between the surrounding 3D point and the target 3D point used for predicting the target 3D point to be encoded or decoded in the layer i.
  • the three-dimensional data encoding device and the three-dimensional data decoding device do not use a three-dimensional point whose distance from the target three-dimensional point is more than THd [i] for prediction. If it is not necessary to divide the value of THd [i] in each LoD, the three-dimensional data coding device may add one prediction threshold value (THd) used in all LoDs to the header. ..
  • the quantization scale indicates the quantization scale used in the quantization and inverse quantization of the layer i.
  • the binarization threshold value (R_TH [i]) is a threshold value for switching the binarization method of the predicted residual of the three-dimensional points belonging to the layer i.
  • the three-dimensional data encoding device binarizes the predicted residual pu with a fixed number of bits when the predicted residual is smaller than the threshold R_TH, and when the predicted residual is equal to or higher than the threshold R_TH, the binary value of the threshold R_TH.
  • the binarization data and the value of (pu-R_TH) are binarized using the exponential gorom.
  • the three-dimensional data encoding device adds one binarization threshold value (R_TH) used in all LoDs to the header. May be good.
  • minimum value minimum number of bits
  • the three-dimensional data encoding device entropy-encodes at least one of NumLoD, Thres_Lod [i], NuNeightborPoint [i], THd [i], QS [i] and R_TH [i] and adds them to the header. May be good.
  • the three-dimensional data coding device may binarize each value and perform arithmetic coding.
  • the three-dimensional data coding apparatus may encode each value with a fixed length in order to reduce the amount of processing.
  • the three-dimensional data coding device does not have to add at least one of NumLoD, Thres_Lod [i], NuNeightborPoint [i], THd [i], QS [i], and R_TH [i] to the header. ..
  • at least one of these values may be specified by a profile or level of a standard or the like. As a result, the bit amount of the header can be reduced.
  • FIG. 58 is a diagram showing an example of syntax of attribute data (attribute_data) according to the present embodiment.
  • This attribute data includes coded data of attribute information of a plurality of three-dimensional points. As shown in FIG. 58, the attribute data includes an n-bit code and a remaining code.
  • the n-bit code (n-bit code) is the coded data of the predicted residual of the value of the attribute information or a part thereof.
  • the bit length of the n-bit code depends on the value of R_TH [i]. For example, when the value indicated by R_TH [i] is 63, the n-bit code is 6 bits, and when the value indicated by R_TH [i] is 255, the n-bit code is 8 bits.
  • the remaining code is the coded data encoded by the exponential Golomb among the coded data of the predicted residuals of the value of the attribute information.
  • the remaining code is encoded or decoded when the n-bit code is the same as R_TH [i]. Further, the three-dimensional data decoding device adds the value of the n-bit code and the value of the remaining code to decode the predicted residual. If the n-bit code is not the same value as R_TH [i], the remaining code may not be encoded or decoded.
  • FIG. 59 is a flowchart of the three-dimensional data coding process by the three-dimensional data coding apparatus.
  • the three-dimensional data coding device encodes the position information (geometry) (S3001). For example, in three-dimensional data coding, coding is performed using an octree representation.
  • the 3D data coding device When the position of the 3D point changes due to quantization or the like after the position information is coded, the 3D data coding device reassigns the attribute information of the original 3D point to the changed 3D point ( S3002). For example, the three-dimensional data coding device reassigns by interpolating the value of the attribute information according to the amount of change in the position. For example, the three-dimensional data encoding device detects N three-dimensional points before the change, which are close to the three-dimensional position after the change, and weights and averages the values of the attribute information of the N three-dimensional points. For example, a three-dimensional data coding device determines weights in a weighted average based on the distances from the changed three-dimensional position to each of the N three dimensions.
  • the three-dimensional data encoding device determines the value obtained by the weighted averaging as the value of the attribute information of the three-dimensional point after the change.
  • the three-dimensional data encoding device uses the value of the attribute information of the three-dimensional points after the change as the value before the change.
  • the average value of the attribute information of two or more three-dimensional points may be assigned.
  • the three-dimensional data coding device encodes the attribute information (Attribute) after reassignment (S3003).
  • the three-dimensional data coding apparatus may encode the plurality of types of attribute information in order.
  • the three-dimensional data coding apparatus may generate a bit stream in which the color coding result and the reflectance coding result are added after the color coding result when the color and the reflectance are encoded as the attribute information. ..
  • the order of the plurality of coding results of the attribute information added to the bit stream is not limited to this order, and may be any order.
  • the three-dimensional data encoding device may add information indicating the encoded data start location of each attribute information in the bit stream to the header or the like.
  • the three-dimensional data decoding device can selectively decode the attribute information that needs to be decoded, so that the decoding process of the attribute information that does not need to be decoded can be omitted. Therefore, the processing amount of the three-dimensional data decoding device can be reduced.
  • the three-dimensional data coding apparatus may encode a plurality of types of attribute information in parallel and integrate the coding results into one bit stream. As a result, the three-dimensional data coding device can encode a plurality of types of attribute information at high speed.
  • FIG. 60 is a flowchart of the attribute information coding process (S3003).
  • the three-dimensional data coding device sets LoD (S3011). That is, the three-dimensional data coding device assigns each three-dimensional point to any of a plurality of LoDs.
  • the three-dimensional data coding device starts a loop in units of LoD (S3012). That is, the three-dimensional data coding apparatus repeats the processes of steps S3013 to S3021 for each LoD.
  • the three-dimensional data coding device starts a loop in units of three-dimensional points (S3013). That is, the three-dimensional data coding apparatus repeats the processes of steps S3014 to S3020 for each three-dimensional point.
  • the three-dimensional data encoding device searches for a plurality of peripheral points that are three-dimensional points existing around the target three-dimensional point used for calculating the predicted value of the target three-dimensional point to be processed (S3014).
  • the three-dimensional data encoding device calculates a weighted average of the values of the attribute information of the plurality of peripheral points, and sets the obtained value as the predicted value P (S3015).
  • the three-dimensional data encoding device calculates the predicted residual, which is the difference between the attribute information of the target three-dimensional point and the predicted value (S3016).
  • the three-dimensional data coding apparatus calculates the quantized value by quantizing the predicted residual (S3017).
  • the three-dimensional data coding device arithmetically encodes the quantized value (S3018).
  • the three-dimensional data coding device calculates the dequantized value by dequantizing the quantized value (S3019).
  • the three-dimensional data coding apparatus generates a decoded value by adding the predicted value to the inverse quantization value (S3020).
  • the three-dimensional data coding device ends the loop in units of three-dimensional points (S3021). Further, the three-dimensional data coding device ends the loop in units of LoD (S3022).
  • the three-dimensional data decoding device is decoded by arithmetically decoding the binarized data of the attribute information in the bit stream generated by the three-dimensional data coding device in the same manner as the three-dimensional data coding device. Generate binarized data.
  • the application method of arithmetic coding is switched between the part binarized with n bits (n-bit code) and the part binarized with exponential gorom (remaining code). In this case, the three-dimensional data decoding device performs decoding according to the application of arithmetic decoding.
  • the three-dimensional data decoding device performs arithmetic decoding using a different coding table (decoding table) for each bit in the arithmetic decoding method of n-bit code.
  • the three-dimensional data decoding device may change the number of coding tables used for each bit. For example, arithmetic decoding is performed using one coding table for the first bit b0 of the n-bit code. Further, the three-dimensional data decoding device uses two coding tables for the next bit b1. Further, the three-dimensional data decoding device switches the coding table used for arithmetic decoding of bit b1 according to the value (0 or 1) of b0.
  • the three-dimensional data decoding device further uses four coding tables for the next bit b2. Further, the three-dimensional data decoding device switches the coding table used for arithmetic decoding of bit b2 according to the values (0 to 3) of b0 and b1.
  • the three-dimensional data decoding apparatus uses 2 n-1 coding tables when arithmetically decoding each bit bn-1 of the n-bit code. Further, the three-dimensional data decoding device switches the coding table to be used according to the value (generation pattern) of the bits before bn-1. As a result, the three-dimensional data decoding apparatus can appropriately decode a bit stream having improved coding efficiency by using an appropriate coding table for each bit.
  • the three-dimensional data decoding device may reduce the number of coding tables used in each bit. For example, when the three-dimensional data decoding device arithmetically decodes each bit bn-1, 2 m codes are used according to the value (generation pattern) of m bits (m ⁇ n-1) before bn-1. You may switch the conversion table. As a result, the three-dimensional data decoding device can appropriately decode the bit stream with improved coding efficiency while suppressing the number of coding tables used for each bit.
  • the three-dimensional data decoding device may update the probability of occurrence of 0 and 1 in each coding table according to the value of the binarized data actually generated. Further, the three-dimensional data decoding device may fix the occurrence probabilities of 0 and 1 in the coding table of some bits. As a result, the number of times the occurrence probability is updated can be suppressed, so that the amount of processing can be reduced.
  • the number of coding tables for b0 is one (CTb0).
  • CTb10, CTb11 There are two coding tables for b1 (CTb10, CTb11). Further, the coding table is switched according to the value of b0 (0 to 1).
  • CTb20, CTb21, CTb22, CTb23 There are four coding tables for b2 (CTb20, CTb21, CTb22, CTb23). Further, the coding table is switched according to the values of b0 and b1 (0 to 3).
  • the coding table for bn-1 is 2 n-1 (CTbn0, CTbn1, ..., CTbn (2 n-1 -1)). Further, the coding table is switched according to the values of b0b1 ... bn-2 (0 to 2 n-1 -1).
  • FIG. 61 is a diagram for explaining processing when, for example, the remaining code is an exponential Golomb code.
  • the portion (remaining code) binarized and encoded by the three-dimensional data coding apparatus using the exponential Golomb includes a prefix portion and a suffix portion as shown in FIG.
  • the three-dimensional data decoding device switches the coding table between the prefix unit and the suffix unit. That is, the three-dimensional data decoding device arithmetically decodes each bit included in the prefix section using the prefix coding table, and arithmetically decodes each bit included in the suffix section using the suffix coding table. Decrypt.
  • the three-dimensional data decoding device may update the probability of occurrence of 0 and 1 in each coding table according to the value of the binarized data generated at the time of decoding.
  • the three-dimensional data decoding device may fix the probability of occurrence of 0 and 1 in either coding table.
  • the number of times the occurrence probability is updated can be suppressed, so that the amount of processing can be reduced.
  • the three-dimensional data decoding device may update the probability of occurrence with respect to the prefix section and fix the probability of occurrence with respect to the suffix section.
  • the three-dimensional data decoding device multi-values the binarized data of the predicted residual that has been arithmetically decoded according to the coding method used in the three-dimensional data coding device, so that the predicted residual after quantization is performed. Decode the difference (unsigned integer value).
  • the three-dimensional data decoding device first calculates the value of the n-bit code decoded by arithmetically decoding the binarized data of the n-bit code.
  • the three-dimensional data decoding device compares the value of the n-bit code with the value of R_TH.
  • the three-dimensional data decoding device determines that the bit encoded by the exponential Golomb exists next, and uses the binarized data encoded by the exponential Golomb. Arithmetically decode some remaining code. Then, the three-dimensional data decoding device calculates the value of the remaining code from the decoded remaining code by using the reverse lookup table showing the relationship between the remaining code and the value.
  • FIG. 62 is a diagram showing an example of a reverse lookup table showing the relationship between the remaining code and its value.
  • the three-dimensional data decoding apparatus obtains a multi-valued predicted residual after quantization by adding the obtained value of the remaining code to R_TH.
  • the three-dimensional data decoding device when the value of the n-bit code and the value of R_TH do not match (the value is smaller than R_TH), the value of the n-bit code is used as it is, and the predicted residue after quantization is quantified. Determine the difference.
  • the three-dimensional data decoding device can appropriately decode the bit stream generated by switching the binarization method according to the value of the predicted residual in the three-dimensional data coding device.
  • the three-dimensional data decoding device decodes the value of the threshold value R_TH from the header and switches the decoding method using the decoded value of the threshold value R_TH. Good. Further, when the threshold value R_TH is added to the header or the like for each LoD, the three-dimensional data decoding device switches the decoding method using the threshold value R_TH decoded for each LoD.
  • the three-dimensional data decoding device obtains the value of the remaining code by decoding the remaining code by the exponential Golomb. For example, in the example shown in FIG. 62, the remaining code is 00100, and 3 is obtained as the value of the remaining code.
  • the three-dimensional data decoding apparatus obtains the predicted residual value 66 by adding the value 63 of the threshold value R_TH and the value 3 of the remaining code.
  • the three-dimensional data decoding device sets the n-bit code value 32 as the predicted residual value.
  • the three-dimensional data decoding device converts the decoded predicted residual after quantization from an unsigned integer value to a signed integer value by, for example, the reverse processing of the processing in the three-dimensional data coding device.
  • the three-dimensional data decoding apparatus can appropriately decode the generated bit stream without considering the generation of negative integers when entropy-coding the predicted residuals.
  • the three-dimensional data decoding device does not necessarily have to convert an unsigned integer value into a signed integer value. For example, when decoding a bit stream generated by separately entropy-coding a sign bit, the sign bit is decoded. You may.
  • the three-dimensional data decoding device generates a decoded value by decoding the predicted residual after quantization converted into a signed integer value by inverse quantization and reconstruction. Further, the three-dimensional data decoding device uses the generated decoded value for the prediction after the three-dimensional point to be decoded. Specifically, the three-dimensional data decoding device calculates the inverse quantization value by multiplying the predicted residual after quantization by the decoded quantization scale, and adds the inverse quantization value and the predicted value. To obtain the decoded value.
  • the decoded unsigned integer value (unsigned quantization value) is converted to a signed integer value by the following processing.
  • the three-dimensional data decoding device sets the signed integer value a2q to ⁇ ((a2u + 1) >> 1).
  • the three-dimensional data decoding apparatus sets the signed integer value a2q to (a2u >> 1) when the LSB of the unsigned integer value a2u is not 1.
  • the three-dimensional data decoding device sets the signed integer value b2q to-((b2u + 1) >> 1) when the LSB of the decoded unsigned integer value b2u is 1.
  • the three-dimensional data decoding device sets the signed integer value b2q to (b2u >> 1) when the LSB of the unsigned integer value n2u is not 1.
  • the details of the inverse quantization and reconstruction processing by the three-dimensional data decoding apparatus are the same as those of the inverse quantization and reconstruction processing in the three-dimensional data coding apparatus.
  • FIG. 63 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device.
  • the three-dimensional data decoding device decodes the position information (geometry) from the bit stream (S3031). For example, a three-dimensional data decoding device decodes using an ocree representation.
  • the three-dimensional data decoding device decodes the attribute information (Attribute) from the bit stream (S3032). For example, when decoding a plurality of types of attribute information, the three-dimensional data decoding device may decode the plurality of types of attribute information in order. For example, when decoding a color and a reflectance as attribute information, the three-dimensional data decoding apparatus decodes the color coding result and the reflectance coding result in the order of being added to the bit stream. For example, in a bitstream, when the reflectance coding result is added after the color coding result, the three-dimensional data decoding device decodes the color coding result, and then the reflectance coding result. To decrypt. The three-dimensional data decoding device may decode the coding result of the attribute information added to the bit stream in any order.
  • the three-dimensional data decoding device may acquire information indicating the encoded data start location of each attribute information in the bit stream by decoding the header or the like. As a result, the three-dimensional data decoding device can selectively decode the attribute information that needs to be decoded, so that the decoding process of the attribute information that does not need to be decoded can be omitted. Therefore, the processing amount of the three-dimensional data decoding device can be reduced. Further, the three-dimensional data decoding device may decode a plurality of types of attribute information in parallel and integrate the decoding results into one three-dimensional point cloud. As a result, the three-dimensional data decoding device can decode a plurality of types of attribute information at high speed.
  • FIG. 64 is a flowchart of the attribute information decoding process (S3032).
  • the three-dimensional data decoding device sets LoD (S3041). That is, the three-dimensional data decoding device assigns each of the plurality of three-dimensional points having the decoded position information to any of the plurality of LoDs.
  • this allocation method is the same as the allocation method used in the three-dimensional data coding apparatus.
  • the three-dimensional data decoding device starts a loop in units of LoD (S3042). That is, the three-dimensional data decoding device repeats the processes of steps S3043 to S3049 for each LoD.
  • the three-dimensional data decoding device starts a loop in units of three-dimensional points (S3043). That is, the three-dimensional data decoding device repeats the processes of steps S3044 to S3048 for each three-dimensional point.
  • the three-dimensional data decoding device searches for a plurality of peripheral points that are three-dimensional points existing around the target three-dimensional point used for calculating the predicted value of the target three-dimensional point to be processed (S3044).
  • the three-dimensional data decoding device calculates a weighted average of the values of the attribute information of the plurality of peripheral points, and sets the obtained value as the predicted value P (S3045).
  • the three-dimensional data decoding device arithmetically decodes the quantized value from the bit stream (S3046). Further, the three-dimensional data decoding apparatus calculates the dequantized value by dequantizing the decoded quantization value (S3047). Next, the three-dimensional data decoding device generates a decoded value by adding the predicted value to the inverse quantization value (S3048). Next, the three-dimensional data decoding device ends the loop in units of three-dimensional points (S3049). Further, the three-dimensional data decoding device ends the loop in units of LoD (S3050).
  • FIG. 65 is a block diagram showing the configuration of the three-dimensional data coding device 3000 according to the present embodiment.
  • the three-dimensional data coding device 3000 includes a position information coding unit 3001, an attribute information reassignment unit 3002, and an attribute information coding unit 3003.
  • the attribute information coding unit 3003 encodes the position information (geometry) of a plurality of three-dimensional points included in the input point group.
  • the attribute information reassignment unit 3002 reassigns the values of the attribute information of a plurality of three-dimensional points included in the input point group by using the coding and decoding results of the position information.
  • the attribute information coding unit 3003 encodes the reassigned attribute information (attribute). Further, the three-dimensional data coding apparatus 3000 generates a bit stream including the coded position information and the coded attribute information.
  • FIG. 66 is a block diagram showing the configuration of the three-dimensional data decoding device 3010 according to the present embodiment.
  • the three-dimensional data decoding device 3010 includes a position information decoding unit 3011 and an attribute information decoding unit 3012.
  • the position information decoding unit 3011 decodes the position information (geometry) of a plurality of three-dimensional points from the bit stream.
  • the attribute information decoding unit 3012 decodes the attribute information (attribute) of a plurality of three-dimensional points from the bit stream. Further, the three-dimensional data decoding device 3010 generates an output point cloud by combining the decoded position information and the decoded attribute information.
  • the three-dimensional point to be encoded may be referred to as a first three-dimensional point, and the surrounding three-dimensional points may be referred to as a second three-dimensional point.
  • the attribute value of the closest three-dimensional point among the encoded and decoded three-dimensional points of the three-dimensional point to be encoded is used as it is. It may be generated as a predicted value. Further, in the generation of the predicted value, the predicted mode information (PredMode) may be added for each three-dimensional point so that the predicted value can be generated by selecting one predicted value from a plurality of predicted values. That is, for example, in the prediction mode of the total number M, the prediction mode 0 is assigned the average value, the prediction mode 1 is assigned the attribute value of the three-dimensional point A, ..., The prediction mode M-1 is assigned the attribute value of the three-dimensional point Z.
  • PredMode predicted mode information
  • the prediction mode used for prediction is added to the bitstream for each three-dimensional point.
  • the attribute information itself of the surrounding three-dimensional points is calculated as the predicted value. It may be smaller than the second prediction mode value indicating the second prediction mode.
  • the "mean value" which is the predicted value calculated in the prediction mode 0, is the average value of the attribute values of the three-dimensional points around the three-dimensional point to be encoded.
  • FIG. 67 is a diagram showing a first example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • FIG. 68 is a diagram showing an example of attribute information used for the predicted value according to the fifth embodiment.
  • FIG. 69 is a diagram showing a second example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • the predicted value of the attribute information of the point b2 can be generated by using the attribute information of the points a0, a1, a2, and b1.
  • the attribute value of each point a0, a1, a2, b1 is predicted based on the distance information from the point b2 to each point a0, a1, a2, b1. You may select the prediction mode to generate as.
  • the prediction mode is added for each three-dimensional point to be encoded.
  • the predicted value is calculated according to the value according to the added prediction mode.
  • the predicted value of the attribute information of the point a2 can be generated by using the attribute information of the points a0 and a1.
  • the prediction mode is added for each three-dimensional point to be encoded.
  • the predicted value is calculated according to the value according to the added prediction mode.
  • the prediction mode in which the prediction value is not assigned in the table may be set as not available.
  • the allocation of values in the prediction mode may be determined in the order of distance from the three-dimensional point to be encoded.
  • the prediction mode value indicating a plurality of prediction modes is smaller as the distance from the three-dimensional point to be encoded to the surrounding three-dimensional points having the attribute information used as the prediction value is shorter.
  • FIG. 67 it is shown that the distance to the point b2, which is the three-dimensional point to be encoded, is short in the order of the points b1, a2, a1, and a0.
  • the attribute information of the point b1 is calculated as the predicted value in the predicted mode in which the predicted mode value of the two or more predicted modes is indicated by "1", and the predicted mode value is indicated by "2".
  • the attribute information of point a2 is calculated as a predicted value in the predicted mode.
  • the prediction mode value indicating the prediction mode in which the attribute information of the point b1 is calculated as the prediction value is the prediction in which the attribute information of the point a2 whose distance from the point b2 is farther than the point b1 is calculated as the prediction value. It is smaller than the predicted mode value indicating the mode.
  • a small prediction mode value can be assigned to a point where the prediction is easy to hit and may be easily selected because the distance is short, and the number of bits for encoding the prediction mode value can be reduced. Further, a small prediction mode value may be preferentially assigned to a three-dimensional point belonging to the same LoD as the three-dimensional point to be encoded.
  • FIG. 70 is a diagram showing a third example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • the third example is an example in which the attribute information used for the predicted value is a value based on the color information (YUV) of the surrounding three-dimensional points.
  • the attribute information used for the predicted value may be color information indicating the color of the three-dimensional point.
  • the predicted value calculated in the predicted mode in which the predicted mode value is indicated by "0" is the average of the components of each YUV that defines the YUV color space.
  • the predicted values are the weighted average Yave of Yb1, Ya2, Ya1, and Ya0, which are the values of the Y components corresponding to the points b1, a2, a1, and a0, respectively, and the points b1, a2, a1, and a0.
  • the weighted average Uave of Ub1, Ua2, Ua1, and Ua0 corresponding to each of the above, and the V component values of Vb1, Va2, Va1, and Va0 corresponding to points b1, a2, a1, and a0, respectively.
  • the predicted values calculated in the predicted modes whose prediction mode values are indicated by “1" to "4" include color information of surrounding three-dimensional points b1, a2, a1, and a0, respectively.
  • the color information is indicated by a combination of the values of the Y component, the U component, and the V component.
  • the color information is indicated by a value defined in the YUV color space, but the color information is not limited to the YUV color space and may be indicated by a value defined in the RGB color space. It may be indicated by a value defined in the color space of.
  • the average or attribute information of 2 or more may be calculated as the predicted value of the prediction mode. Further, the average or attribute information of 2 or more may indicate the values of 2 or more components that define the color space, respectively.
  • the Y component, U component, and V component of the attribute value of the three-dimensional point to be encoded are predicted values, respectively. It may be encoded by using it as Ya2, Ua2, Va2. In this case, "2" as the prediction mode value is added to the bit stream.
  • FIG. 71 is a diagram showing a fourth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • the fourth example is an example in which the attribute information used for the predicted value is a value based on the reflectance information of the surrounding three-dimensional points.
  • the reflectance information is, for example, information indicating the reflectance R.
  • the predicted values calculated in the predicted mode in which the predicted mode value is “0” are the reflectances Rb1, Ra2, Ra1, Ra0 corresponding to the points b1, a2, a1, and a0, respectively. Weighted average Rave. Further, the predicted values calculated in the predicted modes whose prediction mode values are indicated by "1" to "4" are the reflectances Rb1, Ra2, Ra1, Ra0 of the surrounding three-dimensional points b1, a2, a1, and a0, respectively. Is.
  • the prediction mode whose prediction mode value is indicated by "3" is selected in the table of FIG. 71
  • the reflectance of the attribute value of the three-dimensional point to be coded is used as the prediction value Ra1 for coding. May be good.
  • "3" as the prediction mode value is added to the bit stream.
  • the attribute information may include a first attribute information and a second attribute information of a type different from the first attribute information.
  • the first attribute information is, for example, color information.
  • the second attribute information is, for example, reflectance information.
  • the first predicted value may be calculated using the first attribute information, and the second predicted value may be calculated using the second attribute information.
  • the predicted value may be calculated in the prediction mode divided for each component.
  • each of the predicted values using the Y component, the U component, and the V component may be calculated in the prediction mode selected for each component.
  • a prediction mode Y for calculating a predicted value using the Y component
  • a prediction mode U for calculating a predicted value using the U component
  • a prediction mode for calculating a predicted value using the V component For each of V, a prediction mode value may be selected. In this case, the values in the tables of FIGS.
  • the prediction mode values indicating the prediction modes of each component are used as the prediction mode values indicating the prediction modes of each component, and these prediction mode values may be added to the bit stream, respectively.
  • the YUV color space has been described above, the same can be applied to the RGB color space.
  • the predicted value including two or more components out of the plurality of components of the attribute information may be calculated in a common prediction mode.
  • the prediction mode value is selected in each of the prediction mode Y for calculating the prediction value using the Y component and the prediction mode UV for calculating the prediction value using the UV component. You may.
  • the values in the tables of FIGS. 72 and 75, which will be described later, are used as the prediction mode values indicating the prediction modes of each component, and these prediction mode values may be added to the bit stream, respectively.
  • FIG. 72 is a diagram showing a fifth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • the fifth example is an example in which the attribute information used for the predicted value is the value of the Y component of the color information of the surrounding three-dimensional points.
  • the predicted values calculated in the predicted mode Y whose predicted mode value is “0” are the values of the Y components corresponding to the points b1, a2, a1 and a0, respectively, Yb1 and Ya2.
  • Ya1, and Ya0 are weighted average Yaves.
  • the predicted values calculated in the predicted modes whose prediction mode values are indicated by "1" to "4" are the values Yb1, Ya2, and Ya1 of the Y components of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.
  • Ya0 the predicted values calculated in the predicted mode Y whose predicted mode value is “0” are the values of the Y components corresponding to the points b1, a2, a1 and a0, respectively.
  • the prediction mode Y whose prediction mode value is indicated by “2” is selected in the table of FIG. 72
  • the Y component of the attribute value of the three-dimensional point to be encoded is used as the prediction value Ya2 for coding. You may. In this case, "2" as the prediction mode value is added to the bit stream.
  • FIG. 73 is a diagram showing a sixth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • the sixth example is an example in which the attribute information used for the predicted value is the value of the U component of the color information of the surrounding three-dimensional points.
  • the predicted values calculated in the predicted mode U whose predicted mode values are “0” are the values of the U components corresponding to the points b1, a2, a1 and a0, respectively, Ub1 and Ua2.
  • Ua1, Ua0 a weighted average Uave.
  • the predicted values calculated in the predicted modes whose prediction mode values are indicated by "1" to "4" are the values Ub1, Ua2, and Ua1 of the U components of the surrounding three-dimensional points b1, a2, a1, and a0, respectively.
  • Ua0 the predicted values calculated in the predicted mode U whose predicted mode values are “0” are the values of the U components corresponding to the points b1, a2, a1 and a0, respectively.
  • the prediction mode U whose prediction mode value is indicated by “1” is selected in the table of FIG. 73
  • the U component of the attribute value of the three-dimensional point to be encoded is used as the prediction value Ub1 for coding. You may. In this case, "1" as the prediction mode value is added to the bit stream.
  • FIG. 74 is a diagram showing a seventh example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • the seventh example is an example in which the attribute information used for the predicted value is the value of the V component of the color information of the surrounding three-dimensional points.
  • the predicted values calculated in the predicted mode V whose predicted mode value is “0” are the values of the V components corresponding to the points b1, a2, a1 and a0, respectively, Vb1 and Va2. , Va1, Va0, weighted average Vave. Further, the predicted values calculated in the predicted modes whose prediction mode values are indicated by "1" to "4" are the values Vb1, Va2, and Va1 of the V components of the surrounding three-dimensional points b1, a2, a1, and a0, respectively. , Va0.
  • the prediction mode V indicated by "4" is selected in the table of FIG. 74, the V component of the attribute value of the three-dimensional point to be encoded is used as the prediction value Va0. It may be encoded. In this case, "4" as the prediction mode value is added to the bit stream.
  • FIG. 75 is a diagram showing an eighth example of a table showing predicted values calculated in each prediction mode according to the fifth embodiment.
  • the eighth example is an example in which the attribute information used for the predicted value is the value of the U component and the value of the V component of the color information of the surrounding three-dimensional points.
  • the predicted values calculated in the predicted mode U whose predicted mode value is “0” are the values of the U components corresponding to the points b1, a2, a1 and a0, respectively, Ub1 and Ub2.
  • Ua1, Ua0, and the weighted average Wave of Vb1, Vb2, Va1, and Va0 which are the values of the V components corresponding to the points b1, a2, a1, and a0, respectively.
  • the predicted values calculated in the predicted modes whose prediction mode values are indicated by "1" to "4" are the values of the U component and the V component of the surrounding three-dimensional points b1, a2, a1, and a0, respectively. including.
  • the U component and the V component of the attribute value of the three-dimensional point to be encoded are predicted values, respectively. It may be encoded by using it as Ub1 or Vb1. In this case, "1" as the prediction mode value is added to the bit stream.
  • the prediction mode at the time of coding may be selected by RD optimization. For example, it is conceivable to calculate the cost cost (P) when a certain prediction mode P is selected and select the prediction mode P in which the cost (P) is minimized.
  • the cost cost (P) is adjusted with, for example, the predicted residual error (P) when the predicted value of the predicted mode P is used, the number of bits required to encode the predicted mode P bit (P), and the like. It may be calculated by the equation D1 using the parameter ⁇ value.
  • Abs (x) indicates the absolute value of x.
  • the square value of x may be used instead of abs (x).
  • the adjustment parameter ⁇ may be set to a different value depending on the value of the quantization scale. For example, when the quantization scale is small (at a high bit rate), the prediction mode in which the prediction residual residual (P) is small by reducing the ⁇ value is selected to improve the prediction accuracy as much as possible, and when the quantization scale is large. (At a low bit rate), an appropriate prediction mode may be selected by increasing the ⁇ value while considering the number of bits (P) required to encode the prediction mode P.
  • the case where the quantization scale is small is, for example, a case where it is smaller than the first quantization scale.
  • the case where the quantization scale is large is, for example, the case where it is larger than the second quantization scale which is equal to or larger than the first quantization scale.
  • the smaller the quantization scale the smaller the ⁇ value may be set.
  • the predicted residual error (P) is calculated by subtracting the predicted value of the prediction mode P from the attribute value of the three-dimensional point to be encoded. Instead of the predicted residual error (P) at the time of cost calculation, the predicted residual error (P) is quantized and inversely quantized, added to the predicted value to obtain the decoded value, and the attribute of the original three-dimensional point.
  • the difference (coding error) between the value and the decoded value when the prediction mode P is used may be reflected in the cost value. This makes it possible to select a prediction mode with a small coding error.
  • the number of bits (P) required to encode the prediction mode P may be, for example, the number of bits after binarization when the prediction mode is binarized and encoded.
  • the prediction mode value indicating the prediction mode may be binarized by a truncated unary code in which the maximum value is 5 using the number of prediction modes M.
  • 4 bits are used as the number of bits (P) required for encoding each prediction mode value.
  • the smaller the value of the prediction mode the smaller the number of bits. Therefore, the mean value calculated as the predicted value when the predicted mode value is "0", or the attribute information of the three-dimensional point calculated as the predicted value when the predicted mode value is "1", that is, the code.
  • the code amount of the prediction mode value indicating the prediction mode for calculating the prediction value that is easy to be selected, for example, the cost (P) is likely to be minimized, such as the attribute information of the three-dimensional point whose distance is close to the three-dimensional point to be converted. Can be reduced.
  • the prediction mode value indicating the prediction mode may be binarized by the unary code as shown in FIG. 77. Further, when the probability of occurrence of each prediction mode is close, as shown in FIG. 78, the prediction mode value indicating the prediction mode may be binarized by the fixed code to reduce the code amount.
  • the binarized data of the prediction mode value indicating the prediction mode P is arithmetically coded, and the code after arithmetic coding is performed.
  • the amount may be a bit (P) value.
  • FIG. 76 is a diagram showing a first example of a binarization table in the case where the prediction mode value according to the fifth embodiment is binarized and encoded.
  • FIG. 77 is a diagram showing a second example of the binarization table in the case where the prediction mode value according to the fifth embodiment is binarized and encoded.
  • FIG. 78 is a diagram showing a third example of the binarization table in the case where the prediction mode value according to the fifth embodiment is binarized and encoded.
  • the prediction mode value indicating the prediction mode may be arithmetically coded after binarization and added to the bit stream.
  • the prediction mode value may be binarized by, for example, a truncated unary code using a value of the number of prediction modes M. In this case, the maximum number of bits after binarization of the prediction mode value is M-1.
  • the binarized data may be arithmetically coded using a coding table.
  • the coding efficiency may be improved by switching the coding table for each bit of the binary data and coding.
  • the first bit one bit is encoded using the coding table A for one bit, and each bit of the remaining bits reminding bit is used for reminding bit. It may be encoded using the encoding table B.
  • the first bit “1” is encoded by using the encoding table A, and the remaining bits are being remaining.
  • Each bit of "110" of the bit may be encoded by using the encoding table B.
  • FIG. 79 is a diagram for explaining an example of encoding the binary data of the binarization table when the prediction mode according to the fifth embodiment is binarized and encoded.
  • the coding efficiency can be improved by switching the coding table according to the bit position of the binary data while suppressing the number of coding tables.
  • the coding table may be further switched for each bit to perform arithmetic coding, or the coding table may be switched and decoded according to the result of arithmetic coding.
  • the prediction mode used in the truncated unary code is used so that the prediction mode can be specified from the binary data decoded on the decoding side.
  • a number M may be added to the header of the bit stream or the like.
  • the number of prediction modes M may be specified by a profile or level such as a standard without being added to the stream.
  • the predicted mode value binarized using the thresholded unary code is arithmetically coded by switching the coding table between the one bit part and the reminding part as described above.
  • the probability of occurrence of 0 and 1 in each coding table may be updated according to the value of the binary data actually generated. Further, the probability of occurrence of 0 and 1 in either coding table may be fixed. As a result, the number of updates of the occurrence probability may be suppressed to reduce the processing amount. For example, the probability of occurrence of the one bit portion may be updated, and the probability of occurrence of the reminding bit portion may be fixed.
  • FIG. 80 is a flowchart showing an example of coding of the prediction mode value according to the fifth embodiment.
  • FIG. 81 is a flowchart showing an example of decoding the predicted mode value according to the fifth embodiment.
  • the prediction mode value is first binarized by a thresholded mode using the number of prediction modes M (S3401).
  • the binary data of the truncated unary code is arithmetically encoded (S3402).
  • the bitstream contains binary data as a prediction mode.
  • the bit stream is arithmetically decoded using the number of predicted modes M, and binary data of a truncated unary code is generated (S3403).
  • the prediction mode value is calculated from the binary data of the truncated unary code (S3404).
  • L L
  • L L
  • the binary value is more than the case of the truncated unary code with the number of prediction modes M.
  • the number of bits after conversion can be reduced.
  • the number of bits after binarization of the predicted mode value may be reduced by binarizing the number L assigned to the predicted mode as the maximum value in the thresholded number code.
  • the binarized data may be arithmetically coded using a coding table.
  • the coding efficiency may be improved by switching the coding table for each bit of the binary data and coding.
  • the first bit one bit is encoded using the coding table A for one bit, and each bit of the remaining bits remaining bit is used for remaining bit. It may be encoded using the encoding table B of. For example, when encoding the binarized data “11” whose prediction mode value shown in FIG. 83 is “2”, the first bit “1” is encoded by using the encoding table A, and the remaining bits are being remaining.
  • the bit "1" may be encoded using the coding table B.
  • FIG. 83 is a diagram for explaining an example of encoding the binary data of the binarization table when the prediction mode according to the fifth embodiment is binarized and encoded.
  • the coding efficiency can be improved by switching the coding table according to the bit position of the binary data while suppressing the number of coding tables.
  • the coding table may be further switched for each bit to perform arithmetic coding, or the coding table may be switched and decoded according to the result of arithmetic coding.
  • the prediction mode value is binarized and encoded by the truncated unary code using the number L to which the predicted value is assigned
  • the prediction mode is specified at the time of encoding so that the predicted mode can be specified from the binary data decoded on the decoding side.
  • the number L may be calculated by assigning the predicted value to the predicted mode in the same manner, and the predicted mode may be decoded using the calculated L.
  • the predicted mode value binarized using the thresholded unary code is arithmetically coded by switching the coding table between the one bit part and the reminding part as described above.
  • the probability of occurrence of 0 and 1 in each coding table may be updated according to the value of the binary data actually generated. Further, the probability of occurrence of 0 and 1 in either coding table may be fixed. As a result, the number of updates of the occurrence probability may be suppressed to reduce the processing amount. For example, the probability of occurrence of the one bit portion may be updated, and the probability of occurrence of the reminding bit portion may be fixed.
  • FIG. 84 is a flowchart showing another example of coding the prediction mode value according to the fifth embodiment.
  • FIG. 85 is a flowchart showing another example of decoding the predicted mode value according to the fifth embodiment.
  • the number L to which the prediction value is assigned to the prediction mode is calculated (S3411).
  • the prediction mode value is binarized by a truncated unary code using the number L (S3412).
  • the number L in which the predicted value is assigned to the predicted mode is calculated (S3414).
  • bit stream is arithmetically decoded using the number L to generate binary data of a truncated unary code (S3415).
  • the prediction mode value is calculated from the binary data of the truncated unary code (S3416).
  • the prediction mode value does not have to be added for each attribute value. For example, if certain conditions are met, the prediction mode is fixed so that the prediction mode value is not added to the bitstream, and if certain conditions are not met, the prediction mode is selected and the prediction mode value is added to the bitstream. You may. For example, if the condition A is satisfied, the prediction mode value is fixed to "0" and the prediction value is calculated from the average value of the surrounding three-dimensional points. If the condition A is not satisfied, one prediction mode is selected from a plurality of prediction modes. Then, a prediction mode value indicating the selected prediction mode may be added to the bit stream.
  • the maximum absolute of the attribute values (a [0] to a [N-1]) of the surrounding N (encoded and decoded) three-dimensional points at the three-dimensional point to be encoded is calculated, and the maximum absolute difference value maxdiff is less than the threshold value Thfix.
  • Thfix the threshold value of the attribute values of the surrounding 3D points
  • the threshold value Thfix may be added to the header of the bit stream or the like, and the encoder may change the value of the threshold value Thfix so that it can be encoded. For example, when encoding at a high bit rate, the encoder adds a threshold Thfix value to the header smaller than that at a low bit rate, and selects a prediction mode to increase the number of cases of coding. It may be encoded so that the predicted residual is small. Further, the encoder adds the value of the threshold value Thfix to the header at the time of coding at the low bit rate to be larger than that at the time of the high bit rate, and encodes with the prediction mode fixed.
  • the threshold value Thfix may be specified by a standard profile or level without being added to the bit stream.
  • the N 3D points around the 3D point to be coded used for prediction are N coded and decoded 3D points whose distance from the 3D point to be coded is smaller than the threshold THd. It is a point.
  • the maximum value of N may be added to the bitstream as a NumberNightborPoint.
  • the value of N does not always have to match the value of NuNeightborPoint, such as when the surrounding coded and decoded 3D points are less than the value of NuNeightborPoint.
  • the prediction mode value is fixed to "0" if the maximum absolute difference value maxdiff of the attribute values of the three-dimensional points around the three-dimensional point to be encoded, which is used for prediction, is smaller than the threshold Thfix [i].
  • the present invention is not necessarily limited to this, and the prediction mode value may be fixed to any of "0" to "M-1". Further, a fixed prediction mode value may be added to the bitstream.
  • FIG. 86 is a flowchart showing an example of a process of determining whether or not to fix the prediction mode value according to the condition A at the time of coding according to the fifth embodiment.
  • FIG. 87 is a flowchart showing an example of a process of determining whether to set the prediction mode value to a fixed value or to decode according to the condition A at the time of decoding according to the fifth embodiment.
  • the three-dimensional data coding apparatus calculates the maximum absolute difference value maxdiff of the attribute values at the N three-dimensional points around the three-dimensional point to be encoded (S3421).
  • the three-dimensional data coding device determines whether or not the maximum absolute difference value maxdiff is less than the threshold value Thfix (S3422).
  • the threshold value Thfix may be encoded and added to the header of the stream or the like.
  • the three-dimensional data coding device determines the prediction mode value to "0" (S3423).
  • the three-dimensional data coding apparatus selects one prediction mode from a plurality of prediction modes (S3424). Details of the prediction mode selection process will be described later with reference to FIG. 94.
  • the three-dimensional data coding device arithmetically encodes the prediction mode value indicating the selected prediction mode (S3425). Specifically, the three-dimensional data coding apparatus arithmetically encodes the prediction mode value by executing steps S3401 and S3402 described with reference to FIG. 80. In addition, the three-dimensional data coding apparatus may perform arithmetic coding by binarizing the prediction mode PredMode with a truncated unary code using the number of prediction modes assigned by the prediction value. That is, the three-dimensional data coding apparatus may arithmetically code the prediction mode value by executing steps S3411 to S3413 described with reference to FIG. 84.
  • the three-dimensional data coding apparatus calculates the predicted value of the prediction mode determined in step S3423 or the prediction mode selected in step S3425, and outputs the calculated predicted value (S3426).
  • the prediction mode value is the prediction value of the prediction mode indicated by "0", and the attribute values of the surrounding N three-dimensional points are used. Calculate the average.
  • the three-dimensional data decoding device calculates the maximum absolute difference value maxdiff of the attribute values at the N three-dimensional points around the three-dimensional point to be decoded (S3431).
  • the maximum absolute difference value maxdiff may be calculated as shown in FIG. 88.
  • the maximum absolute difference value maxdiff is, for example, a plurality of calculated absolute values of the differences of all the pairs that can be taken when two of the surrounding N three-dimensional points are selected as pairs. It is the maximum value among the absolute values.
  • the three-dimensional data decoding device determines whether or not the maximum absolute difference value maxdiff is less than the threshold value Thfix (S3432).
  • the threshold value Thfix may be set by decoding the header of the stream or the like.
  • the three-dimensional data decoding device determines the prediction mode value to "0" when the maximum absolute difference value maxdiff is less than the threshold value Thfix (Yes in S3432).
  • the three-dimensional data decoding device decodes the prediction mode value from the bit stream when the maximum absolute difference value maxdiff is equal to or greater than the threshold value Thfix (No in S3432).
  • the three-dimensional data decoding device calculates the predicted mode value determined in step S3433 or the predicted mode value indicated by the predicted mode value decoded in step S3434, and outputs the calculated predicted value (S3435). ).
  • the prediction mode value is the average of the attribute values of the surrounding N three-dimensional points as the prediction mode value indicated by "0". Is calculated.
  • FIG. 89 is a diagram showing an example of the syntax according to the fifth embodiment. NumLoD, NuNeigborPoint [i], NupredMode [i], Thfix [i], and NuOfPoint [i] in the syntax of FIG. 89 will be described in order.
  • NumLoD indicates the number of layers of LoD.
  • NumNeigborPoint [i] indicates the upper limit of the number of surrounding points used to generate the predicted value of the three-dimensional points belonging to the layer i.
  • the peripheral score M is less than NumNightborPoint [i] (M ⁇ NumNighborPoint [i])
  • the predicted value may be calculated using M peripheral points.
  • one NuNeighborPoint may be added to the header.
  • NumPredMode [i] indicates the total number (M) of prediction modes used for predicting the attribute value of the layer i.
  • the value of the number of prediction modes NumPredMode [i] may be specified by a profile or level such as a standard without being added to the stream. Further, the number of prediction modes may be defined by NumNeigborPoint [i] + NumPredMode [i]. Further, when it is not necessary to separate the value of NumPredMode [i] for each LoD, one NumPredMode may be added to the header.
  • Thfix [i] indicates a threshold value of the maximum absolute difference value for determining whether or not to fix the prediction mode of the layer i. If the maximum absolute difference value of the attribute values of the surrounding three-dimensional points used for prediction is smaller than Thfix [i], the prediction mode is fixed at 0. Note that Thfix [i] may be specified by a profile or level such as a standard without being added to the stream. Further, when it is not necessary to divide the value of Thfix [i] for each LoD, one Thfix may be added to the header.
  • NuMofPoint [i] indicates the number of three-dimensional points belonging to the layer i.
  • NuMofPoint [NumMoD-1] (the number of 3D points belonging to the lowest layer) is not added to the header.
  • NumPredMode [i] As a setting example of NumPredMode [i], even if the value of NumPredMode [i] is set larger for the upper layer, which is difficult to predict because the three-dimensional points belonging to LoD are far apart, the selectable prediction modes are increased. Good. Further, the lower layer, which is easier to predict, may be set to have a smaller value of NumPredMode [i] to suppress the amount of bits required for coding in the prediction mode. With these settings, the number of selectable prediction modes can be increased in the upper layer to reduce the prediction residual, and the amount of code in the prediction mode can be suppressed in the lower layer to improve the coding efficiency.
  • Thfix [i] the higher the layer, which is difficult to predict because the three-dimensional points belonging to LoD are far apart, the smaller the value of Thfix [i] is set, and the number of cases where the prediction mode is selected is increased. You may.
  • the lower layer, which is easier to predict may have a larger Thfix [i] value, and the prediction mode may be fixed to suppress the amount of bits required for coding the prediction mode.
  • the NumMoD, NumNeigborPoint [i], NumPredMode [i], Thfix [i] or NumOfPoint [i] may be entropy-encoded and added to the header.
  • each value may be binarized and arithmetically coded. Further, each value may be encoded with a fixed length in order to reduce the amount of processing.
  • FIG. 90 is a diagram showing an example of the syntax according to the fifth embodiment.
  • the PredMode, n-bit code, and reminding code in the syntax of FIG. 90 will be described in order.
  • PredMode indicates a prediction mode for encoding and decoding the attribute value of the j-th three-dimensional point of the layer i.
  • the PredMode is not limited to "0", and any value from "0" to "M-1" may be used as the estimated value. Further, the estimated value when the PredMode is not in the bit stream may be added to the header or the like separately. Further, the PredMode may be binarized and arithmetically coded by a truncated unary code using the number of prediction modes assigned by the prediction value.
  • the n-bit code indicates the coded data of the predicted residual of the value of the attribute information.
  • the bit length of the n-bit code depends on the value of R_TH [i].
  • the bit length of the n-bit code is, for example, 6 bits when the value indicated by R_TH [i] is 63, and 8 bits when the value indicated by R_TH [i] is 255.
  • the reminding code indicates the coded data encoded by the Exponential-Golomb among the coded data of the predicted residuals of the value of the attribute information.
  • the reminding code is decoded when the n-bit code is the same as R_TH [i], and the predicted residual is decoded by adding the value of the n-bit code and the value of the reminding code. If the n-bit code is not the same value as R_TH [i], the reminding code does not have to be decoded.
  • FIG. 91 is a flowchart of the three-dimensional data coding process by the three-dimensional data coding apparatus according to the fifth embodiment.
  • the three-dimensional data coding device encodes the position information (geometry) (S3441). For example, in three-dimensional data coding, coding is performed using an octree representation.
  • the 3D data coding device When the position of the 3D point changes due to quantization or the like after the position information is coded, the 3D data coding device reassigns the attribute information of the original 3D point to the changed 3D point ( S3442). For example, the three-dimensional data coding device reassigns by interpolating the value of the attribute information according to the amount of change in the position. For example, the three-dimensional data encoding device detects N three-dimensional points before the change, which are close to the three-dimensional position after the change, and weights and averages the values of the attribute information of the N three-dimensional points. For example, a three-dimensional data coding device determines weights in a weighted average based on the distances from the changed three-dimensional position to each of the N three dimensions.
  • the three-dimensional data encoding device determines the value obtained by the weighted averaging as the value of the attribute information of the three-dimensional point after the change.
  • the three-dimensional data encoding device uses the value of the attribute information of the three-dimensional points after the change as the value before the change.
  • the average value of the attribute information of two or more three-dimensional points may be assigned.
  • the three-dimensional data coding device encodes the attribute information (Attribute) after reassignment (S3443).
  • the three-dimensional data coding apparatus may encode the plurality of types of attribute information in order.
  • the three-dimensional data coding apparatus may generate a bit stream in which the color coding result and the reflectance coding result are added after the color coding result when the color and the reflectance are encoded as the attribute information. ..
  • the order of the plurality of coding results of the attribute information added to the bit stream is not limited to this order, and may be any order.
  • the three-dimensional data encoding device may add information indicating the encoded data start location of each attribute information in the bit stream to the header or the like.
  • the three-dimensional data decoding device can selectively decode the attribute information that needs to be decoded, so that the decoding process of the attribute information that does not need to be decoded can be omitted. Therefore, the processing amount of the three-dimensional data decoding device can be reduced.
  • the three-dimensional data coding apparatus may encode a plurality of types of attribute information in parallel and integrate the coding results into one bit stream. As a result, the three-dimensional data coding device can encode a plurality of types of attribute information at high speed.
  • FIG. 92 is a flowchart of the attribute information coding process (S3443) according to the fifth embodiment.
  • the three-dimensional data coding device sets LoD (S3451). That is, the three-dimensional data coding device assigns each three-dimensional point to any of a plurality of LoDs.
  • the three-dimensional data coding device starts a loop in units of LoD (S3452). That is, the three-dimensional data coding apparatus repeats the processes of steps S3453 to S3461 for each LoD.
  • the three-dimensional data coding device starts a loop in units of three-dimensional points (S3453). That is, the three-dimensional data coding apparatus repeats the processes of steps S3454 to S3460 for each three-dimensional point.
  • the three-dimensional data encoding device searches for a plurality of peripheral points that are three-dimensional points existing around the target three-dimensional point used for calculating the predicted value of the target three-dimensional point to be processed (S3454).
  • the three-dimensional data coding device calculates the predicted value P of the target three-dimensional point (S3455).
  • a specific example of the calculation process of the predicted value P will be described later with reference to FIG. 93.
  • the three-dimensional data encoding device calculates the predicted residual, which is the difference between the attribute information of the target three-dimensional point and the predicted value (S3456).
  • the three-dimensional data coding device calculates the quantized value by quantizing the predicted residual (S3457).
  • the three-dimensional data coding device arithmetically encodes the quantized value (S3458).
  • the three-dimensional data coding device calculates the dequantized value by dequantizing the quantized value (S3459).
  • the three-dimensional data encoding device generates a decoded value by adding the predicted value to the inverse quantization value (S3460).
  • the three-dimensional data coding device ends the loop in units of three-dimensional points (S3461).
  • the three-dimensional data coding device ends the loop in units of LoD (S3462).
  • FIG. 93 is a flowchart of the predicted value calculation process (S3455) in the three-dimensional data coding apparatus according to the fifth embodiment.
  • the 3D data encoding device calculates a weighted average of the attribute values of N 3D points around the target 3D point, which can be used to predict the predicted value of the target 3D point to be processed. , The calculated weighted average is assigned to the prediction mode in which the prediction mode value is “0” (S3420).
  • the three-dimensional data coding device outputs the predicted value of the target three-dimensional point by executing steps S3421 to S3426 described with reference to FIG. 86.
  • FIG. 94 is a flowchart of the prediction mode selection process (S3424) according to the fifth embodiment.
  • the three-dimensional data encoding device sets the attribute information of N three-dimensional points around the target three-dimensional point as prediction mode values from "1" to "N” in order from the one closest to the target three-dimensional point.
  • the value incremented by 1 is assigned up to (S3427). Since the prediction mode values are assigned from "0" to "N", a total of N + 1 prediction modes are generated.
  • N + 1 exceeds the maximum number of prediction modes M (NumPredMode) added to the bit stream, the three-dimensional data encoding device may generate up to M prediction modes.
  • the three-dimensional data coding device calculates the cost of each prediction mode and selects the prediction mode that minimizes the cost (S3428).
  • FIG. 95 is a flowchart of a process (S3428) for selecting a prediction mode that minimizes the cost according to the fifth embodiment.
  • the set initial values of i and mincost are stored in the memory of the three-dimensional data encoding device.
  • the three-dimensional data coding device calculates the cost cost [i] of the i-th prediction mode PredMode [i] using, for example, the formula D1 (S3472).
  • the three-dimensional data coding device determines whether or not the calculated cost cost [i] is smaller than the mincost stored in the memory (S3473).
  • step S3475 when the calculated cost cost [i] is equal to or higher than the mincost stored in the memory (No in S3473), the three-dimensional data coding apparatus proceeds to step S3475.
  • the three-dimensional data coding device increments the value of i by one (S3475).
  • the three-dimensional data coding device determines whether or not i is smaller than the number of prediction modes (S3476).
  • the three-dimensional data coding apparatus returns to step S3472, and if not (No in S3476), ends the process of selecting the prediction mode that minimizes the cost. ..
  • FIG. 96 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding apparatus according to the fifth embodiment.
  • the three-dimensional data decoding device decodes the position information (geometry) from the bit stream (S3444). For example, a three-dimensional data decoding device decodes using an ocree representation.
  • the three-dimensional data decoding device decodes the attribute information (Attribute) from the bit stream (S3445). For example, when decoding a plurality of types of attribute information, the three-dimensional data decoding device may decode the plurality of types of attribute information in order. For example, when decoding a color and a reflectance as attribute information, the three-dimensional data decoding apparatus decodes the color coding result and the reflectance coding result in the order of being added to the bit stream. For example, in a bitstream, when the reflectance coding result is added after the color coding result, the three-dimensional data decoding device decodes the color coding result, and then the reflectance coding result. To decrypt. The three-dimensional data decoding device may decode the coding result of the attribute information added to the bit stream in any order.
  • the three-dimensional data decoding device may acquire information indicating the encoded data start location of each attribute information in the bit stream by decoding the header or the like. As a result, the three-dimensional data decoding device can selectively decode the attribute information that needs to be decoded, so that the decoding process of the attribute information that does not need to be decoded can be omitted. Therefore, the processing amount of the three-dimensional data decoding device can be reduced. Further, the three-dimensional data decoding device may decode a plurality of types of attribute information in parallel and integrate the decoding results into one three-dimensional point cloud. As a result, the three-dimensional data decoding device can decode a plurality of types of attribute information at high speed.
  • FIG. 97 is a flowchart of the attribute information decoding process (S3445) according to the fifth embodiment.
  • the three-dimensional data decoding device sets LoD (S3481). That is, the three-dimensional data decoding device assigns each of the plurality of three-dimensional points having the decoded position information to any of the plurality of LoDs.
  • this allocation method is the same as the allocation method used in the three-dimensional data coding apparatus.
  • the three-dimensional data decoding device starts a loop in units of LoD (S3482). That is, the three-dimensional data decoding device repeats the processes of steps S3483 to S3489 for each LoD.
  • the three-dimensional data decoding device starts a loop in units of three-dimensional points (S3483). That is, the three-dimensional data decoding apparatus repeats the processes of steps S3484 to S3488 for each three-dimensional point.
  • the three-dimensional data decoding device searches for a plurality of peripheral points that are three-dimensional points existing around the target three-dimensional point used for calculating the predicted value of the target three-dimensional point to be processed (S3484). This process is the same as the process in the three-dimensional data coding apparatus.
  • the three-dimensional data decoding device calculates the predicted value P of the target three-dimensional point (S3485).
  • the three-dimensional data decoding device arithmetically decodes the quantized value from the bit stream (S3486).
  • the three-dimensional data decoding device calculates the dequantized value by dequantizing the decoded quantization value (S3487).
  • the three-dimensional data decoding device generates a decoded value by adding the predicted value to the inverse quantization value (S3488).
  • the three-dimensional data decoding device ends the loop in units of three-dimensional points (S3489).
  • the three-dimensional data decoding device ends the loop in units of LoD (S3490).
  • FIG. 98 is a flowchart of the predicted value calculation process (S3485) in the three-dimensional data decoding apparatus according to the fifth embodiment.
  • the 3D data decoding device calculates a weighted average of the attribute values of N 3D points around the target 3D point, which can be used to predict the predicted value of the target 3D point to be processed.
  • the calculated weighted average is assigned to the prediction mode in which the prediction mode value is “0” (S3430).
  • the three-dimensional data decoding device outputs the predicted value of the target three-dimensional point by executing steps S3431 to S3435 described with reference to FIG. 87.
  • step S3430 if Yes is determined in step S3432, or if the predicted mode value decoded in step S3434 is "0", the predicted value of the target three-dimensional point to be processed.
  • the weighted average of the attribute values of N three-dimensional points around the target three-dimensional point, which can be used for the prediction of, may be calculated as the prediction value.
  • FIG. 99 is a flowchart of the prediction mode decoding process (S3434) according to the fifth embodiment.
  • the three-dimensional decoding device sets the attribute information of N three-dimensional points around the target three-dimensional point as prediction mode values from "1" to "N” in order from the one closest to the target three-dimensional point.
  • the value incremented by each is assigned (S3491). Since the prediction mode values are assigned from "0" to "N", a total of N + 1 prediction modes are generated.
  • N + 1 exceeds the maximum number of prediction modes M (NumPredMode) added to the bit stream, the three-dimensional data decoding apparatus may generate up to M prediction modes.
  • the three-dimensional data decoding device arithmetically decodes the prediction mode using the number of prediction modes (S3492). Specifically, the three-dimensional data decoding apparatus may arithmetically decode the prediction mode by executing steps S3403 and S3404 described with reference to FIG. 81. Further, the three-dimensional data decoding apparatus may arithmetically decode the prediction mode by executing steps S3414 to S3416 described with reference to FIG. 85.
  • FIG. 100 is a block diagram showing the configuration of the attribute information coding unit 3400 included in the three-dimensional data coding device according to the fifth embodiment. Note that FIG. 100 shows the details of the attribute information coding unit among the position information coding unit, the attribute information reassignment unit, and the attribute information coding unit included in the three-dimensional data coding device. ..
  • the attribute information coding unit 3400 includes a LoD generation unit 3401, a surrounding search unit 3402, a prediction unit 3403, a prediction residual calculation unit 3404, a quantization unit 3405, an arithmetic coding unit 3406, and an inverse quantization unit. It includes a 3407, a decoding value generation unit 3408, and a memory 3409.
  • the LoD generation unit 3401 generates LoD using the position information (geometry) of the three-dimensional point.
  • the surrounding search unit 3402 searches for neighboring three-dimensional points adjacent to each three-dimensional point by using the LoD generation result by the LoD generation unit 3401 and the distance information indicating the distance between each three-dimensional point.
  • the prediction unit 3403 generates a prediction value of the attribute information of the target three-dimensional point to be encoded. Specifically, the prediction unit 3403 assigns the prediction value to the prediction mode whose prediction mode value is indicated by "0" to "M-1", and selects the prediction mode. The prediction unit 3403 outputs the selected prediction mode, specifically, the prediction mode value indicating the prediction mode to the arithmetic coding unit. The prediction unit 3403 performs, for example, the process of step S3455.
  • the prediction residual calculation unit 3404 calculates (generates) the prediction residual of the predicted value of the attribute information generated by the prediction unit 3403.
  • the prediction residual calculation unit 3404 performs the process of step S3456.
  • the quantization unit 3405 quantizes the predicted residual of the attribute information calculated by the predicted residual calculation unit 3404.
  • the arithmetic coding unit 3406 arithmetically encodes the predicted residual after being quantized by the quantization unit 3405.
  • the arithmetic coding unit 3406 outputs a bit stream including the arithmetically coded predicted residuals to, for example, a three-dimensional data decoding device.
  • the predicted residual may be binarized by, for example, the quantization unit 3405 before being arithmetically coded by the arithmetic coding unit 3406.
  • the arithmetic coding unit 3406 may generate and encode various header information. Further, the arithmetic coding unit 3406 may acquire the prediction mode used for coding from the Precision block, perform arithmetic coding, and add it to the bit stream.
  • the inverse quantization unit 3407 dequantizes the predicted residual after being quantized by the quantization unit 3405.
  • the inverse quantization unit 3407 performs the process of step S3459.
  • the decoding value generation unit 3408 generates a decoding value by adding the predicted value of the attribute information generated by the prediction unit 3403 and the predicted residual after dequantization by the inverse quantization unit 3407.
  • the memory 3409 is a memory that stores the decoded value of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3408. For example, when the prediction unit 3403 generates a predicted value of an unencoded three-dimensional point, the prediction unit 3403 generates the predicted value by using the decoded value of the attribute information of each three-dimensional point stored in the memory 3409. ..
  • FIG. 101 is a block diagram showing a configuration of an attribute information decoding unit 3410 included in the three-dimensional data decoding device according to the fifth embodiment. Note that FIG. 101 shows the details of the attribute information decoding unit among the position information decoding unit and the attribute information decoding unit included in the three-dimensional data decoding device.
  • the attribute information decoding unit 3410 includes a LoD generation unit 3411, a surrounding search unit 3412, a prediction unit 3413, an arithmetic decoding unit 3414, an inverse quantization unit 3415, a decoding value generation unit 3416, and a memory 3417. ..
  • the LoD generation unit 3411 generates LoD using the position information (geometry information) of the three-dimensional points decoded by the position information decoding unit (not shown).
  • the surrounding search unit 3412 searches for neighboring three-dimensional points adjacent to each three-dimensional point by using the LoD generation result by the LoD generation unit 3411 and the distance information indicating the distance between each three-dimensional point.
  • the prediction unit 3413 generates a prediction value of the attribute information of the target three-dimensional point to be decoded.
  • the prediction unit 3413 performs, for example, the process of step S3485.
  • the arithmetic decoding unit 3414 arithmetically decodes the predicted residual in the bit stream acquired from the attribute information coding unit 3400.
  • the arithmetic decoding unit 3414 may decode various header information. Further, the arithmetic decoding unit 3414 may output the arithmetically decoded prediction mode to the prediction unit 3413. In this case, the prediction unit 3413 may calculate the prediction value using the prediction mode obtained by arithmetically decoding in the arithmetic decoding unit 3414.
  • the inverse quantization unit 3415 dequantizes the predicted residuals arithmetically decoded by the arithmetic decoding unit 3414.
  • the decoding value generation unit 3416 adds the prediction value generated by the prediction unit 3413 and the prediction residual after dequantization by the inverse quantization unit 3415 to generate a decoding value.
  • the decoded value generation unit 3416 outputs the decoded attribute information data to another device.
  • the memory 3417 is a memory that stores the decoded value of the attribute information of each three-dimensional point decoded by the decoded value generation unit 3416. For example, when the prediction unit 3413 generates the predicted value of the three-dimensional point that has not been decoded yet, the prediction unit 3413 generates the predicted value by using the decoded value of the attribute information of each three-dimensional point stored in the memory 3417. ..
  • the 3D data coding device has N 3D points around the target 3D point to be encoded, which can be used for prediction in each of the 3D data coding device and the 3D data decoding device.
  • the maximum absolute difference value of the attribute values at (surrounding three-dimensional points) (that is, the maximum absolute value of the difference between the attribute values of any two three-dimensional points out of the N three-dimensional points) is calculated.
  • the three-dimensional data encoding device fixes the prediction mode according to the calculated value, that is, whether to use a prediction mode arbitrarily determined in advance or to select a prediction mode and add it to the bit stream. Switch.
  • the three-dimensional data coding device does not have to switch between fixing the prediction mode according to the maximum absolute difference value or selecting the prediction mode and adding it to the bit stream.
  • the three-dimensional data coding apparatus may select whether to fix the prediction mode or select the prediction mode under the above conditions, and add the result to the bit stream as a prediction mode fixing flag.
  • the 3D data decoding device decodes the prediction mode fixed flag added to the bitstream to either fix the prediction mode by the 3D data encoding device or select the prediction mode to code the prediction mode. It can be determined whether or not it has been converted.
  • the three-dimensional data decoding device can determine that the prediction mode is not encoded in the bit stream.
  • the 3D data decoding apparatus can determine that it is necessary to decode the prediction mode of the bitstream when the prediction mode is selected by the 3D data encoding apparatus. This allows the 3D data decoder to correctly decode the coded prediction mode contained in the bitstream.
  • the three-dimensional data decoding device predicts without calculating the maximum absolute difference value of the attribute values at the N three-dimensional points around the target three-dimensional point to be encoded, which can be used for prediction.
  • the mode can be arithmetically decoded.
  • the three-dimensional data decoding device can execute the arithmetic decoding of the bitstream, the LoD generation, and the like in parallel. Therefore, the three-dimensional data decoding device can improve the overall throughput of the processing.
  • the three-dimensional data coding device may add a prediction mode fixed flag for each three-dimensional point.
  • the three-dimensional data decoding device can switch whether to fix or select the prediction mode for each three-dimensional point based on the prediction mode fixing flag. Therefore, the three-dimensional data coding apparatus can improve the coding efficiency.
  • the three-dimensional data coding device may set the prediction mode fixed flag for each LoD. For example, the three-dimensional data encoding device sets the prediction mode fixing flag to 0 in the upper layer of each layer of LoD where prediction is difficult to hit, so that the prediction mode can be selected, and the prediction mode is fixed in the lower layer where prediction is easy to hit. Set the flag to 1 to fix the prediction mode. As a result, the three-dimensional data coding apparatus can reduce the amount of code to which the prediction mode is added.
  • FIG. 102 is a flowchart showing a process of determining a prediction mode executed by the three-dimensional data coding apparatus according to the present embodiment.
  • the three-dimensional data coding device calculates the maximum absolute difference value maxdiff of the attribute values at the N three-dimensional points around the target three-dimensional point to be encoded (S3501).
  • FIG. 103 is a diagram showing a syntax example of processing for determining a prediction mode executed by the three-dimensional data coding apparatus according to the present embodiment. Specifically, FIG. 103 is a diagram showing an example of syntax in step S3501 of FIG. 102.
  • the attribute values around the target three-dimensional point to be encoded are indicated by a [0] to a [N-1], and the maximum absolute difference value is indicated by maxdiff.
  • the N attribute values around the target three-dimensional point to be encoded are the attribute values encoded by the three-dimensional data encoding device and decoded by the three-dimensional data decoding device.
  • the three-dimensional data coding device calculates the maximum absolute difference value maxdiff according to the syntax example shown in FIG. 103, for example.
  • the N three-dimensional points around the target three-dimensional point to be coded which are used for prediction by the three-dimensional data coding device, are coded so that the distance from the target three-dimensional point to be coded is smaller than the threshold THd. These are N three-dimensional points that have been and have been decoded.
  • the three-dimensional data coding device may add the maximum value of N to the bit stream as a NuNeilightborPoint.
  • the score of the coded and decoded 3D point in the attribute information of the 3D point around the target 3D point to be coded is less than the value of NuNeightborPoint, the value of N is the value of NuNeightborPoint. Does not have to match.
  • the three-dimensional data coding device determines whether or not maxdiff ⁇ Thfix is satisfied (S3502).
  • Thfix is a constant that is arbitrarily determined in advance.
  • the three-dimensional data coding device sets the prediction mode fixed flag (fixedPredMode) to 1 and performs arithmetic coding (S3503).
  • the three-dimensional data coding device determines the value of the prediction mode (PredMode / hereinafter simply referred to as the prediction mode) to 0 (S3504).
  • the prediction mode fixed flag is set to 0 and arithmetic coding is performed (S3505).
  • the three-dimensional data coding device selects the prediction mode (S3506).
  • the three-dimensional data coding device arithmetically encodes the selected prediction mode (S3507).
  • the three-dimensional data device sets the prediction mode to 0 if the maximum absolute difference value of the attribute values of the three-dimensional points around the target three-dimensional point to be encoded, which is used for prediction, is smaller than Thfix [i].
  • An example of fixing is shown, but it is not necessarily limited to this.
  • the three-dimensional data coding device may fix the prediction mode to any of 0 to M-1.
  • the three-dimensional data coding device may add a fixed prediction mode value (also referred to as PredMode or mode number) to the bit stream.
  • a fixed prediction mode value also referred to as PredMode or mode number
  • FIG. 104 is a flowchart showing a process of determining a prediction mode executed by the three-dimensional data decoding apparatus according to the present embodiment.
  • the three-dimensional data decoding device arithmetically decodes the encoded prediction mode fixed flag included in the bit stream (S3511).
  • the value of the prediction mode is determined to be 0 (S3513).
  • the three-dimensional data decoding device determines the predicted value based on the determined prediction mode.
  • the prediction mode fixed flag may be provided at any position.
  • FIG. 105 is a diagram showing an example of syntax of attribute data when a prediction mode fixed flag is provided for each three-dimensional point.
  • the fixedPredMode is a flag indicating whether or not the three-dimensional data coding device fixes the prediction mode. For example, when the value of fixedPredMode is 1, the 3D data coding device may fix the prediction mode, and when the value of fixedPredMode is 0, the 3D data coding device may select the prediction mode.
  • the fixedPredMode may be set for each LoD layer as shown in FIG. 106.
  • FIG. 106 is a diagram showing an example of syntax of attribute data when a prediction mode fixed flag is provided for each LoD layer.
  • FixedPredMode [i] is a flag indicating whether or not to fix the prediction mode of the layer i of LoD.
  • PredMode is a value indicating a prediction mode for encoding and decoding the attribute value of the j-th three-dimensional point of the layer i.
  • the PredMode takes a value from 0 to M-1 (where M is the total number of prediction modes).
  • the three-dimensional data decoding device may estimate the PredMode to 0 when the PredMode is not in the bitstream (specifically, when! FixedPredMode && NumPredMode [i]> 1 is not satisfied).
  • the three-dimensional data decoding device does not have to set the PredMode to 0, and may adopt any value from 0 to M-1 as an estimated value.
  • the three-dimensional data encoding device may separately add an estimated value when the PredMode is not in the bit stream to the header or the like.
  • the three-dimensional data encoding device binarizes the PredMode with the value of the total number of prediction modes M with a truncated unity code, and the binarized value is an arithmetic code. It may be transformed into.
  • the truncated utility code is one of the binarization methods.
  • the truncated utility code for a multi-valued signal having a value other than the maximum value, the same number of 1's as the value indicated by the multi-valued signal are arranged, and a signal with 0 added at the end is generated.
  • the maximum value is set in advance, and a signal in which the same number of 1s as the value indicated by the multi-valued signal are arranged (a signal in which 0 is not added at the end) is generated.
  • each prediction mode is set to the prediction mode. From 0 to 4, the numbers are 0, 10, 110, 1110, and 1111 in that order.
  • the 3D data encoder uses the number of prediction modes (ie, depending on the number of prediction modes) to binarize the prediction modes with a truncated utility sign.
  • the same number of 0s as the value indicated by the multi-valued signal may be arranged, and a signal with 1 added at the end may be generated. .. That is, the above 0 and 1 may be inverted.
  • the three-dimensional data coding device may arithmetically code and add the value of the total number of prediction modes M to the header as NupredMode.
  • NumPredMode is a value indicating the total number of prediction modes.
  • the three-dimensional data decoding device can decode the NumPredMode in the header to calculate the total number of prediction modes M, and can decode the PredMode using the total number of prediction modes M. Therefore, the three-dimensional data decoding device sets (generates) LoD, calculates the three-dimensional points around the target three-dimensional point to be decoded, and calculates the number of prediction modes to which the predicted value is assigned. Bitstream arithmetic decryption can be performed without waiting for. As a result, the three-dimensional data decoding apparatus can execute the arithmetic decoding of the bit stream and the LoD generation in parallel, so that the overall throughput of the processing can be improved.
  • n-bit code (n-bit code) is encoded data of the predicted residual of the value of the attribute information.
  • the bit length of the n-bit code depends on the value of R_TH [i]. For example, when the value indicated by R_TH [i] is 63, it is 6 bits, and when the value indicated by R_TH [i] is 255, it is 8 bits.
  • the value indicated by R_TH may be arbitrarily determined in advance.
  • the remaining code is the coded data encoded by the exponential Golomb code among the coded data of the predicted residuals of the value of the attribute information.
  • the three-dimensional data decoding apparatus decodes the remaining code when the n-bit code is the same as R_TH [i], adds the value of the n-bit code and the value of the remaining code, and decodes the predicted residual.
  • the three-dimensional data decoding device does not have to decode the remaining code.
  • FIG. 107 is a flowchart showing an example of coding processing in the prediction mode of the three-dimensional data coding apparatus.
  • the three-dimensional data coding device binarizes the prediction mode with a transcribed utility code using the total number of prediction modes M (S3521).
  • the three-dimensional data encoding device arithmetically encodes the binarized data of the truncated utility code (S3522).
  • the three-dimensional data coding device adds the total number of prediction modes M to the header as a NumPredMode and encodes it (S3523).
  • the three-dimensional data coding device transmits, for example, a bit stream including the encoded NupredMode to the three-dimensional data decoding device.
  • FIG. 108 is a flowchart showing an example of decoding processing in the prediction mode of the three-dimensional data decoding device.
  • the three-dimensional data decoding device decodes the encoded NupredMode included in the bit stream and sets the total number of prediction modes M (S3524).
  • the three-dimensional data decoding device arithmetically decodes the encoded PredMode using the total number M of the decoded prediction modes, and generates binarized data of the transcribed utility code (S3525).
  • the three-dimensional data decoding device calculates the prediction mode from the binarized data of the truncated utility code (S3526).
  • the three-dimensional data coding device may arithmetically encode the above-mentioned fixedPredMode using a coding table.
  • each three-dimensional data coding apparatus may update the probability of occurrence of 0 and 1 in the coding table according to the value of the actually generated fixedPredMode.
  • the three-dimensional data coding device may fix the probability of occurrence of 0 and 1 in either coding table. As a result, the three-dimensional data coding apparatus can reduce the amount of processing by suppressing the number of updates of the occurrence probabilities of 0 and 1 in the coding table.
  • FIG. 109 is a flowchart of the three-dimensional data coding process by the three-dimensional data coding device according to the present embodiment.
  • the three-dimensional data coding device encodes the position information (geometry) (S3531).
  • the three-dimensional data coding apparatus may be encoded using, for example, an ocree representation.
  • the three-dimensional data coding device redisplays the attribute information of the original three-dimensional point to the changed three-dimensional point. Allocate (S3532).
  • the three-dimensional data coding device may interpolate the value of the attribute information according to the amount of change in the position and reassign it. For example, the three-dimensional data coding device detects N three-dimensional points around the target three-dimensional point to be encoded before the change, which is close to the three-dimensional position after the change, and detects N three-dimensional points. The value of the attribute information is weighted and averaged based on the distance from the target 3D position to be encoded after the change to each of the N 3D points, and the value is used as the value of the attribute information of the changed 3D point. May be good.
  • the three-dimensional data encoding device when two or more three-dimensional points are changed to the same three-dimensional position due to quantization or the like, two or more points before the change are used as the value of the attribute information after the change.
  • the average value of the attribute information at the three-dimensional point may be assigned.
  • the three-dimensional data coding device encodes the attribute information (Attribute) after reassignment (S3533).
  • the three-dimensional data coding apparatus may encode a plurality of attribute information as attribute information in order.
  • the three-dimensional data coding device encodes a color and a reflectance as attribute information, it generates a bit stream in which the reflectance coding result is added after the color coding result. May be good.
  • the three-dimensional data coding device may add the coding result of the attribute information to the bit stream in any order. Further, the three-dimensional data coding device may add the start location of the coded data of each attribute information in the bit stream to the header or the like.
  • the three-dimensional data decoding device can reduce the processing amount by omitting the decoding process of the attribute information that does not need to be decoded.
  • the three-dimensional data coding device may encode a plurality of attribute information in parallel and integrate the coding results into one bit stream.
  • the three-dimensional data coding device can encode a plurality of attribute information at high speed.
  • FIG. 110 is a flowchart of the attribute information coding process (S3533) shown in FIG. 109.
  • the three-dimensional data coding device sets LoD (S35331). That is, the three-dimensional data coding device assigns each three-dimensional point to any of a plurality of LoDs.
  • the three-dimensional data coding device starts a loop in units of LoD (S35332). That is, the three-dimensional data coding apparatus repeats the processes of steps S35333 to S35341 for each LoD.
  • the three-dimensional data coding device starts a loop for each three-dimensional point (S35333). That is, the three-dimensional data coding apparatus repeats the processes of steps S35334 to S35340 for each three-dimensional point for a certain LoD. Note that FIG. 110 shows the coding at the target three-dimensional point P to be coded.
  • the three-dimensional data encoding device searches for a plurality of peripheral points that are three-dimensional points existing around the target three-dimensional point P, which are used to calculate the predicted value of the target three-dimensional point P to be processed (. S35334).
  • the three-dimensional data coding device calculates the predicted value of the target three-dimensional point P (S35335).
  • the three-dimensional data encoding device calculates the predicted residual, which is the difference between the attribute information of the target three-dimensional point P and the predicted value (S35336).
  • the three-dimensional data coding device calculates the quantized value by quantizing the predicted residual (S35337).
  • the three-dimensional data coding device arithmetically encodes the quantized value (S35338).
  • the three-dimensional data coding device calculates the dequantized value by dequantizing the quantized value (S35339).
  • the three-dimensional data encoding device generates a decoded value by adding the predicted value to the inverse quantization value (S35340).
  • the three-dimensional data coding device ends the loop for each three-dimensional point (S35341).
  • the three-dimensional data coding device ends the loop for each LoD (S35342).
  • FIG. 111 is a flowchart of the predicted value calculation process (S35335) shown in FIG. 110.
  • the three-dimensional data encoding device calculates the weight average value of the attribute values of the three-dimensional points around N of the target three-dimensional points to be encoded, which can be used for prediction, and assigns them to the prediction mode 0 ( S353351).
  • the three-dimensional data coding device calculates the maximum absolute difference value maxdiff of the attribute values of the three-dimensional points around N of the target three-dimensional points to be coded (S353352).
  • the three-dimensional data coding device determines whether or not the maximum absolute difference value maxdiff ⁇ Thfix is satisfied (S353353).
  • the prediction mode fixed flag is set to 1 and arithmetic coding is performed (S353354).
  • the three-dimensional data encoding device determines the prediction mode to 0 (prediction mode in which the weight average value is shown as the prediction value) (S353355).
  • the three-dimensional data coding device arithmetically encodes the predicted value of the determined prediction mode and outputs it to, for example, the three-dimensional data decoding device (S353356).
  • the prediction mode fixed flag is set to 0 and arithmetic coding is performed (S353357).
  • the three-dimensional data coding device selects and determines the prediction mode (S353358).
  • the three-dimensional data coding device arithmetically encodes the predicted mode selected and determined (S353359).
  • the three-dimensional data coding apparatus may perform arithmetic coding by binarizing the prediction mode PredMode with a transcribed utility code using the total number of prediction modes M. Further, the three-dimensional data coding device may encode and add the total number of prediction modes M as a NumPredMode to the header.
  • the three-dimensional data decoding device can correctly decode the prediction mode PredMode by decoding the NumPredMode in the header.
  • FIG. 112 is a flowchart of the prediction mode selection process (S353358) shown in FIG. 111.
  • the three-dimensional data coding apparatus obtains the attribute information of N three-dimensional points around the target three-dimensional point to be encoded from the prediction modes 1 to N in order from the one closest to the target three-dimensional point. Allocate (S3541). For example, a three-dimensional data coding device produces N + 1 prediction modes. The three-dimensional data coding apparatus may generate up to M prediction modes when N + 1 exceeds the total number of prediction modes M (NumPredMode) added to the bit stream.
  • NumPredMode the total number of prediction modes M
  • the three-dimensional data coding device calculates the cost of each mode and selects the prediction mode that minimizes the cost (S3542).
  • FIG. 113 is a flowchart showing a specific example of the prediction mode selection process (S3542) shown in FIG. 112.
  • the three-dimensional data coding device calculates the cost (cost [i]) of the i-th prediction mode PredMode [i] (S35422).
  • the three-dimensional data coding device determines whether or not cost [i] ⁇ mincost is satisfied (S35423).
  • the three-dimensional data coding device determines whether or not i ⁇ the number of prediction modes (total number of prediction modes) is satisfied (S35426).
  • step S35426 When the three-dimensional data coding apparatus determines that i ⁇ the number of prediction modes is not satisfied (No in S35426), the selection process is terminated, and when it is determined that i ⁇ the number of prediction modes is satisfied (Yes in S35426), the process is performed. Return to step S35422.
  • FIG. 114 is a flowchart of the three-dimensional data decoding process by the three-dimensional data decoding device according to the present embodiment.
  • the three-dimensional data decoding device decodes the coded position information (geometry) of the three-dimensional point (S3551).
  • the three-dimensional data decoding device may decode the position information using, for example, an ocree representation.
  • the three-dimensional data decoding device decodes the attribute information of the encoded three-dimensional point (S3552).
  • the three-dimensional data decoding device may decode a plurality of attribute information as attribute information in order. For example, when decoding a color and a reflectance as attribute information, a three-dimensional data decoding device decodes a bit stream in which the reflectance coding result is added after the color coding result in this order. May be good.
  • the three-dimensional data decoding device may decode the coding result of the attribute information added to the bit stream in any order.
  • the three-dimensional data decoding device may obtain the start location of the encoded data of each attribute information in the bit stream by decoding the header or the like.
  • the three-dimensional data decoding device can decode the attribute information that needs to be decoded. Therefore, the three-dimensional data decoding device can reduce the processing amount by omitting the decoding process of the attribute information that does not need to be decoded.
  • the three-dimensional data decoding device may decode a plurality of attribute information in parallel and integrate the decoding results into one three-dimensional point cloud.
  • the three-dimensional data decoding device can decode a plurality of attribute information at high speed.
  • FIG. 115 is a flowchart of the calculation process (S3552) of the predicted value of P shown in FIG. 114.
  • the three-dimensional data decoding device acquires the decoded position information of the three-dimensional points included in the bit stream (S35520). Specifically, the three-dimensional data decoding device acquires the decoded position information by decoding the encoded position information of the three-dimensional points included in the bit stream transmitted from the three-dimensional data coding device. To do.
  • the three-dimensional data decoding device sets LoD (S35521). That is, the three-dimensional data decoding device assigns each three-dimensional point to any of a plurality of LoDs.
  • the three-dimensional data decoding device starts a loop in units of LoD (S35522). That is, the three-dimensional data decoding device repeats the processes of steps S35523 to S35528 for each LoD.
  • the three-dimensional data decoding device acquires the attribute information included in the bit stream in parallel with the processing after step S35520 (S355201), for example.
  • the three-dimensional data decoding device decodes the quantized values of PredMode and P (S355211).
  • the three-dimensional data decoding device adds the number NuMofPoint of the number of three-dimensional points for each layer of LoD to the header portion and the like. Therefore, the decoding process of the PredMode in the bit stream and the process of calculating the three-dimensional points around the target three-dimensional point to be decoded, which can be used for the prediction after LoD generation, can be made independent.
  • the three-dimensional data decoding device performs LoD generation (LoD generation) and search for the adjacent point of P, and arithmetic decoding processing (decode PredMode and quantized value of P) of PredMode, n-bit code and remaining code. Can be executed independently. Therefore, the three-dimensional data decoding devices may be processed in parallel.
  • the three-dimensional data decoding device can reduce the overall processing time.
  • the three-dimensional data decoding device starts a loop for each three-dimensional point (S35523). That is, the three-dimensional data decoding device repeats the processes of steps S35524 to S35527 for each three-dimensional point for a certain LoD. Note that FIG. 115 shows decoding at the target three-dimensional point P to be decoded.
  • the three-dimensional data decoding device searches for a plurality of peripheral points that are three-dimensional points existing around the target three-dimensional point P, which are used to calculate the predicted value of the target three-dimensional point P to be processed (S35524). ).
  • the three-dimensional data decoding device calculates the predicted value of the target three-dimensional point P based on the PredMode (that is, the value of the prediction mode) decoded in step S355211 (S35525).
  • the three-dimensional data decoding device calculates the inverse quantization value by inversely quantizing the quantization value based on the quantization value of P decoded in step S355211 (S35526).
  • the three-dimensional data decoding device generates a decoded value by adding the predicted value to the inverse quantization value (S35527).
  • the three-dimensional data decoding device ends the loop for each three-dimensional point (S35527).
  • the three-dimensional data decoding device ends the loop for each LoD (S35528).
  • FIG. 116 is a flowchart showing the details of the calculation process (S35525) of the predicted value of P shown in FIG. 115.
  • the three-dimensional data decoding device calculates the weight average of the attribute values of N three-dimensional points around the target three-dimensional point to be decoded, which can be used for prediction, and assigns it to the prediction mode 0 (S355251).
  • the three-dimensional data decoding device allocates the attribute information of N three-dimensional points from the prediction mode 1 to N in order from the three-dimensional point closest to the target three-dimensional point to be decoded (S355252).
  • the three-dimensional data decoding device outputs the predicted value of the prediction mode decoded in step S355211 shown in FIG. 115 (S355253).
  • the three-dimensional data encoding device determines that the prediction mode is 0, that is, when maxdiff ⁇ Thfix is true or the decoded prediction mode is 0, the surrounding N around the target three-dimensional point to be encoded.
  • the weighted average of the attribute values of the three-dimensional points may be calculated.
  • the three-dimensional data decoding device can reduce the processing amount.
  • FIG. 117 is a flowchart of the prediction mode shown in FIG. 116 and the calculation process of the quantized value of P (S355211).
  • the three-dimensional data decoding device arithmetically decodes the prediction mode fixed flag (S3561).
  • the prediction mode is fixed to 0 (weighted average value) (S3563).
  • the three-dimensional data decoding device decodes the prediction mode from the bit stream (S3564).
  • the three-dimensional data decoding device may arithmetically decode the prediction mode PredMode using the total number of prediction modes M obtained by decoding the header of the bit stream.
  • the three-dimensional data decoding device calculates the value of the remaining code using the table of exponential Golomb codes (S3567).
  • the three-dimensional data decoding device sets the predicted residual (R_TH [LoDN] + value of the remaining code) (S3568).
  • the three-dimensional data decoding device converts the decoded predicted residual from an unsigned integer value to a signed integer value (S3569).
  • the 3D data encoding device has N 3D points around the target 3D point to be encoded, which can be used for prediction in each of the 3D data coding device and the 3D data decoding device. Calculate the maximum absolute difference value of the attribute value in. Further, the three-dimensional data coding device switches between fixing the prediction mode and selecting the prediction mode and adding it to the bit stream according to the calculated value.
  • the three-dimensional data coding device does not have to switch between fixing the prediction mode according to the maximum absolute difference value or selecting the prediction mode and adding it to the bit stream.
  • the three-dimensional data coding device may always select a prediction mode and add the prediction mode to the bit storm.
  • the three-dimensional data decoding device can correctly decode the bit stream by constantly decoding the prediction mode added to the bit stream.
  • the 3D data decoding device arithmetically decodes the prediction mode without calculating the maximum absolute difference value of the attribute values at N 3D points around the target 3D point to be decoded, which can be used for prediction. You will be able to do it. Therefore, the three-dimensional data decoding device can execute the arithmetic decoding of the bit stream and the LoD generation in parallel. As a result, the three-dimensional data decoding device can improve the overall throughput of the processing.
  • FIG. 118 is a flowchart of processing when the three-dimensional data coding apparatus according to the present embodiment does not fix the prediction mode.
  • the three-dimensional data coding device selects the prediction mode (S3571).
  • the three-dimensional data coding device arithmetically encodes the selected prediction mode (S3572).
  • FIG. 119 is a flowchart of processing when the three-dimensional data decoding device according to the present embodiment does not fix the prediction mode.
  • the three-dimensional data decoding device decodes the prediction mode from the bit stream (S3573).
  • the method by which the three-dimensional data encoding device always selects the prediction mode is not particularly limited.
  • the three-dimensional data encoding device can always add the PredMode to the bitstream because the minimum value of maxdiff is 0.
  • the value of Thfix [i] may be any value as long as maxdiff ⁇ Thfix [i] is true. ..
  • FIG. 120 is a diagram showing another example of the syntax of the attribute data according to the present embodiment.
  • PredMode is a value indicating a prediction mode for encoding and decoding the attribute value of the j-th three-dimensional point of the layer i of LoD.
  • the PredMode takes any value from 0 to M-1 (where M is the total number of prediction modes).
  • the three-dimensional data decoding device may estimate the PredMode to 0 when the PredMode is not in the bitstream (specifically, when the NuPredMode [i]> 1 is not satisfied).
  • the three-dimensional data decoding device does not have to set the PredMode to 0 when the PredMode is not in the bit stream, and may adopt any value from 0 to M-1 as an estimated value.
  • the three-dimensional data encoding device may separately add an estimated value when the PredMode is not in the bit stream to the header or the like.
  • the three-dimensional data encoding device may binarize the PredMode with a transcribed utility code using the value of the total number of prediction modes M, and arithmetically encode the binarized value.
  • the three-dimensional data encoding device may encode and add the value of the total number of prediction modes M to the header of the bit stream as NupredMode.
  • the three-dimensional data decoding device can decode the NumPredMode in the header to calculate the total number of prediction modes M, and can decode the PredMode using the total number of prediction modes M. Therefore, the three-dimensional data decoding device generates LoD, calculates the three-dimensional points around the target three-dimensional point to be decoded, and waits for the calculation of the number of prediction modes to which the predicted value is assigned. It is possible to perform arithmetic decoding of bitstreams without.
  • the three-dimensional data decoding device can execute arithmetic decoding of the bitstream and LoD generation in parallel. Therefore, the three-dimensional data decoding device can improve the overall throughput of the processing.
  • the n-bit code is encoded data of the predicted residual of the value of the attribute information.
  • the bit length of the n-bit code depends on the value of R_TH [i]. For example, when the value indicated by R_TH [i] is 63, it is 6 bits, and when the value indicated by R_TH [i] is 255, it is 8 bits.
  • the remaining code is the coded data encoded by the exponential Golomb code among the coded data of the predicted residuals of the value of the attribute information.
  • the three-dimensional data decoding apparatus decodes the remaining code when the n-bit code is the same as R_TH [i], adds the value of the n-bit code and the value of the remaining code, and decodes the predicted residual.
  • the three-dimensional data decoding device does not have to decode the remaining code.
  • FIG. 121 is a flowchart showing an example of coding processing in the prediction mode of the three-dimensional data coding apparatus according to the present embodiment.
  • the three-dimensional data coding device binarizes the prediction mode with a transcribed utility code using the total number of prediction modes M (S3574).
  • the three-dimensional data encoding device arithmetically encodes the binarized data of the truncated utility code (S3575).
  • the three-dimensional data coding device adds the total number of prediction modes M to the header as a NumPredMode and encodes it (S3576).
  • FIG. 122 is a flowchart showing an example of decoding processing in the prediction mode of the three-dimensional data decoding apparatus according to the present embodiment.
  • the three-dimensional data decoding device decodes the encoded NupredMode included in the bit stream and sets the total number of prediction modes M (S3577).
  • the three-dimensional data decoding device arithmetically decodes the encoded PredMode using the total number M of the decoded prediction modes, and generates binarized data of the transcribed utility code (S3578).
  • the three-dimensional data decoding device calculates the prediction mode from the binarized data of the truncated utility code (S3579).
  • FIG. 123 is a flowchart showing another example of the prediction value calculation process (step S35335) shown in FIG. 110.
  • the three-dimensional data encoding device calculates the weight average value of the attribute values of the three-dimensional points around N of the target three-dimensional points to be encoded, which can be used for prediction, and assigns them to the prediction mode 0 ( S3581).
  • the three-dimensional data coding device selects and determines the prediction mode (S3582).
  • the three-dimensional data coding device arithmetically encodes the predicted mode selected and determined (S3583).
  • the three-dimensional data coding device outputs the predicted value of the determined prediction mode (S3584).
  • the three-dimensional data coding device may binarize the prediction mode PredMode using the total number of prediction modes M and perform arithmetic coding. Further, the three-dimensional data coding device may encode and add the total number of prediction modes M as a NumPredMode to the header.
  • the three-dimensional data decoding device can correctly decode the prediction mode PredMode by decoding the NumPredMode in the header.
  • FIG. 124 is a flowchart of the prediction mode selection process (S3582) shown in FIG. 123.
  • the three-dimensional data coding apparatus obtains the attribute information of N three-dimensional points around the target three-dimensional point to be encoded from the prediction modes 1 to N in order from the one closest to the target three-dimensional point. Allocate (S35821). For example, a three-dimensional data coding device produces N + 1 prediction modes. The three-dimensional data coding apparatus may generate up to M prediction modes when N + 1 exceeds the total number of prediction modes M (NumPredMode) added to the bit stream.
  • NumPredMode the total number of prediction modes M
  • the three-dimensional data coding device calculates the cost of each mode and selects the prediction mode that minimizes the cost (S35522).
  • FIG. 125 is a flowchart showing a specific example of the prediction mode selection process (S35522) shown in FIG. 124.
  • the three-dimensional data coding device calculates the cost [i] of the i-th prediction mode PredMode [i] (S358222).
  • the three-dimensional data coding device determines whether or not cost [i] ⁇ mincost is satisfied (S358223).
  • the three-dimensional data coding device determines whether or not i ⁇ the number of prediction modes is satisfied (S358226).
  • the three-dimensional data coding device generates N + 1 prediction modes when the number of three-dimensional points around the target three-dimensional point to be encoded is N.
  • N + 1 exceeds the total number of prediction modes M (NumPredMode) added to the bit stream
  • the three-dimensional data coding apparatus may generate up to M prediction modes.
  • FIG. 126 is a flowchart showing another example of the prediction mode shown in FIG. 115 and the calculation process of the quantized value of P (S355211).
  • the three-dimensional data decoding device decodes the prediction mode from the bit stream (S3552111).
  • the three-dimensional data decoding device may arithmetically decode the prediction mode PredMode using the total number of prediction modes M obtained by decoding the header.
  • the acid salt current data coding device arithmetically decodes the n-bit code (S3552112).
  • the three-dimensional data decoding device calculates the value of the remaining code using the table of exponential Golomb codes (S3552115).
  • the three-dimensional data decoding device sets the predicted residual (R_TH [LoDN] + value of the remaining code) (S3552116).
  • the three-dimensional data decoding device converts the decoded predicted residual from an unsigned integer value to a signed integer value (S3552117).
  • FIG. 127 is a diagram showing an example of attribute information used for the predicted value.
  • the predicted value of the attribute information of the point a2 shown in FIG. 127 is generated by using the attribute information of the points a0 and a1.
  • the predicted value of the attribute information of the point a5 shown in FIG. 127 is generated by using the attribute information of the points a0, a1, a2, a3, and a4.
  • the three-dimensional data coding device may generate LoD from an upper layer (for example, LoD0).
  • the three-dimensional data coding device may generate LoD from a lower layer (for example, LoD2).
  • the predicted value is calculated by the attribute value based on the weighted average of adjacent points.
  • the predicted value a2p of the point a2 is weighted by the attribute values of the points a0 and a1 as shown in the following (Equation F1), (Equation F2), and (Equation F3). Calculated by average.
  • a i is a value of the attribute information of the point ai. Further, ave is the average value of A i.
  • an intermediate value of A i may be used.
  • (ai-ave) 2 and (ai-ave) 2 may be used instead of
  • the predicted value a5p of the point a5 is the attribute information of the points a0, a1, a2, a3, and a4 as shown in the following (Equation F4), (Equation F5), and (Equation F6). It is calculated by the weighted average of.
  • the three-dimensional data coding device calculates, for example, when
  • 0, as
  • 1.
  • the three-dimensional data encoding apparatus is able to calculate the value of w i.
  • the three-dimensional data coding apparatus calculates, for example, as
  • 1 when
  • 0.
  • the three-dimensional data encoding apparatus is able to calculate the value of w i.
  • the predicted value aNp of the point aN is the points N-4, N-3, N-2, and, as shown in the following (Equation F7), (Equation F8), and (Equation F9). It is calculated by the weighted average of the attribute information of N-1. N is a positive integer of 5 or more.
  • the three-dimensional data encoding device calculates as
  • 1 when
  • the three-dimensional data encoding device calculates as
  • 1 when
  • FIG. 128 is a diagram showing an example of a reference relationship.
  • the three-dimensional data coding apparatus calculates the predicted value of the three-dimensional point included in LoD2 by using the attribute information of the three-dimensional point included in LoD0 or LoD1. Further, the three-dimensional data encoding device calculates the predicted value of the three-dimensional point included in LoD1 by using the attribute information of the three-dimensional point included in LoD0.
  • LoD0 or LoD1 does not necessarily have to be the coded and decoded attribute information.
  • the three-dimensional data coding apparatus may use, for example, the value before coding.
  • the three-dimensional data encoding device may generate a predicted value of the attribute information of the three-dimensional point included in LoDn by using the attribute information included in LoDn'(n' ⁇ n).
  • the three-dimensional data coding apparatus can calculate the predicted values of the plurality of three-dimensional points included in LoDn in parallel.
  • a three-dimensional data encoding device calculates the predicted value of the attribute information of a three-dimensional point by averaging the attribute values of N or less three-dimensional points among the three-dimensional points around the target three-dimensional point to be encoded. Is generated by calculating. Further, the three-dimensional data coding device may add a value of N to the header of the bit stream or the like. The three-dimensional data coding apparatus may change the value of N for each three-dimensional point and add the value of N for each three-dimensional point. As a result, an appropriate N can be selected for each three-dimensional point, so that the accuracy of the predicted value can be improved. Therefore, the predicted residual can be reduced.
  • the three-dimensional data encoding device may add the value of N to the header of the bitstream and fix the value of N in the bitstream. This eliminates the need to encode or decode the value of N for each three-dimensional point, so that the amount of processing can be reduced. Further, the three-dimensional data coding apparatus may separately encode the value of N for each LoD. As a result, the coding efficiency can be improved by selecting an appropriate N for each LoD.
  • the three-dimensional data encoding device may calculate the predicted value of the attribute information of the three-dimensional points by the weighted average value of the attribute information of the surrounding N three-dimensional points. For example, the three-dimensional data coding device calculates the weight using the distance information of the target three-dimensional point and the surrounding N three-dimensional points.
  • the higher layer of LoD is set to have a larger N value
  • the lower layer is set to have a smaller N value.
  • the distance between the three-dimensional points belonging to the upper layer of LoD is larger than the distance between the three-dimensional points belonging to the upper layer of LoD. Therefore, in the upper layer, by setting a large value of N, there is a possibility that the prediction accuracy can be improved by averaging many surrounding three-dimensional points. Moreover, the distance between the three-dimensional points belonging to the lower layer of LoD is short. Therefore, in the lower layer, by setting the value of N small, it is possible to perform efficient prediction while suppressing the amount of averaging processing.
  • the peripheral point P' is selected based on the distance from the point P.
  • the predicted value of the attribute information of the point b2 shown in FIG. 128 is generated by using the attribute information of the points a0, a1, and a2.
  • the predicted value is calculated by a distance-dependent weighted average.
  • the predicted value b2p of the point b2 is calculated by the weighted average of the attribute information of the points a0 and a1 as shown in (Equation J1) and (Equation J2).
  • a i is a value of the attribute information of the point ai.
  • d (p, q) is, for example, the Euclidean distance between the three-dimensional point p and the three-dimensional point q.
  • the predicted value aNp of the point aN is calculated by the weighted average of the attribute information of the points aN-4, aN-3, aN-2, and aN-1 as shown in (Equation J3) and (Equation A4). ..
  • the three-dimensional data encoding device calculates the difference value (predicted residual) between the value of the attribute information of the three-dimensional point and the predicted value generated from the surrounding points, and even if the calculated predicted residual is quantized.
  • a three-dimensional data coding device performs quantization by dividing the predicted residual by a quantization scale (also called a quantization step).
  • a quantization scale also called a quantization step.
  • the smaller the quantization scale the smaller the error (quantization error) that can occur due to quantization.
  • the larger the quantization scale the larger the quantization error.
  • the three-dimensional data coding device may change the quantization scale used for each LoD. For example, in a three-dimensional data coding apparatus, the higher layer has a smaller quantization scale, and the lower layer has a larger quantization scale. Since the value of the attribute information of the three-dimensional point belonging to the upper layer may be used as the predicted value of the attribute information of the three-dimensional point belonging to the lower layer, the quantization scale of the upper layer is reduced in the upper layer. Coding efficiency can be improved by suppressing possible quantization errors and improving the accuracy of predicted values.
  • the three-dimensional data coding device may add a quantization scale used for each LoD to the header or the like. As a result, the three-dimensional data decoding device can correctly decode the quantization scale, so that the bit stream can be appropriately decoded.
  • the three-dimensional data coding apparatus may adaptively switch the quantization scale to be used according to the importance of the target three-dimensional point.
  • a 3D data coding device uses a small quantization scale for 3D points of high importance and a large quantization scale for 3D points of low importance.
  • the importance may be calculated from, for example, the number of times the target three-dimensional point is referred to from another three-dimensional point at the time of calculating the predicted value, the weight at that time, or the like.
  • the following QW (Quantization Weight) value is used as the importance.
  • the three-dimensional data coding apparatus sets a large QW value of a three-dimensional point having a high importance and a small quantization scale. As a result, the quantization error of the three-dimensional points having high importance is reduced, so that the coding efficiency can be improved.
  • FIG. 129 is a diagram showing a calculation example of QW.
  • the QS (quantization step) may be changed according to LoD.
  • QS_LoD0 is a QS for LoD0
  • QS_LoD1 is a QS for LoD1.
  • QW is a value representing the importance of the target three-dimensional point.
  • the value obtained by multiplying the QW value of the point c0 by the weight W b2 _c0 with respect to the point b2 calculated when the predicted value of the point c0 is generated. May be added to the value of QW at point b2.
  • the QW value of the three-dimensional point often used for generating the predicted value becomes large, and the prediction efficiency can be improved by suppressing the quantization error of the three-dimensional point.
  • the 3D data coding device may first initialize the QW values of all 3D points with 1, and update the QW of each 3D point according to the prediction structure.
  • the three-dimensional data coding apparatus may change the initial values according to the LoD hierarchy without initializing the QWs of all the three-dimensional points with the value 1.
  • the quantization scale of the upper layer may be reduced by setting the initial value of QW to be larger in the upper layer. As a result, the prediction error of the upper layer can be suppressed, so that the prediction accuracy of the lower layer can be improved and the coding efficiency can be improved.
  • the three-dimensional data coding apparatus may convert a signed integer value (signed quantization value), which is a predicted residual after quantization, into an unsigned integer value (unsigned quantization value). This eliminates the need to consider the occurrence of negative integers when entropy encoding the predicted residuals.
  • the three-dimensional data coding apparatus does not necessarily have to convert the signed integer value into an unsigned integer value, and for example, the sign bit may be separately entropy-encoded.
  • FIG. 130 is a diagram showing a calculation example of the predicted residual.
  • the predicted residual is calculated by subtracting the predicted value from the original value.
  • the prediction residual b2r the point b2, as shown in (Equation J5), from the value B 2 of the attribute information of the point b2, is calculated by subtracting the prediction value b2p the point b2.
  • the predicted residual is quantized by dividing by QS (Quantization Step).
  • QS_LoD1 is a QS for LoD1. That is, the QS may be changed according to LoD.
  • the three-dimensional data encoding device converts the signed integer value, which is the above-mentioned quantization value, into an unsigned integer value as follows.
  • the three-dimensional data coding apparatus sets the unsigned integer value b2u to -1- (2 ⁇ b2q) when the signed integer value b2q is smaller than 0.
  • the signed integer value b2q is 0 or more
  • the three-dimensional data coding apparatus sets the unsigned integer value b2u to 2 ⁇ b2q.
  • the three-dimensional data coding device scans and encodes the predicted residuals (unsigned integer values) after quantization in a certain order.
  • the three-dimensional data coding apparatus encodes a plurality of three-dimensional points in order from the three-dimensional points included in the upper layer of LoD toward the lower layer.
  • the position information of a plurality of three-dimensional points is encoded by using the prediction tree generated based on the position information.
  • FIG. 131 is a diagram showing an example of a prediction tree used in the three-dimensional data coding method according to the eighth embodiment.
  • FIG. 132 is a flowchart showing an example of the three-dimensional data coding method according to the eighth embodiment.
  • FIG. 133 is a flowchart showing an example of the three-dimensional data decoding method according to the eighth embodiment.
  • a prediction tree is generated using a plurality of three-dimensional points, and then the node information included in each node of the prediction tree is encoded. This gives a bitstream containing the encoded node information.
  • Each node information is, for example, information about one node of the prediction tree.
  • Each node information includes, for example, the position information of one node, the index of the one node, the number of child nodes of the one node, and the prediction mode used to encode the position information of the one node. And includes the predicted residuals.
  • each encoded node information included in the bit stream is decoded, and then the position information is decoded while generating a prediction tree. ..
  • FIG. 134 is a diagram for explaining a method of generating a predicted tree according to the eighth embodiment.
  • the three-dimensional data coding device first adds a point 0 as the initial point of the prediction tree.
  • the position information of the point 0 is indicated by the coordinates including the three elements (x0, y0, z0).
  • the position information of the point 0 may be indicated by the coordinates of the three-axis Cartesian coordinate system or may be indicated by the coordinates of the polar coordinate system.
  • Child_count is incremented by 1 each time one child node is added to the node for which the child_count is set.
  • the child_count of each node after the generation of the predicted tree is completed indicates the number of child nodes possessed by each node, and is added to the bit stream.
  • the pred_mode indicates a prediction mode for predicting the value of the position information of each node. The details of the prediction mode will be described later.
  • the three-dimensional data coding device adds point 1 to the prediction tree.
  • the three-dimensional data encoding device may search for the nearest neighbor point of the point 1 from the point cloud already added to the prediction tree, and add the point 1 as a child node of the nearest neighbor point.
  • the position information of the point 1 is indicated by the coordinates including the three elements (x1, y1, z1).
  • the position information of the point 1 may be indicated by the coordinates of the three-axis Cartesian coordinate system or may be indicated by the coordinates of the polar coordinate system.
  • point 0 is the nearest point to point 1
  • point 1 is added as a child node of point 0.
  • the three-dimensional data coding device increments the value indicated by the child_count at point 0 by 1.
  • the predicted value of the position information of each node may be calculated when a node is added to the prediction tree.
  • the three-dimensional data coding device may add point 1 as a child node of point 0 and calculate the position information of point 0 as a predicted value.
  • pred_mode 1 may be set.
  • the pred_mode is prediction mode information (prediction mode value) indicating the prediction mode.
  • the three-dimensional data coding apparatus may calculate the error_value (prediction residual) of the point 1 after calculating the predicted value.
  • residual_value is a difference value obtained by subtracting the predicted value calculated in the predicted mode indicated by pred_mode from the position information of each node.
  • the coding efficiency can be improved by coding the difference value from the predicted value instead of the position information itself.
  • the three-dimensional data coding device adds point 2 to the prediction tree.
  • the three-dimensional data encoding device may search for the nearest neighbor point of the point 2 from the point cloud already added to the prediction tree, and add the point 2 as a child node of the nearest neighbor point.
  • the position information of the point 2 is indicated by the coordinates including the three elements (x2, y2, z2).
  • the position information of the point 2 may be indicated by the coordinates of the three-axis Cartesian coordinate system or may be indicated by the coordinates of the polar coordinate system.
  • the point 1 becomes the nearest point of the point 2, and the point 2 is added as a child node of the point 1.
  • the three-dimensional data coding device increments the value indicated by the child_count of point 1 by 1.
  • the three-dimensional data coding device adds the point 3 to the prediction tree.
  • the three-dimensional data encoding device may search for the nearest neighbor point of the point 3 from the point cloud already added to the prediction tree, and add the point 3 as a child node of the nearest neighbor point.
  • the position information of the point 3 is indicated by the coordinates including the three elements (x3, y3, z3).
  • the position information of the point 3 may be indicated by the coordinates of the three-axis Cartesian coordinate system or may be indicated by the coordinates of the polar coordinate system.
  • point 0 is the nearest point to point 3
  • point 3 is added as a child node of point 0.
  • the three-dimensional data coding device increments the value indicated by the child_count at point 0 by 1.
  • the 3D data coding device adds all the points to the prediction tree and completes the generation of the prediction tree.
  • the three-dimensional data coding device encodes the children_count, pred_mode, and residual_value of each node selected from the nodes of the root in the order of depth priority. That is, when the three-dimensional data coding device selects the nodes in the depth priority order, the three-dimensional data coding device selects a child node that has not yet been selected from one or more child nodes of the selected node as the next node of the selected node. .. If the selected node does not have a child node, the 3D data coding device selects another unselected child node of the parent node of the selected node.
  • the coding order is not limited to the depth priority order, but may be, for example, a breadth first order.
  • the three-dimensional data encoding device selects a node that has not yet been selected from one or more nodes of the same depth (hierarchy) as the selected node as the next node of the selected node. select. If there is no node with the same depth as the selected node, the three-dimensional data coding device selects a node that has not yet been selected from one or more nodes of the next depth.
  • points 0 to 3 are examples of a plurality of three-dimensional points.
  • child_count, pred_mode, and regional_value are calculated when each point is added to the prediction tree, but the present invention is not limited to this, and for example, after the generation of the prediction tree is completed. , They may be calculated.
  • the input order of the plurality of three-dimensional points to the three-dimensional data encoding device may be such that the input three-dimensional points are sorted in ascending or descending order of the Morton orderer and processed in order from the first three-dimensional point.
  • the three-dimensional data coding apparatus can efficiently search for the nearest neighbor point of the three-dimensional point to be processed, and can improve the coding efficiency.
  • the three-dimensional data coding apparatus may process the three-dimensional points in the input order without rearranging them.
  • the 3D data coding device may generate a predictive tree without branches in the input order of a plurality of 3D points.
  • the three-dimensional data encoding device adds the input three-dimensional point next to the input three-dimensional point as a child node of a predetermined three-dimensional point in the input order of a plurality of three-dimensional points. You may.
  • FIG. 135 is a diagram for explaining a first example of the prediction mode according to the eighth embodiment.
  • FIG. 135 is a diagram showing a part of the predicted tree.
  • Eight prediction modes may be set as shown below. For example, as shown in FIG. 135, a case where the predicted value of the point c is calculated will be described as an example.
  • the prediction tree shows that the parent node at point c is point p0, the grandfather node at point c is point p1, and the great-grandfather node at point c is point p2.
  • the point c, the point p0, the point p1, and the point p2 are examples of a plurality of three-dimensional points.
  • the prediction mode in which the prediction mode value is 0 (hereinafter referred to as prediction mode 0) may be set without prediction. That is, the three-dimensional data coding apparatus may calculate the input position information of the point c as the predicted value of the point c in the prediction mode 0.
  • prediction mode 1 the prediction mode in which the prediction mode value is 1 (hereinafter referred to as prediction mode 1) may be set to the difference prediction from the point p0. That is, the three-dimensional data encoding device may calculate the position information of the point p0, which is the parent node of the point c, as the predicted value of the point c.
  • the prediction mode in which the prediction mode value is 2 (hereinafter referred to as prediction mode 2) may be set to linear prediction with points p0 and points p1. That is, the three-dimensional data coding device obtains the prediction result of the point c by linear prediction using the position information of the point p0 which is the parent node of the point c and the position information of the point p1 which is the grandfather node of the point c. It may be calculated as a predicted value. Specifically, the three-dimensional data coding device calculates the predicted value of the point c in the prediction mode 2 using the following formula T1.
  • p0 indicates the position information of the point p0
  • p1 indicates the position information of the point p1.
  • the prediction mode in which the prediction mode value is 3 may be set to Parallelogram prediction using points p0, point p1 and point p2. That is, the three-dimensional data encoding device has the position information of the point p0 which is the parent node of the point c, the position information of the point p1 which is the grandfather node of the point c, and the position information of the point p2 which is the great-grandfather node of the point c.
  • the prediction result by the Parallelog prediction using and may be calculated as the prediction value of the point c.
  • the three-dimensional data coding device calculates the predicted value of the point c in the prediction mode 3 using the following formula T2.
  • p0 indicates the position information of the point p0
  • p1 indicates the position information of the point p1
  • p2 indicates the position information of the point p2.
  • prediction mode 4 may be set to the difference prediction from the point p1. That is, the three-dimensional data encoding device may calculate the position information of the point p1 which is the grandfather node of the point c as the predicted value of the point c.
  • prediction mode 5 may be set to the difference prediction from the point p2. That is, the three-dimensional data coding device may calculate the position information of the point p2, which is the great-grandfather node of the point c, as the predicted value of the point c.
  • the prediction mode in which the prediction mode value is 6 may be set to the average of any two or more position information of points p0, point p1, and points p2. That is, the three-dimensional data encoding device has the position information of the point p0 which is the parent node of the point c, the position information of the point p1 which is the grandfather node of the point c, and the position information of the point p2 which is the great-grandfather node of the point c.
  • the average value of two or more position information of the above may be calculated as the predicted value of the point c.
  • the predicted value of the point c in the prediction mode 6 is calculated using the following equation T3. To do.
  • p0 indicates the position information of the point p0
  • p1 indicates the position information of the point p1.
  • prediction mode 7 the prediction mode in which the prediction mode value is 7 (hereinafter referred to as prediction mode 7) is used for non-linear prediction using the distance d0 between the points p0 and p1 and the distance d1 between the points p2 and p1. It may be set. That is, the three-dimensional data coding apparatus may calculate the prediction result by the non-linear prediction using the distance d0 and the distance d1 as the prediction value of the point c.
  • the prediction method assigned to each prediction mode is not limited to the above example. Further, the above eight prediction modes and the above eight prediction methods do not have to be the above combinations, and may be any combination. For example, when the prediction mode is encoded by using entropy coding such as arithmetic coding, the prediction mode 0 may be assigned a frequently used prediction method. Thereby, the coding efficiency can be improved. Further, the three-dimensional data coding apparatus may improve the coding efficiency by dynamically changing the allocation of the prediction mode according to the frequency of use of the prediction mode while proceeding with the coding process.
  • the three-dimensional data coding apparatus may, for example, count the frequency of use of each prediction mode at the time of coding, and assign a prediction mode indicated by a smaller value as the prediction method has a higher frequency of use. As a result, the coding efficiency can be improved.
  • the three-dimensional data encoding device uses the predicted value (px, py, pz) of the position information (x, y, z) of the three-dimensional point as a code among the three-dimensional points around the three-dimensional point to be encoded.
  • the predicted value used for calculating the position information of the three-dimensional point to be encoded may be calculated by using the position information of the three-dimensional point whose distance is close to the three-dimensional point to be encoded. Further, the three-dimensional data coding apparatus may add prediction mode information (pred_mode) for each three-dimensional point so that the predicted value calculated according to the prediction mode can be selected.
  • the position information of the three-dimensional point p0 of the nearest neighbor point is assigned to the prediction mode 0, ...
  • the position information of the three-dimensional point p2 is assigned to the prediction mode M-1 and used for prediction. It is conceivable to add the predicted mode to the bitstream for each three-dimensional point.
  • FIG. 136 is a diagram showing a second example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • the predicted value of the position information of the point c is calculated using at least one of the position information of the point p0, the point p1, and the point p2.
  • the prediction mode is added for each three-dimensional point to be encoded.
  • the predicted value is calculated according to the added prediction mode.
  • FIG. 137 is a diagram showing a specific example of the second example of the table showing the predicted values calculated in each prediction mode according to the eighth embodiment.
  • the three-dimensional data coding apparatus selects, for example, the prediction mode 1, and encodes the position information (x, y, z) of the three-dimensional point to be encoded by using the predicted values (p0x, p0y, p0z), respectively. It may be converted. In this case, "1", which is a prediction mode value indicating the selected prediction mode 1, is added to the bit stream.
  • the three-dimensional data coding apparatus can be used as one prediction mode for calculating the prediction value of each of the three elements including the position information of the three-dimensional point to be encoded in the selection of the prediction mode.
  • a common prediction mode may be selected for the three elements.
  • FIG. 138 is a diagram showing a third example of a table showing predicted values calculated in each prediction mode according to the eighth embodiment.
  • the predicted value of the position information of the point c is calculated using at least one of the position information of the point p0 and the point p1.
  • the prediction mode is added for each three-dimensional point to be encoded.
  • the predicted value is calculated according to the added prediction mode.
  • the prediction mode in which the predicted value is not assigned may be set to not available.
  • another prediction method may be assigned to the prediction mode.
  • the position information of the point p2 may be assigned as a predicted value to the prediction mode.
  • the prediction mode may be assigned a prediction value assigned to another prediction mode.
  • the position information of the point p1 assigned to the prediction mode 4 may be assigned to the prediction mode 3 in which the not variable is set. At that time, the position information of the point p2 may be newly assigned to the prediction mode 4. In this way, when a prediction mode in which not variable is set occurs, the coding efficiency can be improved by assigning a new prediction method to the prediction mode.
  • the predicted value may be calculated in the prediction mode divided for each of the three elements. For example, if the three elements are represented by x, y, z of the coordinates (x, y, z) of the three-axis Cartesian coordinate system, each of the predicted values of the three elements is the prediction selected in each element. It may be calculated in the mode.
  • the prediction mode pred_mode_x for calculating the predicted value of the element x (that is, the x coordinate)
  • the prediction mode pred_mode_y for calculating the predicted value of the element y (that is, the y coordinate)
  • the predicted value of the element z that is, the z coordinate predicted value
  • a prediction mode value may be selected in each of the prediction modes pred_mode_z for calculation.
  • the values in the tables of FIGS. 139 to 141 which will be described later, are used as the prediction mode values indicating the prediction modes of each element.
  • Each of these prediction mode values may be added to the bit stream.
  • the coordinates of the three-axis orthogonal coordinate system have been described as an example of the position information, but the coordinates of the polar coordinate system have also been described in the same manner. Can be applied.
  • the three-dimensional data coding apparatus can be used as one prediction mode for calculating the prediction value of each of the three elements including the position information of the three-dimensional point to be encoded in the selection of the prediction mode.
  • Independent prediction modes may be selected for each of the three elements.
  • the predicted value including two or more elements among the plurality of elements of the position information may be calculated in a common prediction mode.
  • the prediction mode pred_mode_x for calculating the predicted value using the element x and the element
  • the prediction mode value may be selected in each of the prediction mode pred_mode_yz for calculating the prediction value using y and the element z.
  • the values in the tables of FIGS. 139 and 142 which will be described later, are used as the prediction mode values indicating the prediction modes of each component, and these prediction mode values may be added to the bit stream, respectively.
  • the three-dimensional data coding apparatus can be used as one prediction mode for calculating the prediction value of each of the three elements including the position information of the three-dimensional point to be encoded in the selection of the prediction mode.
  • a common prediction mode may be selected for two of the three elements, and a prediction mode independent of the above two elements may be selected for the remaining one element.
  • FIG. 139 is a diagram showing a fourth example of a table showing predicted values calculated in each prediction mode.
  • the fourth example is an example in which the position information used for the predicted value is the value of the element x of the position information of the surrounding three-dimensional points.
  • the predicted value calculated in the predicted mode pred_mode_x in which the predicted mode value is indicated by "0" is 0.
  • the predicted value calculated in the predicted mode pred_mode_x whose predicted mode value is indicated by "1” is the x-coordinate of the point p0, which is p0x.
  • the predicted value calculated in the predicted mode pred_mode_x whose prediction mode value is indicated by "2” is a prediction result of linear prediction based on the x-coordinate of the point p0 and the x-coordinate of the point p1 (2 x p0x-p1x). Is.
  • the predicted value calculated in the predicted mode pred_mode_x whose prediction mode value is indicated by "3" is the prediction result of Parallelogram prediction by the x-coordinate of the point p0, the x-coordinate of the point p1, and the x-coordinate of the point p2. , (P0x + p1x ⁇ p2x). Further, the predicted value calculated in the predicted mode pred_mode_x whose predicted mode value is indicated by "4" is the x-coordinate of the point p1 and is p1x.
  • the prediction mode pred_mode_x whose prediction mode value is indicated by "1" is selected in the table of FIG. 139
  • the x-coordinate of the position information of the three-dimensional point to be encoded is coded using the prediction value p0x. It may be converted. In this case, "1" as the prediction mode value is added to the bit stream.
  • FIG. 140 is a diagram showing a fifth example of a table showing predicted values calculated in each prediction mode.
  • the fifth example is an example in which the position information used for the predicted value is the value of the element y of the position information of the surrounding three-dimensional points.
  • the predicted value calculated in the predicted mode pred_mode_y in which the predicted mode value is indicated by "0" is 0.
  • the predicted value calculated in the predicted mode pred_mode_y whose prediction mode value is indicated by "1” is the y coordinate of the point p0, which is p0y.
  • the predicted value calculated in the predicted mode pred_mode_y whose prediction mode value is indicated by "2” is a prediction result of linear prediction based on the y coordinate of the point p0 and the y coordinate of the point p1 (2 ⁇ p0y ⁇ p1y). Is.
  • the predicted value calculated in the predicted mode pred_mode_y whose prediction mode value is indicated by "3" is the prediction result of Parallelogram prediction by the y coordinate of the point p0, the y coordinate of the point p1, and the y coordinate of the point p2. , (P0y + p1y ⁇ p2y). Further, the predicted value calculated in the predicted mode pred_mode_y whose prediction mode value is indicated by "4" is the y coordinate of the point p1 and is p1y.
  • the prediction mode pred_mode_y whose prediction mode value is indicated by "1" is selected in the table of FIG. 140
  • the y-coordinate of the position information of the three-dimensional point to be encoded is coded using the prediction value p0y. It may be converted. In this case, "1" as the prediction mode value is added to the bit stream.
  • FIG. 141 is a diagram showing a sixth example of a table showing predicted values calculated in each prediction mode.
  • the sixth example is an example in which the position information used for the predicted value is the value of the element z of the position information of the surrounding three-dimensional points.
  • the predicted value calculated in the predicted mode pred_mode_z whose predicted mode value is indicated by "0" is 0.
  • the predicted value calculated in the predicted mode pred_mode_z whose prediction mode value is indicated by "1” is the z coordinate of the point p0, which is p0z.
  • the predicted value calculated in the predicted mode pred_mode_z whose prediction mode value is indicated by "2” is a prediction result of linear prediction based on the z coordinate of the point p0 and the z coordinate of the point p1 (2 ⁇ p0z ⁇ p1z). Is.
  • the predicted value calculated in the predicted mode pred_mode_z whose prediction mode value is indicated by "3" is the prediction result of Parallelogram prediction by the z coordinate of the point p0, the z coordinate of the point p1, and the z coordinate of the point p2. , (P0z + p1z-p2z). Further, the predicted value calculated in the predicted mode pred_mode_z whose prediction mode value is indicated by "4" is the z coordinate of the point p1 and is p1z.
  • the prediction mode pred_mode_z whose prediction mode value is indicated by "1" is selected in the table of FIG. 141, the z coordinate of the position information of the three-dimensional point to be encoded is coded using the predicted value p0z. It may be converted. In this case, "1" as the prediction mode value is added to the bit stream.
  • FIG. 142 is a diagram showing a seventh example of a table showing predicted values calculated in each prediction mode.
  • the seventh example is an example in which the position information used for the predicted value is the value of the element y and the element z of the position information of the surrounding three-dimensional points.
  • the predicted value calculated in the predicted mode pred_mode_yz whose predicted mode value is indicated by "0" is 0.
  • the predicted values calculated in the predicted mode pred_mode_yz whose predicted mode value is indicated by "1” are the y-coordinate and the z-coordinate of the point p0, and are (p0y, p0z).
  • the predicted value calculated in the predicted mode pred_mode_yz whose prediction mode value is indicated by "2" is a prediction result of linear prediction based on the y-coordinate and z-coordinate of the point p0 and the y-coordinate and z-coordinate of the point p1.
  • the predicted values calculated in the predicted mode pred_mode_yz whose prediction mode value is indicated by "3" are the y-coordinate and z-coordinate of the point p0, the y-coordinate and z-coordinate of the point p1, and the y-coordinate and z of the point p2. It is a prediction result of the Parallelog prediction based on the coordinates, and is (p0y + p1y-p2y, p0z + p1z-p2z). Further, the predicted values calculated in the predicted mode pred_mode_yz whose predicted mode value is indicated by "4" are the y-coordinate and the z-coordinate of the point p1, and are (p1y, p1z).
  • the prediction mode pred_mode_yz whose prediction mode value is indicated by "1" is selected in the table of FIG. 142, the y-coordinate and z-coordinate of the position information of the three-dimensional point to be encoded are set to the predicted value (p0y). , P0z). In this case, "1" as the prediction mode value is added to the bit stream.
  • the correspondence between the prediction mode and the prediction method of the calculated predicted value is the same as the above correspondence in the table in the second example.
  • the prediction mode at the time of coding may be selected by RD optimization. For example, it is conceivable to calculate the cost cost (P) when a certain prediction mode P is selected and select the prediction mode P in which the cost (P) is minimized.
  • the cost cost (P) is adjusted with, for example, the predicted residual error_value (P) when the predicted value of the predicted mode P is used, and the number of bits required to encode the predicted mode P bit (P). It may be calculated by the equation D1 using the parameter ⁇ value.
  • Abs (x) indicates the absolute value of x.
  • the square value of x may be used instead of abs (x).
  • the adjustment parameter ⁇ may be set to a different value depending on the value of the quantization scale. For example, when the quantization scale is small (at a high bit rate), the prediction mode in which the prediction residual residual_value (P) is small by reducing the ⁇ value is selected to improve the prediction accuracy as much as possible, and when the quantization scale is large. (At a low bit rate), an appropriate prediction mode may be selected by increasing the ⁇ value while considering the number of bits (P) required to encode the prediction mode P.
  • the case where the quantization scale is small is, for example, a case where it is smaller than the first quantization scale.
  • the case where the quantization scale is large is, for example, the case where it is larger than the second quantization scale which is equal to or larger than the first quantization scale.
  • the smaller the quantization scale the smaller the ⁇ value may be set.
  • the predicted residual error_value (P) is calculated by subtracting the predicted value of the prediction mode P from the position information of the three-dimensional point to be encoded. Instead of the predicted residual error_value (P) at the time of cost calculation, the predicted residual error_value (P) is quantized and inversely quantized, added to the predicted value to obtain the decoded value, and the position of the original three-dimensional point is obtained.
  • the difference (coding error) between the information and the decoded value when the prediction mode P is used may be reflected in the cost value. This makes it possible to select a prediction mode with a small coding error.
  • the number of bits (P) required to encode the prediction mode P may be, for example, the number of bits after binarization when the prediction mode is binarized and encoded.
  • the prediction mode value indicating the prediction mode may be binarized by a truncated unary code in which the maximum value is 5 using the number of prediction modes M.
  • 4 bits are used as the number of bits (P) required for encoding each prediction mode value.
  • the code amount of the prediction mode value indicating the prediction mode for calculating the prediction value that is easy to be selected, for example, the cost (P) is likely to be minimized, such as the position information of the three-dimensional point that is close to the three-dimensional point to be converted. Can be reduced.
  • the three-dimensional data encoding device may encode the prediction mode value indicating the selected prediction mode by using the number of prediction modes. Specifically, the three-dimensional data encoding device may encode the prediction mode value with a truncated number code having the maximum number of prediction modes.
  • the prediction mode value indicating the prediction mode may be binarized by the unary code as shown in FIG. 144. Further, when the probability of occurrence of each prediction mode is close, as shown in FIG. 145, the prediction mode value indicating the prediction mode may be binarized by the fixed code to reduce the code amount.
  • the number of bits (P) required to encode the prediction mode value indicating the prediction mode P the binary data of the prediction mode value indicating the prediction mode P is arithmetically coded, and the code amount after the arithmetic coding is performed. May be the value of bit (P).
  • the cost can be calculated using a more accurate required number of bits (P), so that a more appropriate prediction mode can be selected.
  • FIG. 143 is a diagram showing a first example of a binarization table in the case where the prediction mode value according to the eighth embodiment is binarized and encoded.
  • FIG. 144 is a diagram showing a second example of the binarization table in the case where the prediction mode value according to the eighth embodiment is binarized and encoded.
  • FIG. 145 is a diagram showing a third example of the binarization table in the case where the prediction mode value according to the eighth embodiment is binarized and encoded.
  • the prediction mode value indicating the prediction mode may be arithmetically encoded and added to the bit stream after binarization.
  • the prediction mode value may be binarized by, for example, a truncated unary code using a value of the number of prediction modes M. In this case, the maximum number of bits after binarization of the prediction mode value is M-1.
  • the binarized data may be arithmetically coded using a coding table.
  • the coding efficiency may be improved by switching the coding table for each bit of the binary data and coding.
  • the first bit one bit is encoded using the coding table A for one bit, and each bit of the remaining bits reminding bit is used for reminding bit. It may be encoded using the encoding table B.
  • the first bit “1” of one bit is encoded using the encoding table A, and the remaining bits are being remaining.
  • Each bit of "110" of the bit may be encoded by using the encoding table B.
  • FIG. 146 is a diagram for explaining an example of encoding the binary data of the binarization table when the prediction mode according to the eighth embodiment is binarized and encoded.
  • the coding efficiency can be improved by switching the coding table according to the bit position of the binary data while suppressing the number of coding tables.
  • the coding table may be further switched for each bit to perform arithmetic coding, or the coding table may be switched and decoded according to the result of arithmetic coding.
  • the prediction mode used in the truncated unary code is used so that the prediction mode can be specified from the binary data decoded on the decoding side.
  • a number M may be added to the header of the bit stream or the like.
  • Bitstream headers are, for example, sequence parameter sets (SPS), positional parameter sets (GPS), slice headers, and the like.
  • SPS sequence parameter sets
  • GPS positional parameter sets
  • slice headers are, for example, sequence parameter sets (SPS), positional parameter sets (GPS), slice headers, and the like.
  • the number of prediction modes M may be specified by a profile or level such as a standard without being added to the stream.
  • the predicted mode value binarized using the thresholded unary code is arithmetically coded by switching the coding table between the one bit part and the reminding part as described above.
  • the probability of occurrence of 0 and 1 in each coding table may be updated according to the value of the binary data actually generated. Further, the probability of occurrence of 0 and 1 in either coding table may be fixed. As a result, the number of updates of the occurrence probability may be suppressed to reduce the processing amount. For example, the probability of occurrence of the one bit portion may be updated, and the probability of occurrence of the reminding bit portion may be fixed.
  • FIG. 147 is a flowchart showing an example of coding of the prediction mode value according to the eighth embodiment.
  • FIG. 148 is a flowchart showing an example of decoding the predicted mode value according to the eighth embodiment.
  • the prediction mode value is first binarized by a thresholded mode using the number of prediction modes M (S9701).
  • the binary data of the truncated unary code is arithmetically encoded (S9702).
  • the bitstream contains binary data as a prediction mode.
  • the bit stream is arithmetically decoded using the number of predicted modes M, and binary data of a truncated unary code is generated (S9711).
  • the prediction mode value is calculated from the binary data of the truncated unary code (S9712).
  • the binary value is more than the case of the truncated unary code with the number of prediction modes M.
  • the number of bits after conversion can be reduced.
  • the number of bits after binarization of the predicted mode value may be reduced by binarizing the number L assigned to the predicted mode as the maximum value in the thresholded number code.
  • the binarized data may be arithmetically coded using a coding table.
  • the coding efficiency may be improved by switching the coding table for each bit of the binary data and coding.
  • the first bit one bit is encoded using the coding table A for one bit, and each bit of the remaining bits reminding bit is used for reminding bit. It may be encoded using the encoding table B.
  • the first bit one bit "1” is encoded by using the coding table A. Since there is no remaining bit remaining bit, it is not necessary to encode it. If there are remaining bit reminding bits, the remaining bits may be encoded using the coding table B.
  • FIG. 150 is a diagram for explaining an example of encoding the binary data of the binarization table when the prediction mode according to the eighth embodiment is binarized and encoded.
  • the coding efficiency can be improved by switching the coding table according to the bit position of the binary data while suppressing the number of coding tables.
  • the coding table may be further switched for each bit to perform arithmetic coding, or the coding table may be switched and decoded according to the result of arithmetic coding.
  • the prediction mode value is binarized and encoded by the truncated unary code using the number L to which the predicted value is assigned
  • the prediction mode is specified at the time of encoding so that the predicted mode can be specified from the binary data decoded on the decoding side.
  • the number L may be calculated by assigning the predicted value to the predicted mode in the same manner, and the predicted mode may be decoded using the calculated L.
  • the predicted mode value binarized using the thresholded unary code is arithmetically coded by switching the coding table between the one bit part and the reminding part as described above.
  • the probability of occurrence of 0 and 1 in each coding table may be updated according to the value of the binary data actually generated. Further, the probability of occurrence of 0 and 1 in either coding table may be fixed. As a result, the number of updates of the occurrence probability may be suppressed to reduce the processing amount. For example, the probability of occurrence of the one bit portion may be updated, and the probability of occurrence of the reminding bit portion may be fixed.
  • FIG. 151 is a flowchart showing another example of coding the prediction mode value according to the eighth embodiment.
  • FIG. 152 is a flowchart showing another example of decoding the predicted mode value according to the eighth embodiment.
  • the number L in which the prediction value is assigned to the prediction mode is calculated (S9721).
  • the prediction mode value is binarized by a truncated unary code using the number L (S9722).
  • the number L in which the predicted value is assigned to the predicted mode is calculated (S9731).
  • bit stream is arithmetically decoded using the number L to generate binary data of a truncated unary code (S9732).
  • the prediction mode value is calculated from the binary data of the truncated unary code (S9733).
  • the prediction mode value does not have to be added for all position information. For example, if certain conditions are met, the prediction mode is fixed so that the prediction mode value is not added to the bitstream, and if certain conditions are not met, the prediction mode is selected and the prediction mode value is added to the bitstream. You may. For example, if the condition A is satisfied, the prediction mode value is fixed to "2" and the prediction value is calculated from the linear prediction of the surrounding three-dimensional points. If the condition A is not satisfied, one prediction mode is selected from a plurality of prediction modes. Then, a prediction mode value indicating the selected prediction mode may be added to the bit stream.
  • the three-dimensional data encoding device determines that the difference between the predicted value by linear prediction and the position information of the point to be processed is small, and fixes the prediction mode value to "2". By not encoding the prediction mode value, it is possible to generate an appropriate prediction value while reducing the amount of coding for encoding the prediction mode.
  • the three-dimensional data coding apparatus may select the prediction mode and encode the prediction mode value indicating the selected prediction mode.
  • the threshold value Thfix may be added to the header of the bit stream or the like, and the encoder may change the value of the threshold value Thfix so that it can be encoded. For example, when encoding at a high bit rate, the encoder adds a threshold Thfix value to the header smaller than that at a low bit rate, and selects a prediction mode to increase the number of cases of coding. It may be encoded so that the predicted residual is small. Further, the encoder adds the value of the threshold value Thfix to the header at the time of coding at the low bit rate to be larger than that at the time of the high bit rate, and encodes with the prediction mode fixed.
  • the threshold value Thfix may be specified by a standard profile or level without being added to the bit stream.
  • the N 3D points around the 3D point to be coded used for prediction are N coded and decoded 3D points whose distance from the 3D point to be coded is smaller than the threshold THd. It is a point.
  • the maximum value of N may be added to the bitstream as a NumberNightborPoint.
  • the value of N does not always have to match the value of NuNeightborPoint, such as when the surrounding coded and decoded 3D points are less than the value of NuNeightborPoint.
  • the prediction mode value is fixed to "2" if the difference absolute value dustdiff used for prediction is smaller than the threshold value Thfix [i], but the prediction mode value is not necessarily limited to "0" to "0” to “. It may be fixed to any of "M-1". Further, a fixed prediction mode value may be added to the bitstream.
  • FIG. 153 is a flowchart showing an example of a process of determining whether or not to fix the prediction mode value according to the condition A at the time of coding according to the eighth embodiment.
  • FIG. 154 is a flowchart showing an example of a process of determining whether to set the prediction mode value to a fixed value or to decode according to the condition A at the time of decoding according to the eighth embodiment.
  • the three-dimensional data coding device determines whether or not the difference absolute value dustdiff is less than the threshold value Thfix (S9742).
  • the threshold value Thfix may be encoded and added to the header of the stream or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
PCT/JP2020/038359 2019-10-09 2020-10-09 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置 Ceased WO2021070952A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN202080066408.2A CN114467119A (zh) 2019-10-09 2020-10-09 三维数据编码方法、三维数据解码方法、三维数据编码装置及三维数据解码装置
JP2021551729A JP7633172B2 (ja) 2019-10-09 2020-10-09 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
US17/708,615 US12047605B2 (en) 2019-10-09 2022-03-30 Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
US18/736,882 US12581117B2 (en) 2019-10-09 2024-06-07 Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
JP2025018467A JP7812949B2 (ja) 2019-10-09 2025-02-06 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201962912865P 2019-10-09 2019-10-09
US62/912,865 2019-10-09
US201962913466P 2019-10-10 2019-10-10
US62/913,466 2019-10-10

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/708,615 Continuation US12047605B2 (en) 2019-10-09 2022-03-30 Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

Publications (1)

Publication Number Publication Date
WO2021070952A1 true WO2021070952A1 (ja) 2021-04-15

Family

ID=75436841

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/038359 Ceased WO2021070952A1 (ja) 2019-10-09 2020-10-09 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置

Country Status (4)

Country Link
US (2) US12047605B2 (https=)
JP (2) JP7633172B2 (https=)
CN (1) CN114467119A (https=)
WO (1) WO2021070952A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022224888A1 (ja) * 2021-04-21 2022-10-27 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
CN115361563A (zh) * 2022-07-07 2022-11-18 华南理工大学 H.264加密及密文域信息隐藏方法、装置及存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12249106B2 (en) * 2020-06-05 2025-03-11 Lg Electronics Inc. Point cloud data processing device and processing method
WO2021246796A1 (ko) * 2020-06-05 2021-12-09 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법
WO2021261499A1 (ja) * 2020-06-22 2021-12-30 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022025278A1 (ja) * 2020-07-31 2022-02-03 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022062369A1 (zh) 2020-09-25 2022-03-31 Oppo广东移动通信有限公司 点云编解码方法与系统、及点云编码器与点云解码器
US12033360B2 (en) 2020-10-05 2024-07-09 Qualcomm Incorporated Clipping laser indices in predictive geometry coding for point cloud compression
JP7678112B2 (ja) * 2020-12-22 2025-05-15 オッポ広東移動通信有限公司 点群の符号化方法、復号化方法、エンコーダ及びデコーダ
US12493293B2 (en) * 2021-03-31 2025-12-09 Denso Corporation Multi-dimensional extended trajectory planning with changing dimensionality based on horizon steps
CN115102935B (zh) * 2022-06-17 2024-02-09 腾讯科技（深圳）有限公司 一种点云编码方法、点云解码方法及相关设备
CN115379191B (zh) * 2022-08-22 2024-03-19 腾讯科技（深圳）有限公司 一种点云解码方法、点云编码方法及相关设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190080483A1 (en) * 2017-09-14 2019-03-14 Apple Inc. Point Cloud Compression
WO2019065298A1 (ja) * 2017-09-29 2019-04-04 ソニー株式会社 情報処理装置および方法
WO2019073939A1 (en) * 2017-10-12 2019-04-18 Mitsubishi Electric Corporation POINT CLOUD ENCODER

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104246831B (zh) 2012-07-30 2016-12-28 三菱电机株式会社 地图显示装置
US20150063455A1 (en) * 2013-09-02 2015-03-05 Humax Holdings Co., Ltd. Methods and apparatuses for predicting depth quadtree in three-dimensional video
CN104159117B (zh) 2014-07-22 2017-05-03 宁波大学 一种hevc视频的信息隐藏方法
US20200202608A1 (en) * 2018-12-21 2020-06-25 Point Cloud Compression, B.V. Method and apparatus for receiving a volumetric video stream
US11039115B2 (en) * 2018-12-21 2021-06-15 Samsung Electronics Co., Ltd. Low complexity color smoothing of reconstructed point clouds
US20200211290A1 (en) * 2018-12-26 2020-07-02 Lg Electronics Inc. Xr device for providing ar mode and vr mode and method for controlling the same
US11210812B2 (en) * 2019-05-15 2021-12-28 Samsung Electronics Co., Ltd. Single-pass boundary detection in video-based point cloud compression
US11122102B2 (en) * 2019-07-03 2021-09-14 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus and point cloud data reception method
JP7640526B2 (ja) * 2019-08-14 2025-03-05 エルジー エレクトロニクス インコーポレイティド ポイントクラウドデータ送信装置、ポイントクラウドデータ送信方法、ポイントクラウドデータ受信装置及びポイントクラウドデータ受信方法。
KR102476057B1 (ko) * 2019-09-04 2022-12-09 주식회사 윌러스표준기술연구소 클라우드 가상 현실을 위한 imu 센서 데이터를 활용한 비디오 인코딩 및 디코딩 가속 방법 및 장치
WO2021066615A1 (en) * 2019-10-03 2021-04-08 Lg Electronics Inc. Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method
PL4011088T3 (pl) * 2019-10-03 2024-09-16 Lg Electronics Inc. Urządzenie transmisyjne danych z chmury punktów, sposób transmisji danych z chmury punktów, urządzenie odbiorcze danych z chmury punktów oraz sposób odbioru danych z chmury punktów
US11551334B2 (en) * 2020-01-09 2023-01-10 Tencent America LLC Techniques and apparatus for coarse granularity scalable lifting for point-cloud attribute coding
US11373276B2 (en) * 2020-01-09 2022-06-28 Tencent America LLC Techniques and apparatus for alphabet-partition coding of transform coefficients for point cloud compression

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190080483A1 (en) * 2017-09-14 2019-03-14 Apple Inc. Point Cloud Compression
WO2019065298A1 (ja) * 2017-09-29 2019-04-04 ソニー株式会社 情報処理装置および方法
WO2019073939A1 (en) * 2017-10-12 2019-04-18 Mitsubishi Electric Corporation POINT CLOUD ENCODER

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022224888A1 (ja) * 2021-04-21 2022-10-27 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
US12413781B2 (en) 2021-04-21 2025-09-09 Panasonic Intellectual Property Corporation Of America Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
CN115361563A (zh) * 2022-07-07 2022-11-18 华南理工大学 H.264加密及密文域信息隐藏方法、装置及存储介质

Also Published As

Publication number Publication date
US20220224941A1 (en) 2022-07-14
US20240323438A1 (en) 2024-09-26
US12581117B2 (en) 2026-03-17
JPWO2021070952A1 (https=) 2021-04-15
JP2025065349A (ja) 2025-04-17
CN114467119A (zh) 2022-05-10
JP7633172B2 (ja) 2025-02-19
US12047605B2 (en) 2024-07-23
JP7812949B2 (ja) 2026-02-10

Similar Documents

Publication Publication Date Title
JP7812949B2 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
US20250234039A1 (en) Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2021261516A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022075301A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2021256559A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2021002443A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
US20240373044A1 (en) Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2022075428A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2021261555A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2021187561A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022075319A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2021210548A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022075326A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
US20250095219A1 (en) Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2021141090A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022149589A1 (ja) 三次元データ復号方法、三次元データ符号化方法、三次元データ復号装置、及び三次元データ符号化装置
WO2021187565A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022075234A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
EP4138040A1 (en) Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2022004682A1 (ja) 三次元データ符号化方法及び三次元データ符号化装置
US12536707B2 (en) Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2021261458A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022071505A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2021193899A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2022059697A1 (ja) 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20874265

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021551729

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20874265

Country of ref document: EP

Kind code of ref document: A1