WO2022054744A1 - 情報処理装置および方法 - Google Patents

情報処理装置および方法 Download PDF

Info

Publication number
WO2022054744A1
WO2022054744A1 PCT/JP2021/032594 JP2021032594W WO2022054744A1 WO 2022054744 A1 WO2022054744 A1 WO 2022054744A1 JP 2021032594 W JP2021032594 W JP 2021032594W WO 2022054744 A1 WO2022054744 A1 WO 2022054744A1
Authority
WO
WIPO (PCT)
Prior art keywords
slice
information
geometry
track
depth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/032594
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
遼平 高橋
光浩 平林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to US18/040,820 priority Critical patent/US12341981B2/en
Priority to JP2022547574A priority patent/JP7746995B2/ja
Priority to CN202180053432.7A priority patent/CN116157838B/zh
Publication of WO2022054744A1 publication Critical patent/WO2022054744A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present disclosure relates to information processing devices and methods, and more particularly to information processing devices and methods capable of suppressing an increase in the load of reproduction processing.
  • HEVC High Efficiency Video Coding
  • ISOBMFF International Organization for Standardization Base Media
  • MPEG-4 Motion Picture Experts Group-4
  • the coded data can be hierarchically structured according to the resolution, for example, to support scalable decoding. Then, a file format has also been proposed in which the coded data is divided into tracks for each layer and the coded data of a desired layer can be selectively transmitted (see, for example, Non-Patent Document 2).
  • G-PCC Geometry-based Point Cloud Compression
  • Non-Patent Document 5 did not correspond to the slice structure as described in Non-Patent Document 4. Therefore, in order to decode a part of the slice of the G-PCC content, the entire G-PCC content must be transmitted and parsed (analyzed), which may increase the load of the reproduction process.
  • This disclosure has been made in view of such a situation, and is intended to enable the increase in the load of the reproduction process to be suppressed.
  • the information processing device of one aspect of the present technology includes depth information indicating the quality hierarchy level of each slice in the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice, and the G- A scalable decoding information generation unit that generates scalable decoding information regarding the scalable decoding of the G-PCC content based on the dependency between the first slice and the second slice in the PCC content, and the G-PCC.
  • the information processing method of one aspect of the present technology includes depth information indicating the quality hierarchy level of each slice in G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice, and the G- A content file that generates scalable decoding information regarding the scalable decoding of the G-PCC content based on the dependency between the first slice and the second slice in the PCC content and stores the G-PCC content.
  • G-PCC Geometry-based Point Cloud Compression
  • the information processing device of another aspect of the present technology is scalable decoding stored in the metadata area of the content file that stores the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice. It is provided with an extraction unit that extracts an arbitrary slice of the G-PCC content from the content file based on the information, and a decoding unit that decodes the slice of the G-PCC content extracted by the extraction unit.
  • the scalable decoding information is information related to the scalable decoding of the G-PCC content, and is depth information indicating the quality hierarchy level of the slice in the G-PCC content, and the first slice and the first slice in the G-PCC content. It is an information processing device that is information generated based on the dependency between two slices.
  • Another aspect of the information processing method of the present technology is scalable decoding stored in the metadata area of the content file that stores the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice.
  • G-PCC Geometry-based Point Cloud Compression
  • an arbitrary slice of the G-PCC content is extracted from the content file, the slice of the extracted G-PCC content is decoded, and the scalable decoding information is the G-PCC content.
  • Information about scalable decoding based on depth information indicating the quality hierarchy level of the slice in the G-PCC content and the dependency between the first slice and the second slice in the G-PCC content. It is an information processing method that is generated information.
  • the depth information indicating the quality hierarchy level of each slice in the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice is used.
  • G-PCC Geometry-based Point Cloud Compression
  • scalable decoding information regarding the scalable decoding of the G-PCC content is generated, and the content that stores the G-PCC content.
  • a file is generated and its scalable decryption information is stored in the metadata area of the content file.
  • the information processing apparatus and method of another aspect of the present technology is stored in the metadata area of the content file that stores the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice.
  • G-PCC Geometry-based Point Cloud Compression
  • any slice of the G-PCC content is extracted from the content file, and the slice of the extracted G-PCC content is decoded.
  • Non-Patent Document 1 (above) Non-Patent Document 2: (above) Non-Patent Document 3: (above) Non-Patent Document 4: (above) Non-Patent Document 5: (above) Non-Patent Document 6: https://www.matroska.org/index.html
  • HEVC High Efficiency Video Coding
  • MPEG-4 Motion Picture Experts Group-4
  • ISOBMFF International Organization for Standardization Base Media File Format
  • the coded data can be hierarchically structured according to the resolution, for example, to support scalable decoding.
  • a file format (L-HEVC file) in which coded data can be stored separately in tracks for each layer and only the coded data in a desired layer can be selectively transmitted. format) was also proposed.
  • the bitstream of the moving image has a hierarchical structure, and one or more layers (layers (s)) of the bitstream are made into individual tracks of ISOBMFF. Can be stored.
  • the information of the layer included in each sample is stored in the sample group (Layer Information sample group).
  • the hierarchical structure does not change frequently because inter prediction is applied. Therefore, it is preferable to store the hierarchical information in a sample group. This sample group can also be used as information for track selection.
  • 3D data expressing a three-dimensional object also referred to as a 3D object
  • a 3D object that is a three-dimensional structure is expressed as a set of a large number of points.
  • the point cloud is composed of position information (also referred to as geometry) and attribute information (also referred to as attribute) of each point.
  • Attributes can contain any information.
  • the attributes may include color information, reflectance information, normal information, etc. of each point.
  • the point cloud has a relatively simple data structure and can express the three-dimensional shape of a 3D object with sufficient accuracy by using a sufficiently large number of points.
  • Non-Patent Document 3 discloses a coding technique called Geometry-based Point Cloud Compression (G-PCC), which encodes this point cloud separately into geometry and attributes.
  • G-PCC is in the process of being standardized in MPEG-I Part 9 (ISO / IEC 23090-9).
  • Tree structure coding as shown in FIG. 2 is applied to the geometry coding.
  • the geometry is first tree-structured.
  • the three-dimensional space is recursively divided, and the geometry is quantized in the divided regions of each stage to form a tree structure of the geometry as shown in FIG.
  • a tree structure layered in depths also referred to as LODs
  • LODs also referred to as LODs
  • the geometry of each layer is encoded according to this tree structure. For example, the difference between the geometry of each layer and the geometry of the next higher layer to which the geometry belongs is calculated, and the difference is encoded.
  • each layer may be encoded independently, the coding efficiency can be improved by encoding the difference in this way.
  • the depth and the coded data of the layer higher than the depth may be decoded.
  • a geometry having a resolution of depth 2 can be obtained.
  • decoding of Depth 3 to Depth 6 is unnecessary.
  • a decoding method capable of restoring (generating) information in a desired hierarchy only by decoding a part of the coded data in this way is referred to as scalable decoding. That is, by tree-structuring and encoding as described above, the geometry can be scalablely decoded by resolution.
  • a bifurcated tree is shown as an example of a tree structure of this geometry, but any tree structure can be applied.
  • it may be an octree or a kd tree.
  • the coded data (bitstream) generated by encoding the geometry as described above is also referred to as a geometry bitstream.
  • bitstream that combines a geometry bitstream and an attribute bitstream into one is also called a G-PCC bitstream or G-PCC content.
  • Non-Patent Document 4 it was proposed to form a slice structure in a bitstream in this G-PCC.
  • a slice is a unit that divides geometry and attribute data.
  • a slice of geometry is also referred to as a geometry slice.
  • the attribute slice is also referred to as an attribute slice.
  • Slices are separated by the depth of the geometric tree structure. That is, a slice is composed of data of a single depth or a plurality of consecutive depths. For example, in FIG. 2, the data area separated by the thick line frame indicates the slice. The numbers circled in the figure indicate the identification information of each slice. For example, the data of depth 0 to depth 3 form slice # 1.
  • the slices can also be separated by positions (regions) in the three-dimensional space. For example, in FIG. 2, the depth 4 and depth 5 data are divided into regions A to D, forming four slices of slice # 2, slice # 3, slice # 4, and slice # 5. Similarly, the depth 6 data is divided into regions A to D to form four slices, slice # 6, slice # 7, slice # 8, and slice # 9.
  • Geometry and attribute data is encoded for each slice. That is, the geometry bitstream and the attribute bitstream can be decoded for each slice.
  • the geometry encodes the difference from the upper layer, so the slice must be an independent slice that can be decoded independently and a dependent slice that requires another slice for decoding. There are two types of slice).
  • the geometry of slice # 1 may be decoded.
  • the geometry of slice # 2 for example, the geometry of region A of depth 4
  • decoding of slice # 3 is also required for decoding of slice # 3 to slice # 5.
  • Decoding of slice # 7 requires decoding of slice # 1 and slice # 3.
  • Decoding of slice # 8 requires decoding of slice # 1 and slice # 4.
  • Decoding of slice # 9 requires decoding of slice # 1 and slice # 5.
  • slice # 1 is an independent slice
  • slices # 2 to # 9 are dependent slices.
  • FIG. 3 is a diagram illustrating a main configuration example of a bit stream having such a slice structure.
  • the bitstream 30, shown in gray in FIG. 3 is a point cloud bitstream in which the geometry is tree-structured and sliced and encoded as in FIG. In FIG. 3, a part thereof is illustrated.
  • the bitstream 30 includes a sequence parameter set (SPS (Sequence Parameter Set)), a geometry parameter set (GPS (Geometry Parameter Set)), an attribute parameter set (APS (Attribute Parameter Set)), and tiles.
  • SPS Sequence Parameter Set
  • GPS Global System Parameter Set
  • APS Attribute Parameter Set
  • tiles Has an inventory (tile inventory).
  • a sequence parameter set is a parameter set for the entire sequence.
  • a geometry parameter set is a set of parameters related to geometry. The geometry parameter set may be different for each geometry slice.
  • An attribute parameter set is a parameter set for an attribute. The attribute parameter set may be different for each attribute slice.
  • the tile inventory stores the position information of the tile. The number of tiles and their position information are variable for each frame.
  • a point cloud bitstream is placed for each sample.
  • the sample is a point cloud at a certain time, which corresponds to a frame of a moving image.
  • the bitstream is arranged slice by slice.
  • the geometry bitstream and the attribute bitstream are arranged in that order.
  • each square indicates a data unit.
  • the data unit of "geom_slice # 1" is a data unit that stores the geometry bitstream of slice # 1.
  • the data unit of "attr slice (s)” following this data unit is a data unit that stores the attribute bit stream of slice # 1.
  • Geometry and attribute data units that make up the same slice are assigned the same slice identification information (slice_id) to each other.
  • the attributes can be divided into slices for each parameter, and a plurality of attribute slices can be stored in one data unit. That is, these data units are data units corresponding to slice # 1 which is an independent slice, and store the data of depth 0 to depth 3 of the tree structure of FIG.
  • a data unit that stores a bitstream of geometry is also referred to as a geometry data unit.
  • a data unit that stores a bitstream of attributes is also referred to as an attribute data unit.
  • the data unit of "geom_slice # 2" is a geometry data unit that stores the geometry bitstream of slice # 2.
  • the data unit of "attr slice (s)" following this geometry data unit is an attribute data unit that stores the attribute bitstream of slice # 2. That is, these data units are data units corresponding to slice # 2, which is a dependent slice, and store the data in the area A of the depth 4 and the depth 5 of the tree structure of FIG. Slice # 2 is subordinate to slice # 1 as shown by arrow 32.
  • the data unit of "geom_slice # 6" is a geometry data unit that stores the geometry bitstream of slice # 6.
  • the data unit of "attr slice (s)" following this geometry data unit is an attribute data unit that stores the attribute bitstream of slice # 6. That is, these data units are data units corresponding to slice # 6, which is a dependent slice, and store the data in the region A of the depth 6 of the tree structure of FIG. Slice # 6 is directly dependent on slice # 2, as indicated by arrow 33. In other words, slice # 6 is indirectly dependent on slice # 1.
  • the data unit of "geom_slice # 3" is a geometry data unit that stores the geometry bitstream of slice # 3.
  • the data unit of "attr slice (s)" following this geometry data unit is a data unit that stores the attribute bitstream of slice # 3. That is, these data units are data units corresponding to slice # 3, which is a dependent slice, and store the data in the area B of the depth 4 and the depth 5 of the tree structure of FIG. Slice # 3 is subordinate to slice # 1 as shown by arrow 34.
  • the data unit of "geom_slice # 7" is a geometry data unit that stores the geometry bitstream of slice # 7.
  • the data unit of "attr slice (s)" following this geometry data unit is a data unit that stores the attribute bitstream of slice # 7. That is, these data units are data units corresponding to slice # 7, which is a dependent slice, and store the data in the region B of the depth 6 of the tree structure of FIG.
  • Slice # 7 is directly dependent on slice # 3, as shown by arrow 35. In other words, slice # 7 is indirectly dependent on slice # 1.
  • data units (geometry data unit and attribute data unit) corresponding to each of slice # 4, slice # 8, slice # 5, and slice # 9 are similarly arranged thereafter.
  • the slice and the tile are linked by the tile identification information (tile_id) stored in the geometry data unit.
  • the decoder parses the bitstream to understand which depth of data corresponds to which part of the bitstream. I had to do it.
  • the bitstream forms the slice structure as described above, the decoder can easily select the data to be decoded in slice units.
  • the decoder may decode the bitstream stored in the data unit corresponding to slice # 1 of FIG. Further, when obtaining the data of the slice # 2, the decoder may decode the bit stream stored in the data unit corresponding to the slice # 1 and the bit stream stored in the data unit corresponding to the slice # 2.
  • the decoder corresponds to the bit stream stored in the data unit corresponding to slice # 1, the bit stream stored in the data unit corresponding to slice # 2, and the slice # 6. The bit stream stored in the data unit may be decoded.
  • the decoder may decode the bitstream stored in the data unit corresponding to slice # 1 and the bitstream stored in the data unit corresponding to slice # 3. .
  • the decoder corresponds to the bit stream stored in the data unit corresponding to slice # 1, the bit stream stored in the data unit corresponding to slice # 3, and the slice # 7.
  • the bit stream stored in the data unit may be decoded.
  • the decoder may decode the bit stream stored in the data unit corresponding to slice # 1 and the bit stream stored in the data unit corresponding to slice # 4.
  • the decoder corresponds to the bit stream stored in the data unit corresponding to slice # 1, the bit stream stored in the data unit corresponding to slice # 4, and the slice # 8.
  • the bit stream stored in the data unit may be decoded.
  • the decoder may decode the bit stream stored in the data unit corresponding to slice # 1 and the bit stream stored in the data unit corresponding to slice # 5.
  • the decoder corresponds to the bit stream stored in the data unit corresponding to slice # 1, the bit stream stored in the data unit corresponding to slice # 5, and the slice # 9.
  • the bit stream stored in the data unit may be decoded.
  • the decoder can perform scalable decoding more easily.
  • an independent slice of geometry is also referred to as an independent geometry slice.
  • Dependent slices of geometry are also referred to as dependent geometry slices.
  • Independent slices of attributes are also referred to as independent attribute slices.
  • Dependent slices of an attribute are also called dependent attribute slices.
  • Non-Patent Document 5 discloses a method of storing G-PCC content in ISOBMFF for the purpose of improving the efficiency of reproduction processing of G-PCC content (G-PCC bitstream) from local storage and network distribution. This method is being standardized in MPEG-I Part 18 (ISO / IEC 23090-18).
  • FIG. 4 is a diagram showing an example of the file structure in that case.
  • a content file in which G-PCC contents are stored in ISOBMFF is also referred to as a content file.
  • the sequence parameter set is stored in GPCCDecoderConfigurationRecord in the metadata area of the content file.
  • the GPCCDecoderConfigurationRecord may further include a geometry parameter set, an attribute parameter set, and a tile inventory, depending on the sample entry type.
  • the sample of the media data box (Media) includes a geometry slice (geometry slice) and an attribute slice (attribute slice) corresponding to a 1-point cloud frame (point cloud frame).
  • geometry slice geometry slice
  • attribute slice attribute slice
  • tile inventories depending on the sample entry type.
  • Non-Patent Document 5 did not disclose that G-PCC content having such a slice structure is stored in ISOBMFF. Therefore, even when decoding a part of the slice of the G-PCC content, the entire G-PCC content must be transmitted, which may increase the amount of data to be transmitted. Further, even if the bitstream is divided by using tracks, the information necessary for scalable decoding such as which depth of data is contained in which track exists only in the bitstream, so that the decoder determines the bitstream. I needed to parse. Furthermore, since G-PCC is applied with intra coding, it is generally possible that the depth hierarchy changes for each sample (frame). Therefore, the decoder had to parse the entire bitstream to get the information needed for scalable decoding. In this way, the load of the reproduction process may increase.
  • the scalable decoding information of the G-PCC content having a slice structure stored in the metadata area of the content file is transmitted ().
  • Method 1 the scalable decoding information is information relating to the scalable decoding of the G-PCC content having a slice structure (used for the scalable decoding information). Further, this scalable decoding information is set based on the depth of each slice of the G-PCC content having a slice structure and the dependency between the slices.
  • depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC content including the first slice and the second slice, and the first slice in the G-PCC content.
  • a scalable decoding information generator that generates scalable decoding information regarding the scalable decoding of the G-PCC content and a content file that stores the G-PCC content are generated. It is provided with a content file generation unit that stores the scalable decryption information in the metadata area of the content file.
  • the depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC content including the first slice and the second slice, and the first slice in the G-PCC content.
  • the second slices Based on the dependencies between the second slices, it generates scalable decryption information about the scalable decryption of the G-PCC content, generates a content file to store the G-PCC content, and puts the scalable decryption information into it. Store it in the metadata area of the content file.
  • the information processing apparatus from the content file, based on the scalable decoding information stored in the metadata area of the content file containing the G-PCC content including the first slice and the second slice. It is provided with an extraction unit that extracts an arbitrary slice of G-PCC content and a decoding unit that decodes a slice of G-PCC content extracted by the extraction unit.
  • the scalable decoding information is information related to scalable decoding of the G-PCC content, and the depth information indicating the quality hierarchy level of the geometry included in the slice in the G-PCC content and the first slice in the G-PCC content. Information generated based on the dependencies between the second slices.
  • the G- Extract any slice of PCC content and decode the extracted slice of G-PCC content.
  • the scalable decoding information is information related to scalable decoding of the G-PCC content, and the depth information indicating the quality hierarchy level of the geometry included in the slice in the G-PCC content and the first slice in the G-PCC content. Information generated based on the dependencies between the second slices.
  • the decoder extracts and decodes the slices necessary to reproduce the point cloud of the desired depth and area based on the scalable decoding information in the metadata area of the content file, and presents the information. Can be generated. This makes it possible to reduce unnecessary processing of the decoder (transmission of unnecessary information, parsing of bitstream, etc.). Therefore, it is possible to suppress an increase in the load of the reproduction process.
  • G-PCC content there is encoding of large-scale point cloud data such as point cloud map data and virtual assets in movie production (digital data of a real movie set).
  • large-scale point cloud the amount of data is very large as a whole, and it is not realistic to reproduce all of them from the viewpoint of processing load and processing delay. Therefore, it is desired to perform scalable decoding in which only a part of the data is reproduced by limiting the reproduction area or reducing the resolution.
  • the scalable decoding information is stored in the metadata area of the content file, and by supporting the above-mentioned scalable decoding, only the necessary information is transmitted. Or, it will be possible to more easily decode only the necessary information.
  • the larger the data the more the increase in the load of the reproduction process can be suppressed, and the larger effect can be obtained.
  • a structure that stores geometry and attributes in one track also called a single track encapsulation structure
  • a structure that stores geometry and attributes in different tracks also called multi-track encapsulation structure. Also called).
  • multi-track encapsulation structure a structure that stores geometry and attributes in different tracks.
  • the present technology can be applied to the case of multi-track as well as the case of single track. Even in the case of a single track, the number of tracks may be multiple (there may be a plurality of tracks including geometry and attributes).
  • SPS, GPS, APS, and tile inventory are assumed to be stored in the GPCCDecoderConfigurationRecord, and the sample contains only geometry slices and attribute slices.
  • the sample contains only geometry slices and attribute slices.
  • some or all of the SPS, GPS, APS, and tile inventory may be stored in the sample.
  • the scalable decoding information may include slice composition information for each sample (method 1-1).
  • the slice composition information is information regarding the composition of slices in the sample. That is, the decoder can obtain the configuration information of each sample slice from the metadata area of the content file. Therefore, the decoder can grasp the composition of each sample slice without parsing the bitstream (ie, more easily).
  • this slice configuration information is the codec specific parameters of the subsample information box (SubSampleInformationBox) in the metadata area of the content file. ) May be stored (method 1-1-1).
  • the content file generation unit of the encoder may set a subsample for each slice and store the slice configuration information in the codec spiritual parameters of the subsample information box in the metadata area of the content file.
  • the extractor of the decoder is based on the slice configuration information stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice, from the content file. , Any slice of G-PCC content may be extracted.
  • the increase in file size is suppressed by setting a subsample for each slice and storing the slice configuration information in the codec spiritual parameters of the subsample information box in the metadata area of the content file. Can be done. Further, since the decoder only needs to confirm the codec spiritual parameters of the desired subsample information box, the slice configuration information can be confirmed more easily. That is, it is possible to suppress an increase in the load of the reproduction process.
  • slice dependency information is information indicating a dependency between slices or slice groups.
  • the slice dependency information indicates the dependency between the first slice and the second slice contained in the G-PCC content.
  • a slice group is a plurality of slices corresponding to the same depth.
  • bitstream 100 shown in FIG. 7 shows a part of the same G-PCC content as the bitstream 30 in FIG.
  • the bitstream 101 shown in gray shows a partial configuration in the sample of the bitstream 100.
  • Each data unit of the bitstream 101 has a dependency between slices similar to that in FIG. 3, as shown by arrows 111 to 114.
  • the geometry data unit of "geom_slice # 2" followed by the attribute data unit of "attr slice (s)” is the data unit corresponding to slice # 2, and as shown by arrow 111, of slice # 1. It depends on the data unit (the geometry data unit of "geom_slice # 1" followed by the attribute data unit of "attr slice (s)"). That is, slice # 2 is subordinate to slice # 1.
  • the geometry data unit of "geom_slice # 6" and the attribute data unit of "attr slice (s)" following it are the data units corresponding to slice # 6, and as shown by arrow 112, the attribute data unit of slice # 2 It is directly dependent on the data unit (the geometry data unit of "geom_slice # 2" followed by the attribute data unit of "attr slice (s)"). That is, slice # 6 is directly dependent on slice # 2. In other words, slice # 6 is indirectly dependent on slice # 1.
  • the geometry data unit of "geom_slice # 3" and the attribute data unit of "attr slice (s)" following it are data units corresponding to slice # 3, and as shown by arrow 113, the attribute data unit of slice # 1 It depends on the data unit (the geometry data unit of "geom_slice # 1" followed by the attribute data unit of "attr slice (s)"). That is, slice # 3 is subordinate to slice # 1.
  • the geometry data unit of "geom_slice # 7" and the attribute data unit of "attr slice (s)" following it are the data units corresponding to slice # 7, and as shown by arrow 114, the attribute data unit of slice # 3 It is directly dependent on the data unit (the geometry data unit of "geom_slice # 3" followed by the attribute data unit of "attr slice (s)"). That is, slice # 7 is directly dependent on slice # 3. In other words, slice # 7 is indirectly dependent on slice # 1.
  • the information indicated by the arrows (for example, arrows 111 to 114) between the slices surrounded by the dotted frame 121 is the slice dependency information.
  • This slice dependency information is stored in the metadata area of the content file. By doing so, the decoder can obtain this slice dependency information from the metadata area of the content file. Therefore, the decoder can grasp the slice dependency information without parsing the bitstream (ie, more easily).
  • this slice dependency information may be stored in the codec spiritual parameters of the subsample information box in the metadata area of the content file.
  • the content file generation unit of the encoder may set a subsample for each slice and store the slice dependency information in the codec spiritual parameters of the subsample information box in the metadata area of the content file.
  • the extractor of the decoder is based on the slice dependency information stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice. Any slice of G-PCC content may be extracted from.
  • a subsample is set for each slice, and the slice dependency information is stored in the codec spiritual parameters of the subsample information box in the metadata area of the content file to suppress the increase in file size. be able to.
  • the slice dependency can be confirmed more easily. That is, it is possible to suppress an increase in the load of the reproduction process.
  • this slice dependency information may include the reference source geometry slice identification information and the reference destination geometry slice identification information (method 1-). 1-2-2).
  • the reference source geometry slice identification information is the identification information of the geometry slice to which the information corresponds. That is, the reference source geometry slice identification information is the identification information of the geometry slice that is the reference source (the start point of the arrow in the dotted line frame 121 in FIG. 7) in the above-mentioned dependency relationship between the slices or slice groups.
  • the referenced geometry slice identification information is the identification information of another geometry slice referred to by the corresponding geometry slice.
  • the reference-destination geometry slice identification information is the identification information of the geometry slice that becomes the reference destination (the end point of the arrow in the dotted line frame 121 in FIG. 7) in the above-mentioned dependency relationship between the slices or the slice group.
  • the decoder can obtain the reference source geometry slice identification information and the reference destination geometry slice identification information from the metadata area of the content file. Therefore, the decoder can grasp the reference source geometry slice identification information and the reference destination geometry slice identification information without parsing the bitstream (that is, more easily).
  • the slice corresponding to this slice dependency information (that is, the slice corresponding to the reference source geometry slice identification information) is an independent geometry slice.
  • the reference source geometry slice identification information and the reference destination geometry slice identification information may be the same (method 1-1-2-2-1). That is, the independent geometry slice can be decoded without the need for another slice, so the reference destination may be set to the independent geometry slice itself. Further, for the independent geometry slice, the storage of the referenced geometry slice identification information may be omitted.
  • the reference source geometry slice identification information and the reference destination geometry slice identification information are codec speeds of the subsample information box in the metadata area of the content file. It may be stored in Siffic Parameters (Method 1-1-2-2-1).
  • the content file generator of the encoder sets a subsample for each slice, and sets the reference source geometry slice identification information and the reference destination geometry slice identification information in the codec spiritual parameters of the subsample information box of the metadata area. May be stored.
  • the extractor of the decoder uses the reference source geometry slice identification information and the reference destination geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice. Based on that, any slice of G-PCC content may be extracted from the content file.
  • Figure 8 shows an example of the syntax of the subsample information box.
  • flags are set as shown in the line underlined 131.
  • codec spiritual parameters are provided.
  • the codec spiritual parameters store subsample information determined for each codec codec.
  • Figure 9 shows an example of the codec spiritual parameters syntax.
  • the reference source geometry slice identification information (geom_slice_id) and the reference destination geometry slice identification information (ref_geom_slice_id) are stored in this codec spiritual parameters.
  • the slice identification information (slice_id) of the geometry slice corresponding to this information is set in geom_slice_id.
  • the ref_geom_slice_id is set to the slice identification information (slice_id) of other geometry slices referenced by that geometry slice.
  • the reference relationship from the attribute slice to the geometry slice may be specified.
  • the slice dependency information may include the attribute geometry slice identification information (method 1-1-2-3).
  • the attribute geometry slice identification information is the identification information of the geometry slice referenced by the attribute slice.
  • the content file generation unit of the encoder may set a subsample for each slice and store the attribute geometry slice identification information in the codec spiritual parameters of the subsample information box of the metadata area.
  • the extractor of the decoder uses the attribute geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice from the content file. Any slice of G-PCC content may be extracted.
  • FIG. 10 shows an example of the codec Spicic Parameters syntax in this case.
  • This codec-spicy parameters stores the attribute geometry slice identification information (ref_attr_geom_slice_id), as shown in the underlined line.
  • This ref_attr_geom_slice_id is set to the slice identification information (slice_id) of the geometry slice referenced by the attribute slice to which the information corresponds.
  • a subsample is set for each slice, and the attribute geometry slice identification information is stored in the codec spiritual parameters of the subsample information box in the metadata area of the content file to suppress the increase in file size. be able to.
  • the decoder since the decoder only needs to confirm the codec-spicy parameters of the desired subsample information box, it is possible to more easily confirm the geometry slice referenced from the attribute. That is, it is possible to suppress an increase in the load of the reproduction process.
  • Non-scalable encoding may be applied to the attributes.
  • Non-scalable coding is a coding method that does not support scalable decoding. For example, if a non-scalable encoding is applied to an attribute, the attribute slice is set to be able to be decoded independently of the other attribute slices (ie, without reference to the other attribute slices). Therefore, data (depth) overlaps between attribute data units.
  • FIG. 11 shows an example of a bitstream configuration when non-scalable coding is applied to the attributes in this way.
  • the gray bitstream 141 shown in FIG. 11 shows a part of the configuration of the G-PCC content (G-PCC bitstream).
  • the attribute data unit 151 stores the attribute slice corresponding to the geometry of depth 0 to depth 3.
  • the attribute data unit 152 stores the attribute slices corresponding to the geometries of depth 0 to depth 5.
  • the attribute data unit 153 stores the attribute slices corresponding to the geometries of depth 0 to depth 6.
  • the attribute data unit 153 by decoding the attribute data unit 153, the attribute corresponding to the geometry of the depth 0 to the depth 6 can be obtained.
  • the attribute data unit 153 may be decoded (there is no need to decode other attribute data units).
  • the attribute data unit 152 by decoding the attribute data unit 152, the attributes corresponding to the geometries of depth 0 to depth 5 can be obtained. In other words, in order to obtain the attribute corresponding to the geometry of depth 4 or depth 5, the attribute data unit 152 may be decoded (no need to decode other attribute data units).
  • the attribute data unit 151 may be decoded (no need to decode other attribute data units) in order to obtain the attribute corresponding to any of the 0 to 3 depth geometry.
  • the attribute geometry slice identification information includes a slice of the geometry slice that corresponds to the bottom layer (maximum depth) of the depth that corresponds to that attribute slice (attribute data unit). Identification information is set.
  • the reference relationship between geometry slices is indicated by ref_geom_slice_id.
  • the attribute geometry slice identification information when the non-scalable coding is applied to the attribute is also referred to as the non-scalable coding attribute geometry slice identification information.
  • the slice dependency information is non-scalable coding which is the identification information of the geometry slice referred to by the attribute slice to which the non-scalable coding is applied. Attribute geometry slice identification information may be included (Method 1-1-2-4).
  • this non-scalable coded attribute geometry slice identification information is the geometry slice or the group of geometry slices referred to by the attribute slice corresponding to the information.
  • the geometry slice containing the maximum depth information or the identification information of the geometry slice group may be included (method 1-1-2-4-1).
  • the geometry slice group is a plurality of geometry slices corresponding to the same depth.
  • the non-scalable coded attribute geometry slice identification information is stored in the codec spiritual parameters of the subsample information box in the metadata area of the content file. It may be done (method 1-1-2-4-2).
  • the content file generator of the encoder sets a subsample for each slice and stores the non-scalable coded attribute geometry slice identification information in the codec spiritual parameters of the subsample information box in the metadata area of the content file. You may.
  • the extractor of the decoder is based on the non-scalable coded attribute geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice. Then, any slice of G-PCC content may be extracted from the content file.
  • the file size The increase can be suppressed.
  • the decoder only needs to check the codec-sensitive parameters of the desired subsample information box, making it easier to reference geometry slices from the attribute, even if non-scalable coding is applied to the attribute. Can be confirmed. That is, it is possible to suppress an increase in the load of the reproduction process.
  • the slice dependency information may include a non-scalable coding flag (method 1-1-2-4-3).
  • the non-scalable coding flag is flag information indicating whether or not non-scalable coding has been applied to the attribute slice. By storing such information, the decoder can easily determine whether or not non-scalable coding has been applied.
  • the non-scalable coding flag may be stored in the codec spiritual parameters of the subsample information box in the metadata area of the content file.
  • the content file generation unit of the encoder may set a subsample for each slice and store the non-scalable coding flag in the codec-spicy parameters of the subsample information box in the metadata area of the content file.
  • the extractor of the decoder is based on the non-scalable coding flag stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice. Any slice of G-PCC content may be extracted from the file.
  • FIG. 12 shows an example of the codec Spicic Parameters syntax in this case.
  • the codec-spicy parameters store the non-scalable coding flag (non_scalable_flag) and the non-scalable coding attribute geometry slice identification information (ref_attr_geom_slice_id).
  • the non_scalable_flag value is set to 0 (false). If the attribute slice is non-scalable encoded, the value of non_scalable_flag is set to 1 (true).
  • ref_attr_geom_slice_id is set as non-scalable coded attribute geometry slice identification information. That is, in this case, the ref_attr_geom_slice_id is set with the identification information (slice_id) of the geometry slice or the geometry slice group including the maximum depth among the geometry slices or the geometry slice group referred to by the attribute slice corresponding to the information.
  • ref_attr_geom_slice_id is set as attribute geometry slice identification information. That is, in this case, ref_attr_geom_slice_id is set to the slice identification information (slice_id) of the geometry slice referenced by the attribute slice corresponding to that information.
  • the payload type (PayloadType) of the slice dependency information of the independent geometry slice and the payload of the slice dependency information of the dependent geometry slice may be different from each other.
  • FIG. 13 is a diagram showing an example of the codec Spicic Parameters syntax in that case.
  • the payload type of the slice dependency information of the independent geometry slice is set to "2" and the payload type of the slice dependency information of the dependent geometry slice is set to "9" as shown in the dotted frame 162. Set.
  • the slice composition information may include the geometry slice depth information (method 1-1-3).
  • geometry slice depth information is information about the depth information of the geometry contained in the geometry slice or geometry slice group corresponding to the information.
  • the slice configuration information may include both slice dependency information (method 1-1-2) and geometry slice depth information.
  • the geometry data unit of "geom_slice # 1" corresponds to slice # 1.
  • Slice # 1 contains geometry from depth 0 to depth 3.
  • the geometry data unit of "geom_slice # 2" corresponds to slice # 2.
  • Slice # 2 contains the geometry of Depth 4 and Depth 5.
  • the geometry data unit of "geom_slice # 6" corresponds to slice # 6.
  • Slice # 6 contains the geometry of depth 6.
  • the geometry data unit of "geom_slice # 3" corresponds to slice # 3.
  • Slice # 3 contains the geometry of Depth 4 and Depth 5.
  • the geometry data unit of "geom_slice # 7" corresponds to slice # 7.
  • Slice # 7 contains the geometry of depth 6.
  • the depth information of each slice shown in the dotted line frame 122 is the geometry slice depth information.
  • This geometry slice depth information is stored in the metadata area of the content file. By doing so, the decoder can obtain this geometry slice depth information from the metadata area of the content file. Therefore, the decoder can grasp the geometry slice depth information without parsing the bitstream (ie, more easily).
  • geometry slice can contain geometry of multiple depths.
  • the geometry slice depth information may include the minimum depth information (method 1-1-3-1).
  • the minimum depth information indicates information indicating the minimum value of the depth information in the geometry slice or the group of geometry slices corresponding to the information.
  • the decoder can obtain this minimum depth information from the metadata area of the content file. Therefore, the decoder can grasp the minimum depth contained in the geometry slice without parsing the bitstream (ie, more easily).
  • the geometry slice depth information may include the maximum depth information (method 1-1-3-2).
  • the maximum depth information indicates information indicating the maximum value of the depth information in the geometry slice or the group of geometry slices corresponding to the information. By doing so, the decoder can obtain this maximum depth information from the metadata area of the content file. Therefore, the decoder can grasp the maximum depth contained in the geometry slice without parsing the bitstream (ie, more easily).
  • the geometry slice depth information may include both the minimum depth information and the maximum depth information.
  • the decoder can grasp the range of depth contained in the geometry slice without parsing the bitstream (ie, more easily).
  • this geometry slice depth information may be stored in the codec spiritual parameters of the subsample information box in the metadata area of the content file (as shown).
  • Method 1-1-3-3 For example, even if the content file generator of the encoder sets a subsample for each slice or slice group and stores the geometry slice depth information in the codec spiritual parameters of the subsample information box in the metadata area of the content file. good.
  • the extractor of the decoder is based on the geometry slice depth information stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice or slice group. Any slice of G-PCC content may be extracted from the content file.
  • FIG. 14 shows an example of the codec Spicic Parameters syntax in this case.
  • the minimum depth information (min_depth) and the maximum depth information (max_depth) are set as shown in the dotted line frame 171.
  • the file size can be increased. It can be suppressed.
  • the decoder since the decoder only needs to confirm the codec spiritual parameters of the desired subsample information box, the depth included in the geometry slice can be confirmed more easily. That is, it is possible to suppress an increase in the load of the reproduction process.
  • a plurality of geometry slices having the same depth range are continuous, they are regarded as one subsample as a geometry slice group as described above. Then, the geometry slice depth information is generated for each subsample. That is, the geometry slice depth information of each slice of the geometry slice group is combined into one geometry slice depth information. Therefore, it is possible to suppress an increase in the sub-sample entry (sub-sample entry) as compared with the case where the geometry slice depth information is generated for each slice of the geometry slice group. Therefore, it is possible to suppress an increase in bit cost (that is, the amount of data in the bitstream).
  • the minimum depth information (min_depth) and the maximum depth information (max_depth) are set to the same value.
  • the flags of the subsample information box of the codec Spicy Parameters that stores this geometry slice depth information and the slice dependency information are stored.
  • the flags in the subsample information box of Codec Geometric Parameters may be set to different values (Method 1-1-3-3-1).
  • the slice configuration information may further include slice dependency information indicating the dependency between the first slice and the second slice, in addition to the geometry slice depth information.
  • the content file generation unit of the encoder may set the flags of the subsample information box that stores the geometry slice depth information to a value different from the flags of the subsample information box that stores the slice dependency information. Further, the flags of the subsample information box in which the geometry slice depth information is stored and the flags of the subsample information box in which the slice dependency information is stored may be set to different values.
  • the flags of the subsample information box where the slice dependency information is stored is set to "0", whereas the flags of the subsample information box where the geometry slice depth information is stored. Is set to "2".
  • these values are examples, and the values of flags are not limited to this example (arbitrary).
  • the geometry slice depth information and the slice dependency information can be stored in different subsample information boxes having flags of different values, and these can be used together.
  • the scalable decoding information may include track configuration information (method 1-2).
  • the track configuration information is information regarding the configuration of a track that stores G-PCC content in slice units in a content file. That is, the decoder can obtain the configuration information of the tracks included in the content file from the metadata area of the content file. Therefore, the decoder can grasp the composition of the track of the content file without parsing the bitstream (that is, more easily).
  • the scalable decoding information may include both slice configuration information (method 1-1) and track configuration information for each sample.
  • Track depth information As shown in the seventh row from the top of the table shown in FIG. 6, the track configuration information may include track depth information (method 1-2-1). As used herein, track depth information is information about the depth information of the geometry of all slices contained in the track corresponding to that information.
  • ISOBMFF content file
  • each data unit (slice) of the bitstream 101 is stored in each track as shown by the dotted line arrow shown in FIG. That is, the geometry and attributes of slice # 1 are stored on track 1.
  • the geometry and attributes of slice # 2 are stored on track 2.
  • the geometry and attributes of slice # 3 are stored on track 2.
  • the geometry and attributes of slice # 6 are stored on track 3.
  • the geometry and attributes of slice # 7 are stored on track 7.
  • the track depth information indicates which depth data is stored in each track.
  • Track depth information is set for each track. That is, the track depth information indicates which depth of data is stored in the track corresponding to the information.
  • a track can store data of multiple depths.
  • data of a plurality of samples can be stored in the track.
  • the slice composition of each sample does not have to be the same as each other. That is, the depth contained in each track can change from sample to sample.
  • the track depth information may include the track minimum depth information indicating the minimum value of the depth information in the track corresponding to the information (method 1-. 2-1-1). That is, the track minimum depth information indicates the minimum value of the depth information among all the samples included in the track corresponding to the information.
  • the track depth information may include the track maximum depth information indicating the maximum value of the depth information in the track corresponding to the information (method). 1-2-1-2). That is, the track maximum depth information indicates the maximum value of the depth information among all the samples included in the track corresponding to the information.
  • the track depth information may include both the track minimum depth information (method 1-2-1-1) and the track maximum depth information.
  • the track depth information may include a match flag (method 1-2-1-3).
  • this match flag is a track in which the sample minimum depth information, which is the minimum value of the depth information in each sample included in the track corresponding to the information, matches the track minimum depth information and corresponds to the information.
  • the track depth information may include all of the matching flag, the track minimum depth information (method 1-2-1-1), and the track maximum depth information (method 1-2-1-2).
  • the track depth information is stored in the depth information box (DepthInfoBox) of the sample entry (SampleEntry) corresponding to each track in the metadata area.
  • the content file generator of the encoder may store the track depth information in the depth information box of the sample entry in the metadata area.
  • the extraction unit of the decoder may extract an arbitrary slice of G-PCC content from the content file based on the track depth information stored in the depth information box of the sample entry in the metadata area.
  • FIG. 16 is a diagram showing an example of the syntax of the depth information box (DepthInfoBox). As shown in FIG. 16, in the depth information box, the track minimum depth information (track_min_depth), the track maximum depth information (track_max_depth), and the match flag (fixed_depth) are set.
  • Track_min_depth the track minimum depth information
  • track_max_depth the track maximum depth information
  • fixed_depth the match flag
  • the sample minimum depth information and sample maximum depth information of all the samples in the track are values within the range of the track minimum depth information (track_min_depth) to the track maximum depth information (track_max_depth). Indicates to take. That is, in this case, the minimum and / or maximum depth of each sample may vary from sample to sample.
  • the match flag is "1" (true)
  • the sample minimum depth information of each sample in the track matches the track minimum depth information (track_min_depth)
  • the sample maximum depth information of each sample in the track is the track maximum depth. Indicates that it matches the information (track_min_depth). That is, in this case, the minimum value and the maximum value of the depth of each sample have a common value for all the samples.
  • the value of the match flag (fixed_depth) is set to "1"
  • the track minimum depth information (track_min_depth) and the track maximum depth information (track_max_depth) are set.
  • track_min_depth track_max_depth
  • the decoder can determine which depth data is stored in each track based on the information. It can be grasped more easily (without parsing the bitstream). That is, it is possible to suppress an increase in the load of the reproduction process.
  • the desired LoD can be reliably obtained by processing this track and the referenced track. That is, these track depth information can be useful information during the track selection process of the client.
  • non-scalable coded attribute flag may be added to clearly indicate whether or not the track contains an attribute slice to which non-scalable coding is applied.
  • this non-scalable coded attribute flag (non_scalable_attribute_flag) is "1" (true)
  • this non-scalable coded attribute flag (non_scalable_attribute_flag) is "0" (false)
  • this non-scalable coded attribute flag (non_scalable_attribute_flag) is "0" (false)
  • the track configuration information may include track dependency information (method 1-2-2).
  • the track dependency information is information indicating a dependency between tracks (for example, a dependency between a first track and a second track).
  • the track configuration information may include both track depth information (method 1-2-1) and track dependency information.
  • each track (track 1 to track 3) has a dependency relationship between tracks as shown by arrows 184 to 185.
  • track 2 is subordinate to track 1 as shown by arrow 184. That is, in order to decode the bitstream of track 2 and restore the geometry or attributes of depth 4 or depth 5, it is also necessary to decode the bitstream of track 1 corresponding to depths 0 to 3.
  • track 3 is directly subordinate to track 2, as shown by arrow 185. That is, track 3 is indirectly subordinate to track 1, as shown by arrow 186.
  • the track dependency information indicates such a dependency between tracks.
  • this track dependency information indicates other tracks including the slices necessary for decoding the dependent slice contained in the track corresponding to the information.
  • Information may be included (Method 1-2-2-1).
  • the dependent information is information indicating the dependent destination, that is, the track on the end point side of the above-mentioned arrow, as the track dependency information on the start point side of the above-mentioned arrows (arrows 184 to 186).
  • this dependent information includes all other tracks including the slice required for decoding the dependent slice included in the track corresponding to the information. It may be shown (method 1-2-2-1-1).
  • the dependent information of the track 3 may include both the information corresponding to the arrow 185 and the information corresponding to the arrow 186.
  • this dependent information may indicate another track containing the slice referred to from the information (method 1-2-2). 2-1-2). That is, the dependent information may indicate only the slice to which the slice corresponding to the information is directly dependent.
  • the dependent information of the track 3 may include only the information corresponding to the arrow 185.
  • the track dependency information may be stored as a track reference in the metadata area.
  • dependent information as shown in FIG. 15 may be stored as a track reference.
  • the track reference may be used to link only the track including the slice directly referred to when decoding the dependent slice included in the track.
  • sampleEntry4CC The sample entry 4CC (SampleEntry4CC) of a track that includes an independent slice and can be decoded by the track alone may be specified as'gpc1'. Further, the sample entry 4CC (SampleEntry 4CC) of a track that does not include an independent slice and cannot be decoded by the track alone may be specified as ‘lgp1’.
  • the track dependency information includes other dependent slices that require an independent slice included in the track corresponding to the information at the time of decoding.
  • Independent information indicating the track may be included (method 1-2-2-2).
  • the independent slice of track 1 is referred to when decoding the dependent slice of track 2.
  • the independent slice of track 1 is referenced when decoding the dependent slice of track 3.
  • the independent slice stored in track 1 is used to decode the dependent slice of track 2. Also, as indicated by arrow 192, the independent slices stored in track 1 are used to decode the dependent slices of track 3.
  • Independent information is information indicating such a dependency relationship. In other words, independent information is reverse lookup information of dependent information.
  • track 3 is indirectly subordinate to track 1. Therefore, the dependency indicated by arrow 192 may or may not be included in the independent information. That is, as shown in the 17th column from the top of the table shown in FIG. 6, the independent information may indicate other tracks including other slices that refer to the independent slice at the time of decoding. Further, the track dependency information may include both dependent information and independent information.
  • the track dependency information may be stored as a track reference in the metadata area.
  • independent information as shown in FIG. 17 may be stored as a track reference.
  • Matryoshka media container In the above, an example of applying ISOBMFF as a file format has been described, but the file that stores the G-PCC bitstream is arbitrary and may be other than ISOBMFF. For example, G-PCC content may be stored in a Matroska Media Container. A main configuration example of the matryoshka media container is shown in FIG.
  • the tile management information may be stored as a newly defined element under the track entry element.
  • the tile management information (tile identification information) is stored in the timed metadata (timed metadata)
  • the timed metadata is different from the track entry (track entry) in which the G-PCC content is stored. It may be stored in the track entry (Track entry).
  • FIG. 19 is a block diagram showing an example of a configuration of a file generation device, which is an aspect of an information processing device to which the present technology is applied.
  • the file generation device 300 shown in FIG. 19 applies G-PCC to encode point cloud data, and stores the G-PCC content (G-PCC bit stream) generated by the encoding in a content file (ISOBMFF). It is a device to do.
  • the file generation device 300 is ⁇ 2. Transmission of scalable decryption information by content file> The above-mentioned technology is applied in the chapter. That is, the file generation device 300 generates scalable decoding information based on the depth of the slice in the G-PCC content and the dependency between the slices, generates a content file for storing the G-PCC content, and generates the scalable decoding. Store the information in the metadata area of the generated content file.
  • FIG. 19 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the file generation device 300, there may be a processing unit that is not shown as a block in FIG. 19, or there may be a processing or data flow that is not shown as an arrow or the like in FIG.
  • the file generation device 300 has an extraction unit 311, a coding unit 312, a bitstream generation unit 313, a scalable decoding information generation unit 314, and a file generation unit 315.
  • the coding unit 312 includes a geometry coding unit 321, an attribute coding unit 322, and a metadata generation unit 323.
  • the extraction unit 311 extracts geometry data and attribute data from the point cloud data input to the file generation device 300, respectively.
  • the extraction unit 311 supplies the data of the extracted geometry to the geometry coding unit 321 of the coding unit 312. Further, the extraction unit 311 supplies the extracted attribute data to the attribute coding unit 322 of the coding unit 312.
  • the coding unit 312 encodes the data in the point cloud.
  • the geometry coding unit 321 encodes the geometry data supplied from the extraction unit 311 and generates a geometry bit stream.
  • the geometry coding unit 321 supplies the generated geometry bitstream to the metadata generation unit 323. Further, the geometry coding unit 321 also supplies the generated geometry bitstream to the attribute coding unit 322.
  • the attribute coding unit 322 encodes the attribute data supplied from the extraction unit 311 and generates an attribute bit stream.
  • the attribute coding unit 322 supplies the generated attribute bit stream to the metadata generation unit 323.
  • the metadata generation unit 323 refers to the supplied geometry bitstream and attribute bitstream, and generates metadata.
  • the metadata generation unit 323 supplies the generated metadata to the bitstream generation unit 313 together with the geometry bitstream and the attribute bitstream.
  • the bitstream generation unit 313 multiplexes the supplied geometry bitstream, attribute bitstream, and metadata to generate G-PCC content (G-PCC bitstream).
  • the bitstream generation unit 313 supplies the generated G-PCC content to the scalable decoding information generation unit 314.
  • the scalable decoding information generation unit 314 acquires the G-PCC content including the first slice and the second slice supplied from the bitstream generation unit 313.
  • the scalable decoding information generation unit 314 has ⁇ 2. Transmission of scalable decryption information by content file> Applying the above technology in the chapter, the depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC content and the first depth information in the G-PCC content. Generates scalable decoding information about the scalable decoding of the G-PCC content based on the dependency between the slice and the second slice.
  • the scalable decoding information generation unit 314 supplies the generated scalable decoding information to the file generation unit 315 together with the G-PCC content.
  • the file generation unit 315 is ⁇ 2. Transmission of scalable decryption information by content file> Applying the above-mentioned technology, a content file for storing G-PCC content supplied from the scalable decoding information generation unit 314 is generated, and the scalable decoding information is generated as the generated content. Store in the metadata area of the file. The file generation unit 315 outputs the content file generated as described above to the outside of the file generation device 300.
  • the scalable decoding information generation unit 314 may generate scalable decoding information including slice configuration information for each sample.
  • the file generation unit 315 may set a subsample for each slice and store the slice configuration information in the codec spiritual parameters of the subsample information box in the metadata area of the content file.
  • the scalable decoding information generation unit 314 may generate slice configuration information including slice dependency information.
  • the file generation unit 315 may set a subsample for each slice and store the slice dependency information in the codec spiritual parameters of the subsample information box in the metadata area of the content file.
  • the scalable decoding information generation unit 314 may generate slice dependency information including reference source geometry slice identification information and reference destination geometry slice identification information. Then, when the slice corresponding to this slice dependency information (that is, the slice corresponding to the reference source geometry slice identification information) is an independent geometry slice, the above-mentioned reference source geometry slice identification information and the reference destination geometry slice identification information are the same. It may be set to.
  • the file generation unit 315 sets a subsample for each slice and stores the reference source geometry slice identification information and the reference destination geometry slice identification information in the codec spiritual parameters of the subsample information box of the metadata area. good.
  • the scalable decoding information generation unit 314 may generate slice dependency information including attribute geometry slice identification information.
  • the file generation unit 315 may set a subsample for each slice and store the attribute geometry slice identification information in the codec spiritual parameters of the subsample information box of the metadata area.
  • the scalable decoding information generation unit 314 may generate slice dependency information including non-scalable coded attribute geometry slice identification information which is identification information of the geometry slice referred to by the attribute slice to which the non-scalable coding is applied.
  • the scalable decoding information generation unit 314 is a non-scalable encoding containing the identification information of the geometry slice or the geometry slice group containing the geometry having the largest depth information among the geometry slices or the geometry slice group referred to by the attribute slice corresponding to the information. Attribute geometry slice identification information may be generated.
  • the file generation unit 315 may set a subsample for each slice and store the non-scalable coded attribute geometry slice identification information in the codec spiritual parameters of the subsample information box in the metadata area of the content file.
  • the scalable decoding information generation unit 314 may generate slice dependency information including a non-scalable coding flag. Further, the file generation unit 315 may set a subsample for each slice and store the non-scalable coding flag in the codec spiritual parameters of the subsample information box of the metadata area of the content file.
  • the file generation unit 315 sets the payload type of the slice dependency information of the independent geometry slice (PayloadType) and the payload type of the slice dependency information of the dependent geometry slice to different values. May be good.
  • the scalable decoding information generation unit 314 may generate slice configuration information including geometry slice depth information. At that time, the scalable decoding information generation unit 314 may generate geometry slice depth information including the minimum depth information. Further, the scalable decoding information generation unit 314 may generate geometry slice depth information including the maximum depth information. Then, the file generation unit 315 may set a subsample for each slice or slice group, and store this geometry slice depth information in the codec spiritual parameters of the subsample information box in the metadata area of the content file. ..
  • the scalable decoding information generation unit 314 may generate slice configuration information including slice dependency information indicating the dependency between the first slice and the second slice in addition to the geometry slice depth information. Then, the file generation unit 315 may set the flags of the subsample information box that stores the geometry slice depth information to a value different from the flags of the subsample information box that stores the slice dependency information.
  • the scalable decoding information generation unit 314 may generate scalable decoding information including track configuration information.
  • the scalable decoding information generation unit 314 may generate track configuration information including track depth information.
  • the scalable decoding information generation unit 314 may generate track depth information including the track minimum depth information indicating the minimum value of the depth information in the track corresponding to the information. Further, the scalable decoding information generation unit 314 may generate track depth information including the track maximum depth information indicating the maximum value of the depth information in the track corresponding to the information. Further, the scalable decoding information generation unit 314 may generate track depth information including a match flag. Then, the file generation unit 315 may store such track depth information in the depth information box of the sample entry in the metadata area.
  • the scalable decoding information generation unit 314 may generate track configuration information including track dependency information. Further, the scalable decoding information generation unit 314 may generate track dependency information including dependent information indicating other tracks including slices necessary for decoding the dependent slice included in the track corresponding to the information. Further, the scalable decoding information generation unit 314 may generate dependent information indicating all other tracks including the slice necessary for decoding the dependent slice included in the track corresponding to the information. Further, the scalable decoding information generation unit 314 may generate dependent information indicating other tracks including slices referred to from the information.
  • the scalable decoding information generation unit 314 may generate track dependency information including independent information indicating other tracks including dependent slices that require an independent slice included in the track corresponding to the information at the time of decoding.
  • the scalable decoding information generation unit 314 may generate independent information indicating other tracks including other slices that refer to the independent slice at the time of decoding. Further, the scalable decoding information generation unit 314 may generate independent information in which the track dependency information includes both dependent information and independent information.
  • the file generation unit 315 may store the track dependency information in the metadata area as a track reference.
  • the extraction unit 311 of the file generation device 300 extracts the geometry and the attribute from the point cloud in step S301, respectively.
  • step S302 the coding unit 312 encodes the geometry and the attribute extracted in step S301 to generate a geometry bit stream and an attribute bit stream.
  • the coding unit 312 further generates the metadata thereof.
  • step S303 the bitstream generation unit 313 multiplexes the geometry bitstream, attribute bitstream, and metadata generated in step S302 to generate a G-PCC bitstream (G-PCC content).
  • step S304 the scalable decoding information generation unit 314 has ⁇ 2. Transmission of scalable decryption information by content file> Applying the above technology in the chapter, based on the slice depth in the G-PCC content generated in step S303 and the dependency between the slices in the G-PCC content. Then, the scalable decoding information regarding the scalable decoding of the G-PCC content is generated.
  • step S305 the file generation unit 315 generates other information and generates a content file (for example, ISOBMFF) for storing the G-PCC content generated in step S303. Then, the file generation unit 315 has ⁇ 2. Transmission of scalable decoding information by content file> The above-mentioned technique is applied in the chapter, and the scalable decoding information generated in step S304 is stored in the metadata area of the generated content file.
  • a content file for example, ISOBMFF
  • the file generation unit 315 outputs the generated content file (content file in which the scalable decoding information is stored) to the outside of the file generation device 300.
  • the file generation unit 315 transmits the content file to another device (for example, a playback device or the like) via a network or the like.
  • the file generation unit 315 supplies the content file to an external storage medium of the file generation device 300 and stores the content file. In this case, the content file is supplied to the playback device or the like via the storage medium.
  • step S306 When the process of step S306 is completed, the file generation process is completed.
  • the file generation device 300 has ⁇ 2. Transmission of scalable decryption information by content file> The technology described in the chapter is applied, and the scalable decryption information is stored in the metadata area of the content file. By doing so, it is possible to reduce the processing (decoding and the like) of unnecessary information, and it is possible to suppress an increase in the load of the reproduction processing.
  • FIG. 21 is a block diagram showing an example of a configuration of a reproduction device, which is an aspect of an information processing device to which the present technology is applied.
  • the playback device 400 shown in FIG. 21 is a device that decodes a G-PCC file, constructs a point cloud, renders it, and generates presentation information.
  • the reproduction device 400 is ⁇ 2.
  • Transmission of scalable decoding information by content file> Applying this technology described above in the chapter, the slices required for playback of the desired depth in the point cloud are extracted from the content file generated by the file generator 300. Decrypt and play.
  • FIG. 21 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. 21. That is, in the reproduction device 400, there may be a processing unit that is not shown as a block in FIG. 21, or there may be a processing or data flow that is not shown as an arrow or the like in FIG. 21.
  • the reproduction device 400 has a control unit 401, a file acquisition unit 411, a reproduction processing unit 412, and a presentation processing unit 413.
  • the reproduction processing unit 412 includes a file processing unit 421, a decoding unit 422, and a presentation information generation unit 423.
  • the control unit 401 controls each processing unit in the reproduction device 400.
  • the file acquisition unit 411 acquires a content file for storing the point cloud to be reproduced and supplies the content file to the reproduction processing unit 412 (file processing unit 421).
  • the reproduction processing unit 412 performs processing related to reproduction of the point cloud stored in the supplied content file.
  • the file processing unit 421 of the reproduction processing unit 412 acquires the content file supplied from the file acquisition unit 411 and extracts the bit stream from the content file. At that time, the file processing unit 421 is described in ⁇ 2. Transmission of scalable decoding information by content file> The above-mentioned technique is applied in the chapter, and only the bitstream of slices necessary for reproducing the desired depth is extracted. The file processing unit 421 supplies the extracted bit stream to the decoding unit 422.
  • the decoding unit 422 decodes the bitstream supplied from the file processing unit 421 and generates geometry and attribute data.
  • the decoding unit 422 supplies the generated geometry and attribute data to the presentation information generation unit 423.
  • the presentation information generation unit 423 constructs a point cloud using the supplied geometry and attribute data, and generates presentation information which is information for presenting (for example, displaying) the point cloud.
  • the presentation information generation unit 423 renders using the point cloud, and generates a display image of the point cloud as the presentation information as viewed from a predetermined viewpoint.
  • the presentation information generation unit 423 supplies the presentation information thus generated to the presentation processing unit 413.
  • the presentation processing unit 413 performs a process of presenting the supplied presentation information.
  • the presentation processing unit 413 supplies the presentation information to a display device or the like outside the reproduction device 400 and causes the presentation information to be presented.
  • FIG. 22 is a block diagram showing a main configuration example of the reproduction processing unit 412.
  • the file processing unit 421 has a bitstream extraction unit 431.
  • the decoding unit 422 has a geometry decoding unit 441 and an attribute decoding unit 442.
  • the presentation information generation unit 423 has a point cloud construction unit 451 and a presentation processing unit 452.
  • the bitstream extraction unit 431 extracts a bitstream from the content file supplied from the file acquisition unit 411. At that time, the bitstream extraction unit 431 has ⁇ 2. Transmission of scalable decoding information by content file> The above-mentioned technique is applied in the chapter, and only the bitstream of slices necessary for reproducing the desired depth is extracted. That is, the bitstream extraction unit 431 extracts an arbitrary slice of the G-PCC content from the content file based on the scalable decoding information stored in the metadata area of the content file that stores the G-PCC content.
  • this scalable decoding information is information related to scalable decoding of G-PCC content, and is depth information indicating the quality hierarchy level of the geometry included in the slice in the G-PCC content and between slices in the G-PCC content (for example,). , Between the first slice and the second slice).
  • this scalable decoding information may include slice configuration information for each sample.
  • the bitstream extraction unit 431 grasps the composition of slices in each sample based on the slice configuration information for each sample, and extracts an arbitrary slice of G-PCC content from the content file based on the configuration. May be good. From the content file, the bit stream extraction unit 431 is based on the slice configuration information stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice. Any slice of G-PCC content may be extracted.
  • This slice configuration information may include slice dependency information.
  • the bitstream extraction unit 431 grasps the dependency between slices or slices based on the slice dependency information, and extracts any slice of G-PCC content from the content file based on the dependency. You may.
  • the bit stream extraction unit 431 is based on the slice dependency information stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice, from the content file. , Any slice of G-PCC content may be extracted.
  • This slice dependency information may include the reference source geometry slice identification information and the reference destination geometry slice identification information.
  • the bitstream extraction unit 431 grasps the dependency between slices or slice groups based on the reference source geometry slice identification information and the reference destination geometry slice identification information, and G-from the content file based on the dependency. Other geometry slices needed to decode the desired geometry slice of PCC content may be extracted.
  • the slice corresponding to this slice dependency information that is, the slice corresponding to the reference source geometry slice identification information
  • the above-mentioned reference source geometry slice identification information and the reference destination geometry slice identification information are the same. You may do so.
  • the bit stream extraction unit 431 is based on the reference source geometry slice identification information and the reference destination geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice. Then, any slice of G-PCC content may be extracted from the content file.
  • the slice dependency information may include the attribute geometry slice identification information.
  • the bitstream extraction unit 431 identifies the correspondence between the attribute slice and the geometry slice based on the attribute geometry slice identification information, and any slice of G-PCC content from the content file based on the identified correspondence. You may extract (attribute slices and geometry slices). Based on the attribute geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice, the bitstream extraction unit 431 G from the content file. -You may extract any slices of PCC content.
  • the slice dependency information may include non-scalable coded attribute geometry slice identification information which is the identification information of the geometry slice referenced by the attribute slice to which the non-scalable coding is applied.
  • the bitstream extraction unit 431 identifies the correspondence between the attribute slice to which the non-scalable coding is applied and the geometry slice based on the non-scalable coded attribute geometry slice identification information, and the content is based on the correspondence.
  • Arbitrary slices of G-PCC content (attribute slices and geometry slices with non-scalable encoding) may be extracted from the file.
  • This non-scalable coded attribute geometry slice identification information may include the identification information of the geometry slice or the geometry slice group including the maximum depth among the geometry slices or the geometry slice group referred to by the attribute slice corresponding to the information. ..
  • the bit stream extractor 431 is based on the non-scalable coded attribute geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the subsample of the subsample set for each slice. , You may extract any slice of G-PCC content from the content file.
  • the slice dependency information may include a non-scalable coding flag.
  • the bitstream extraction unit 431 identifies whether or not non-scalable coding has been applied based on the non-scalable coding flag, and identifies the correspondence between the attribute slice and the geometry slice based on the discrimination result.
  • Arbitrary slices of G-PCC content (attribute slices and geometry slices) may be extracted from the content file based on the correspondence.
  • the bit stream extraction unit 431 is based on the non-scalable coding flag stored in the codec spiritual parameters of the subsample information box of the subsample information box of the subsample set for each slice, and the content file. Any slice of G-PCC content may be extracted from.
  • the payload type of the slice dependency information of the independent geometry slice (PayloadType) and the payload type of the slice dependency information of the dependent geometry slice may have different values.
  • the bitstream extraction unit 431 identifies whether it is slice dependency information of an independent geometry slice or slice dependency information of a dependent geometry slice based on this payload type, and slices based on the discrimination result. Dependency information may be analyzed.
  • the slice configuration information may include geometry slice depth information.
  • the bitstream extraction unit 431 grasps the depth of the geometry contained in the geometry slice or the geometry slice group based on the geometry slice depth information, and based on the depth information, any of the G-PCC contents from the content file. Slices may be extracted.
  • the geometry slice depth information may include the minimum depth information.
  • the geometry slice depth information may include the maximum depth information.
  • the bit stream extraction unit 431 based on the geometry slice depth information stored in the codec spiritual parameters of the subsample information box of the metadata area of the content file of the subsample set for each slice or slice group. Any slice of G-PCC content may be extracted from the content file.
  • the slice configuration information may further include slice dependency information indicating the dependency between slices in addition to the geometry slice depth information. Then, the flags of the subsample information box in which the geometry slice depth information is stored and the flags of the subsample information box in which the slice dependency information is stored may be set to different values. For example, the bitstream extraction unit 431 identifies whether it is a subsample information box in which geometry slice depth information is stored or a subsample information box in which slice dependency information is stored, based on the flags. May be good.
  • the scalable decoding information may include track configuration information.
  • the bitstream extraction unit 431 may grasp the configuration of the track based on the track configuration information and extract an arbitrary slice of G-PCC content from the content file based on the track configuration.
  • the track configuration information may include track depth information.
  • the bitstream extraction unit 431 grasps the depth information of the geometry of all slices contained in the track based on the track depth information, identifies the track in which the desired slice is stored based on the depth information, and identifies the track in which the desired slice is stored.
  • the desired slice may be extracted from the track.
  • the track depth information may include track minimum depth information indicating the minimum value of the depth information in the track corresponding to the information.
  • the track depth information may include track maximum depth information indicating the maximum value of the depth information in the track corresponding to the information.
  • the track depth information may include a match flag.
  • the bitstream extraction unit 431 grasps the depth information of the geometry contained in the track based on this information, identifies the track in which the desired slice is stored based on the depth information, and obtains the desired slice from the track. Slices may be extracted.
  • the bitstream extraction unit 431 may extract an arbitrary slice of G-PCC content from the content file based on the track depth information stored in the depth information box of the sample entry in the metadata area.
  • the track configuration information may include track dependency information.
  • the bitstream extraction unit 431 may grasp the dependency of the track based on the track dependency information and extract an arbitrary slice of the G-PCC content from the content file based on the dependency.
  • This track dependency information may include dependent information indicating other tracks containing the slices required to decode the dependent slices contained in the track corresponding to that information.
  • the bitstream extraction unit 431 may identify another track including the slice necessary for decoding the dependent slice contained in the track corresponding to the information based on the dependent information.
  • This dependent information may indicate all other tracks containing the slices needed to decode the dependent slices contained in the track corresponding to that information.
  • the dependent information may also indicate other tracks containing slices referenced from that information.
  • the track dependency information may include independent information indicating other tracks including dependent slices that require an independent slice contained in the track corresponding to the information at the time of decoding.
  • the bitstream extraction unit 431 may identify another track including a dependent slice that requires an independent slice included in the track corresponding to the information at the time of decoding based on the independent information.
  • This independent information may indicate other tracks containing other slices that reference the independent slice during decoding.
  • the track dependency information may be stored as a track reference in the metadata area.
  • the bitstream extraction unit 431 supplies the extracted geometry bitstream to the geometry decoding unit 441. Further, the bitstream extraction unit 431 supplies the extracted attribute bitstream to the attribute decoding unit 442.
  • the geometry decoding unit 441 decodes the supplied geometry bitstream and generates geometry data.
  • the geometry decoding unit 441 supplies the generated geometry data to the point cloud construction unit 451.
  • the attribute decoding unit 442 decodes the supplied attribute bit stream and generates attribute data.
  • the attribute decoding unit 442 supplies the generated attribute data to the point cloud construction unit 451.
  • the point cloud construction unit 451 constructs a point cloud using the supplied geometry and attribute data. That is, the point cloud construction unit 451 can construct a point cloud of a desired depth. The point cloud construction unit 451 supplies the constructed point cloud data to the presentation processing unit 452.
  • the presentation processing unit 452 generates presentation information using the supplied point cloud data.
  • the presentation processing unit 452 supplies the generated presentation information to the presentation processing unit 413.
  • the playback device 400 can more easily obtain desired tiles based on the tile management information (tile identification information) stored in the G-PCC file without having to parse the entire bitstream. Only can be extracted, decrypted, constructed and presented. Therefore, it is possible to suppress an increase in the load of the reproduction process.
  • the file acquisition unit 411 of the reproduction apparatus 400 acquires the content file to be reproduced in step S401.
  • step S402 the bitstream extraction unit 431 extracts an arbitrary slice from the content file acquired in step S401. At that time, the bitstream extraction unit 431 has ⁇ 2.
  • the above-mentioned technique is applied in Transmission of scalable decoding information by content file>, and slices are extracted based on the scalable decoding information stored in the metadata area of the content file.
  • step S403 the geometry decoding unit 441 of the decoding unit 422 decodes the geometry bitstream of the slice extracted in step S402 and generates the geometry of a desired depth. Further, the attribute decoding unit 442 decodes the attribute bit stream of the slice extracted in step S402 to generate an attribute corresponding to the geometry of a desired depth.
  • step S404 the point cloud construction unit 451 constructs a point cloud using the geometry and attributes generated in step S403. That is, the point cloud construction unit 451 can construct a point cloud of a desired depth.
  • step S405 the presentation processing unit 452 generates presentation information by rendering using the point cloud constructed in step S404.
  • step S406 the presentation processing unit 413 supplies the presentation information to the outside of the reproduction device 400 and causes the presentation information to be presented.
  • step S406 When the process of step S406 is completed, the reproduction process is completed.
  • the reproduction device 400 has ⁇ 2. Transmission of scalable decryption information by content file> The technique described in the chapter is applied, and a desired slice is extracted from the content file and decoded based on the scalable decoding information stored in the metadata area of the content file. By doing so, it is possible to reduce the processing (decoding and the like) of unnecessary information, and it is possible to suppress an increase in the load of the reproduction processing.
  • This technology can also be applied to, for example, MPEG-DASH (Moving Picture Experts Group phase-Dynamic Adaptive Streaming over HTTP).
  • MPEG-DASH Motion Picture Experts Group phase-Dynamic Adaptive Streaming over HTTP
  • MPD Media Presentation Description
  • the adaptation set configuration information which is the information regarding the configuration of the adaptation set that describes the information regarding the track of the content file, may be stored in the MPD.
  • the adaptation set configuration information based on the depth of each slice of the G-PCC content having a slice structure and the dependency between the slices stored in the control file. (Method 2).
  • depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC content including the first slice and the second slice, and the first slice in the G-PCC content.
  • the adaptation set configuration information generator that generates the adaptation set configuration information and the control file that controls the playback of the content file that stores the G-PCC content are generated and adapted. It is provided with a control file generator that stores set configuration information in the control file. Then, the content file stores the G-PCC content in the track in slice units. Further, the adaptation set configuration information is information regarding the configuration of the adaptation set that describes the information regarding the track of the content file.
  • the depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC content including the first slice and the second slice, and the first slice in the G-PCC content.
  • it Based on the dependencies between the second slices, it generates adaptation set configuration information, generates a control file that controls the playback of the content file that stores the G-PCC content, and controls the adaptation set configuration information. Try to store it in a file. Then, the content file stores the G-PCC content in the track in slice units.
  • the adaptation set configuration information is information regarding the configuration of the adaptation set that describes the information regarding the track of the content file.
  • a control file that controls playback of a content file that stores G-PCC content including a first slice and a second slice in a track in slice units is analyzed and stored in the control file.
  • the analysis unit that identifies the adaptation set required to obtain an arbitrary slice of G-PCC content, and the track corresponding to the adaptation set specified by the analysis unit of the content file. It is provided with an acquisition unit to be acquired and a decoding unit for decoding a slice of G-PCC content stored in the track acquired by the acquisition unit.
  • the adaptation set configuration information is information about the configuration of the adaptation set that describes the information about the track of the content file, and the depth information and the G-PCC content indicating the quality hierarchy level of the geometry contained in the slice in the G-PCC content.
  • the information is generated based on the dependency between the first slice and the second slice in.
  • the control file that controls the playback of the content file that stores the G-PCC content in the track in slice units is analyzed, and the G- is based on the adaptation set configuration information stored in the control file. Identify the adaptation set needed to get any slice of PCC content, get the track corresponding to the identified adaptation set in that content file, and store the G-PCC content in that retrieved track. Try to decrypt the slice.
  • the adaptation set configuration information is information about the configuration of the adaptation set that describes the information about the track of the content file, and is the depth information indicating the quality hierarchy level of the geometry contained in the slice in the G-PCC content and the G-PCC content. Information generated based on the dependencies between the first slice and the second slice.
  • the decoder selects the track containing the slices needed to play the point cloud of the desired depth or region, based on the adaptation set configuration information stored in the control file. You can get that track. Therefore, it is possible to suppress the transmission of unnecessary data. As a result, it is possible to suppress an increase in the load on the transmission line and communication processing. Further, since the decoder can suppress an increase in the amount of data to be processed, it is possible to suppress an increase in the load of the reproduction process. As a result, it is possible to suppress an increase in delay related to data transmission and reproduction processing.
  • the adaptation set configuration information may include the adaptation set depth information (method 2-1).
  • the adaptation set depth information is information about the geometry depth information of all slices contained in the track corresponding to the adaptation set corresponding to the information. This adaptation set depth information is described in ⁇ 2-2-1. This is the same information as the track depth information explained in Track Depth Information>. That is, the adaptation set depth information is the track depth information applied to the control file (MPD).
  • MPD control file
  • the decoder can easily figure out which adaptation slice corresponds to which adaptation set (ie, track) (without parsing the bitstream). .. Therefore, the decoder can more easily select the track to acquire (without parsing the bitstream).
  • the adaptation set depth information may include the adaptation set minimum depth information (method 2-1-1).
  • the adaptation set minimum depth information is information indicating the minimum value of the depth information in the track corresponding to the adaptation set corresponding to the information.
  • FIG. 25 is a diagram showing an example of parameters to be added to the MPD as adaptation set depth information.
  • @MinDepth shown in FIG. 25 is the minimum depth information of the adaptation set, and is ⁇ 2-2-1. This is the same information as track_min_depth (track minimum depth information) of the track depth information explained in Track Depth Information>. In other words, @minDepth applies track_min_depth to the control file (MPD).
  • MPD control file
  • the adaptation set depth information may include the adaptation set maximum depth information (method 2-1-2).
  • the adaptation set maximum depth information is information indicating the maximum value of the depth information in the track corresponding to the adaptation set corresponding to the information.
  • @MaxDepth shown in FIG. 25 is the adaptation set maximum depth information, and is ⁇ 2-2-1. This is the same information as track_max_depth (track maximum depth information) of the track depth information explained in Track Depth Information>. That is, @maxDepth applies track_max_depth to the control file (MPD).
  • MPD control file
  • the adaptation set depth information may include a match flag (method 2-1-3).
  • the match flag the sample minimum depth information of each sample contained in the track corresponding to the adaptation set corresponding to the information matches the adaptation set minimum depth information, and the sample maximum depth information of each sample contained in the track is set.
  • Flag information indicating whether or not the adaptation set maximum depth information is matched (flag information indicating whether or not the minimum depth and the maximum depth are common to all samples).
  • the sample minimum depth information indicates the minimum value of the depth information in the sample.
  • the sample maximum depth information indicates the maximum value of the depth information in the sample.
  • @FixedDepth shown in FIG. 25 is a match flag of the adaptation set.
  • @fixedDepth is "0" (false)
  • the sample minimum depth information and sample maximum depth information of all the samples in the track corresponding to the adaptation set are the adaptation set minimum depth information (@minDepth) to the adaptation set maximum depth information.
  • @minDepth the adaptation set minimum depth information
  • the sample minimum depth information of each sample in the track matches the track minimum depth information (track_min_depth), and the sample maximum depth of each sample in the track. Indicates that the information matches the track maximum depth information (track_min_depth). That is, in this case, the minimum value and the maximum value of the depth of each sample have a common value for all the samples.
  • these parameters (@fixedDepth, @minDepth, @maxDepth) of the adaptation set depth information may be set in the above-mentioned Supplemental Property or Essential Property.
  • the decoder can determine which depth data is stored in each track based on this information. It can be grasped more easily (without parsing the bitstream). Therefore, the decoder can more easily select the track to acquire (without parsing the bitstream).
  • the adaptation set configuration information may include the representation dependency information (method 2-2).
  • the representation dependency information is information indicating the dependency between representations (for example, the dependency between the first representation and the second representation).
  • MPD tracks are managed by the representation of the adaptation set.
  • this representation dependency information is ⁇ 2-2-2. This is the same information as the track dependency information explained in Track Dependency Information>. That is, the representation dependency information is the track dependency information applied to the control file (MPD).
  • the decoder can grasp the dependency between tracks more easily (without parsing the bitstream) based on this representation dependency information stored in the MPD. Therefore, the decoder can more easily select the track to acquire (without parsing the bitstream).
  • the decoder bases the dependent information on other representations (ie, tracks) needed to decode the representation (ie, track) being processed. , Can be checked more easily without parsing the bitstream. Therefore, it is possible to suppress an increase in the load of the reproduction process.
  • this dependent information may indicate all other representations necessary for decoding the representation corresponding to that information.
  • this dependent information may indicate another representation referenced from the representation corresponding to that information (method 2). -2-1-2). That is, this dependent information may indicate a representation that the referencing representation directly references.
  • this independent information may indicate all other representations that require a representation corresponding to that information upon decoding. (Method 2-2-1).
  • the representation dependency information is a control file (MPD) as two parameters of the representation association identification information (Representation @ associationId) and the association type (associationType). ) May be stored (method 2-2-3). That is, the association of the dependent information and the independent information may be performed using two parameters of the representation association identification information (Representation @ associationId) and the association type (associationType).
  • MPD representation association identification information
  • associationType associationType
  • associationType “depd”
  • associationType “indd”
  • control file may store flag information (@nonScalableAttributeFlag) that clearly indicates whether the adaptation set contains attribute slices to which non-scalable encoding is applied.
  • the adaptation set depth information and the representation dependency information are stored in the MPD, and the decoder parses the representation (that is, the track) including the desired slice and the bitstream based on the information. It can be confirmed more easily without doing. Therefore, it is possible to suppress an increase in the load of the reproduction process.
  • FIG. 27 is a block diagram showing an example of the configuration of a file generation device, which is an aspect of an information processing device to which the present technology is applied. Similar to the file generation device 300, the file generation device 600 shown in FIG. 27 applies G-PCC to encode the point cloud data, and the G-PCC content (G-PCC bit stream) generated by the encoding is used. It is a device that stores in a content file (ISOBMFF). However, the file generation device 600 further generates an MPD corresponding to the content file.
  • G-PCC G-PCC bit stream
  • the file generator 600 is ⁇ 2. Transmission of scalable decryption information by content file> and ⁇ 4. Transmission of scalable decoding information by control file>
  • the above-mentioned technique can be applied in the chapter. That is, the file generation device 600 generates scalable decoding information based on the depth of the slice in the G-PCC content and the dependency between the slices, generates a content file for storing the G-PCC content, and generates the generated content file.
  • the generated scalable decoding information at least slice configuration information for each sample can be stored in the metadata area of. Further, the file generation device 600 can store the adaptation set configuration information in the MPD among the generated scalable decoding information.
  • FIG. 27 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. 27. That is, in the file generation device 600, there may be a processing unit that is not shown as a block in FIG. 27, or there may be a processing or data flow that is not shown as an arrow or the like in FIG. 27.
  • the file generation device 600 has basically the same configuration as the file generation device 300 (FIG. 19). However, the file generation device 600 has a content file generation unit 615 and an MPD generation unit 616 instead of the file generation unit 315.
  • the scalable decoding information generation unit 314 also has ⁇ 2. Transmission of scalable decoding information by content file> The above-mentioned technology is applied to generate scalable decoding information (at least slice configuration information for each sample). That is, the scalable decoding information generation unit 314 has depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC content including the first slice and the second slice, and the first depth information in the G-PCC content. Generate scalable decoding information (at least slice configuration information for each sample) based on the dependency between the slice and the second slice. Further, in this case, the scalable decoding information generation unit 314 has ⁇ 4.
  • adaptation set configuration information is generated as scalable decoding information instead of track configuration information. That is, the scalable decoding information generation unit 314, as the adaptation set configuration information generation unit, provides depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC content including the first slice and the second slice. Adaptation set configuration information is generated based on the dependency between the first slice and the second slice in the G-PCC content.
  • the scalable decoding information generation unit 314 supplies the generated scalable decoding information together with the G-PCC content to the content file generation unit 615. Further, the scalable decoding information generation unit 314 supplies the generated scalable decoding information to the MPD generation unit 616 together with the G-PCC content.
  • the content file generation unit 615 is ⁇ 2. Transmission of scalable decoding information by content file> Applying the above-mentioned technology, a content file is generated in which the supplied G-PCC content is stored in a track in slice units, and the scalable decoding information (at least slice configuration for each sample) is generated. Information) is stored in the metadata area of the generated content file. The content file generation unit 615 outputs the content file generated as described above to the outside of the file generation device 300.
  • the MPD generation unit 616 generates an MPD as a control file generation unit, and stores information about the supplied G-PCC content in the MPD. Further, the MPD generation unit 616 is described in ⁇ 4. Transmission of scalable decoding information by control file> The above-mentioned technology is applied in the chapter, and the supplied scalable decoding information (adaptation set configuration information) is stored in the MPD.
  • the MPD generation unit 616 outputs the MPD generated as described above to the outside of the file generation device 300 (for example, a content file distribution server or the like).
  • the adaptation set configuration information may include the adaptation set depth information. That is, the scalable decoding information generation unit 314 may generate the adaptation set configuration information including the adaptation set depth information, and the MPD generation unit 616 may store the adaptation set configuration information in the adaptation set of the MPD. Further, the adaptation set depth information may include the adaptation set minimum depth information. Further, the adaptation set depth information may include the adaptation set maximum depth information. In addition, the adaptation set depth information may include a match flag. That is, the scalable decoding information generation unit 314 may generate the adaptation set depth information including these information, and the MPD generation unit 616 may store the adaptation set depth information in the adaptation set of the MPD. For example, the MPD generation unit 616 may newly define a supplemental property or an essential property and store the adaptation set depth information therein.
  • the adaptation set configuration information may include the representation dependency information. That is, the scalable decoding information generation unit 314 may generate the adaptation set configuration information including the representation dependency information, and the MPD generation unit 616 may store the adaptation set configuration information in the adaptation set of the MPD.
  • the representation dependency information may include dependent information indicating other representations necessary for decoding the representation corresponding to the information. In addition, this dependent information may indicate all other representations needed to decode the representation corresponding to that information. Also, this dependent information may indicate another representation referenced by the representation corresponding to that information. That is, the scalable decoding information generation unit 314 may generate the representation dependency information including such dependent information, and the MPD generation unit 616 may store the representation dependency information in the adaptation set of the MPD.
  • the representation dependency information may include independent information indicating other representations that require a representation corresponding to the information at the time of decryption. Then, this independent information may indicate all other representations that require a representation corresponding to that information at the time of decryption. That is, the scalable decoding information generation unit 314 may generate the representation dependency information including such independent information, and the MPD generation unit 616 may store the representation dependency information in the adaptation set of the MPD.
  • the representation dependency information may be stored in the control file (MPD) as two parameters, the representation association identification information (Representation @ associationId) and the association type (associationType).
  • each process of steps S601 to S603 is executed in the same manner as each process of steps S301 to S303 in the flowchart of the file generation process of FIG.
  • step S604 the scalable decoding information generation unit 314 has ⁇ 2. Transmission of scalable decoding information by content file> The technology described in the chapter is applied to generate slice configuration information for each sample as scalable decoding information. Further, the scalable decoding information generation unit 314 is described in ⁇ 4. Transmission of scalable decoding information by control file> The technology described in the chapter is applied to generate adaptation set configuration information as scalable decoding information.
  • step S605 the content file generation unit 615 ⁇ 2. Transmission of scalable decryption information by content file> The technology described in the chapter is applied. That is, the content file generation unit 615 generates a content file and stores the G-PCC content in the track of the content file in slice units. Then, the content file generation unit 615 stores slice configuration information for each sample in the metadata area of the content file.
  • the content file generation unit 615 outputs the generated content file (content file in which scalable decoding information is stored) to the outside of the file generation device 600.
  • the content file generation unit 615 transmits the content file to another device (for example, a playback device or the like) via a network or the like.
  • the content file generation unit 615 supplies the content file to a storage medium external to the file generation device 600 and stores the content file. In this case, the content file is supplied to the playback device or the like via the storage medium.
  • step S607 the MPD generation unit 616 is ⁇ 4. Transmission of scalable decoding information by control file> The technology described in the chapter is applied. That is, the MPD generation unit 616 generates an MPD corresponding to the content file generated in step S605, and stores the adaptation set configuration information generated in step S604 in the MPD.
  • step S608 the MPD generation unit 616 outputs the MPD to the outside of the file generation device 600.
  • the MPD is provided to a content file distribution server or the like.
  • step S608 When the process of step S608 is completed, the file generation process is completed.
  • the file generation device 600 has ⁇ 2. Transmission of scalable decryption information by content file> and ⁇ 4. Transmission of scalable decryption information by control file> This technology explained in the chapter is applied, and the scalable decryption information is stored in the metadata area of the content file or MPD. By doing so, it is possible to reduce the transmission and processing (decoding, etc.) of unnecessary information, and it is possible to suppress an increase in the load of data transmission and reproduction processing.
  • FIG. 29 is a block diagram showing an example of a configuration of a reproduction device, which is an aspect of an information processing device to which the present technology is applied. Similar to the playback device 400, the playback device 700 shown in FIG. 29 is a device that decodes a content file, constructs a point cloud, renders the content file, and generates presentation information. At that time, the reproduction device 700 is ⁇ 2. Transmission of scalable decryption information by content file> and ⁇ 4. Transmission of scalable decoding information by control file> The above-mentioned technique can be applied in the chapter.
  • FIG. 29 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. 29. That is, in the reproduction device 700, there may be a processing unit that is not shown as a block in FIG. 29, or there may be a processing or data flow that is not shown as an arrow or the like in FIG. 29.
  • the reproduction device 700 basically has the same configuration as the reproduction device 400 (FIG. 21). However, the reproduction device 700 has a file acquisition unit 711 and an MPD analysis unit 712 instead of the file acquisition unit 411.
  • the file acquisition unit 711 acquires the MPD corresponding to the desired content file (content file to be reproduced) and supplies it to the MPD analysis unit 712. Further, the file acquisition unit 711 requests and acquires the track requested by the MPD analysis unit 712 among the tracks of the content file from the supply source of the content file to be reproduced. The file acquisition unit 711 supplies the acquired track (bitstream stored in the track) to the reproduction processing unit 412 (file processing unit 421).
  • the MPD analysis unit 712 When the MPD analysis unit 712 acquires the MPD from the file acquisition unit 711, the MPD analysis unit 712 analyzes the MPD and selects a desired track. At that time, the MPD analysis unit 712 is described in ⁇ 4. Transmission of scalable decoding information by control file> The above-mentioned technique can be applied in the chapter. That is, the MPD analysis unit 712 identifies the adaptation set (that is, the track) necessary for obtaining an arbitrary slice of the G-PCC content based on the adaptation set configuration information stored in the adaptation set of the MPD. The MPD analysis unit 712 requests the file acquisition unit 711 to acquire the track corresponding to the specified adaptation set.
  • the adaptation set configuration information may include the adaptation set depth information.
  • the MPD analysis unit 712 grasps the depth information of the geometry of all the slices included in the track based on the adaptation set depth information stored in the adaptation set of the MPD, and obtains an arbitrary slice based on the depth information.
  • the required adaptation set ie, track
  • the adaptation set depth information may include the adaptation set minimum depth information.
  • the adaptation set depth information may include the adaptation set maximum depth information.
  • the adaptation set depth information may include a match flag.
  • the MPD analysis unit 712 needs to grasp the depth information of the geometry of all the slices included in the track based on these information stored in the adaptation set of the MPD, and obtain an arbitrary slice based on the depth information.
  • Adaptation sets ie, tracks
  • the adaptation set configuration information may include the representation dependency information.
  • the MPD analysis unit 712 grasps the dependency between representations (that is, tracks) based on this representation dependency information stored in the adaptation set of MPD, and obtains an arbitrary slice based on the dependency. You may identify the representation (ie, track) required for this.
  • the representation dependency information may include dependent information indicating other representations required to decrypt the corresponding representation.
  • this dependent information may indicate all other representations needed to decode the representation corresponding to that information.
  • this dependent information may indicate another representation referenced by the representation corresponding to that information. For example, even if the MPD analysis unit 712 identifies other representations (that is, other tracks) necessary for decoding the representation corresponding to that information based on this dependent information stored in the MPD's adaptation set. good.
  • the representation dependency information may include independent information indicating other representations that require a representation corresponding to the information at the time of decryption. Then, this independent information may indicate all other representations that require a representation corresponding to that information at the time of decryption.
  • the MPD analysis unit 712 identifies other representations (ie, other tracks) that require a corresponding representation during decoding, based on this dependent information stored in the MPD's adaptation set. You may.
  • the representation dependency information may be stored in the control file (MPD) as two parameters, the representation association identification information (Representation @ associationId) and the association type (associationType). That is, the MPD analysis unit 712 refers to these parameters stored in the adaptation set of the MPD, grasps the dependency between representations (that is, tracks) based on these parameters, and arbitrarily based on the dependency. You may identify the representation (ie, track) needed to obtain a slice of.
  • MPD control file
  • associationType associationType
  • the decoding unit 422 is ⁇ 4. Transmission of scalable decoding information by control file> The above-mentioned technology is applied in the chapter to decode slices of G-PCC content stored in the track supplied from the file acquisition unit 711.
  • the file acquisition unit 711 of the reproduction apparatus 700 acquires the MPD corresponding to the content file to be reproduced in step S701.
  • step S702 the MPD analysis unit 712 identifies the adaptation set required to obtain the G-PCC content of the desired depth based on the adaptation set configuration information stored in the MPD.
  • step S703 the file acquisition unit 711 acquires the coded data stored in the track corresponding to the adaptation set specified in step S702 of the content file to be reproduced.
  • step S704 the file processing unit 421 obtains sliced coded data necessary for obtaining G-PCC content of a desired depth from the acquired coded data based on the slice configuration information for each sample. Get from.
  • Each process of steps S705 to S708 is executed in the same manner as each process of steps S403 to S406 of the reproduction process of FIG. When the process of step S708 is completed, the reproduction process is completed.
  • the reproduction device 700 has ⁇ 2. Transmission of scalable decryption information by content file> and ⁇ 4. Transmission of scalable decoding information by control file> Applying this technology described in the chapter, based on the adaptation set configuration information stored in the MPD and the slice configuration information for each sample stored in the metadata area of the content file. Get the desired track of the content file and decrypt it. By doing so, it is possible to reduce the processing of unnecessary information (data transmission, decoding, etc.), and it is possible to suppress an increase in the load of the reproduction processing.
  • the series of processes described above can be executed by hardware or software.
  • the programs constituting the software are installed in the computer.
  • the computer includes a computer embedded in dedicated hardware and, for example, a general-purpose personal computer capable of executing various functions by installing various programs.
  • FIG. 31 is a block diagram showing a configuration example of computer hardware that executes the above-mentioned series of processes programmatically.
  • the CPU Central Processing Unit
  • ROM ReadOnly Memory
  • RAM RandomAccessMemory
  • the input / output interface 910 is also connected to the bus 904.
  • An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input / output interface 910.
  • the input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like.
  • the output unit 912 includes, for example, a display, a speaker, an output terminal, and the like.
  • the storage unit 913 is composed of, for example, a hard disk, a RAM disk, a non-volatile memory, or the like.
  • the communication unit 914 is composed of, for example, a network interface.
  • the drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • the CPU 901 loads the program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executes the above-mentioned series. Is processed.
  • the RAM 903 also appropriately stores data and the like necessary for the CPU 901 to execute various processes.
  • the program executed by the computer can be recorded and applied to the removable media 921 as a package media or the like, for example.
  • the program can be installed in the storage unit 913 via the input / output interface 910 by mounting the removable media 921 in the drive 915.
  • the program can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting.
  • the program can be received by the communication unit 914 and installed in the storage unit 913.
  • this program can also be installed in advance in ROM 902 or storage unit 913.
  • ISOBMFF has been described as an example of a file format for storing G-PCC contents
  • any file format can be applied as long as scalable decoding information can be stored. Further, as long as it does not conflict with the present technology, some of the above-mentioned processes and specifications may be omitted or combined with the above-mentioned technology.
  • this technology can be applied to any configuration.
  • this technique can be applied to various electronic devices.
  • this technology includes a processor as a system LSI (Large Scale Integration), a module using a plurality of processors, a unit using a plurality of modules, or a set in which other functions are added to the unit. It can also be implemented as a partial configuration of the device.
  • LSI Large Scale Integration
  • this technology can be applied to a network system composed of a plurality of devices.
  • the present technology may be implemented as cloud computing that is shared and jointly processed by a plurality of devices via a network.
  • this technology is implemented in a cloud service that provides services related to images (moving images) to any terminal such as computers, AV (AudioVisual) devices, portable information processing terminals, and IoT (Internet of Things) devices. You may try to do it.
  • the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems. ..
  • Systems, devices, processing units, etc. to which this technology is applied can be used in any field such as transportation, medical care, crime prevention, agriculture, livestock industry, mining, beauty, factories, home appliances, weather, nature monitoring, etc. .. The use is also arbitrary.
  • this technology can be applied to systems and devices used for providing ornamental contents and the like.
  • the present technology can be applied to systems and devices used for traffic such as traffic condition supervision and automatic driving control.
  • the present technology can be applied to systems and devices used for security purposes.
  • the present technology can be applied to a system or device used for automatic control of a machine or the like.
  • the present technology can be applied to systems and devices used for agriculture and livestock industry.
  • the present technology can also be applied to systems and devices for monitoring natural conditions such as volcanoes, forests and oceans, and wildlife. Further, for example, the present technology can be applied to systems and devices used for sports.
  • the "flag” is information for identifying a plurality of states, and is not only information used for identifying two states of true (1) or false (0), but also three or more states. It also contains information that can identify the state. Therefore, the value that this "flag” can take may be, for example, 2 values of 1/0 or 3 or more values. That is, the number of bits constituting this "flag” is arbitrary, and may be 1 bit or a plurality of bits.
  • the identification information (including the flag) is assumed to include not only the identification information in the bit stream but also the difference information of the identification information with respect to a certain reference information in the bit stream. In, the "flag” and “identification information” include not only the information but also the difference information with respect to the reference information.
  • various information (metadata, etc.) regarding the coded data may be transmitted or recorded in any form as long as it is associated with the coded data.
  • the term "associate" means, for example, to make the other data available (linkable) when processing one data. That is, the data associated with each other may be combined as one data or may be individual data.
  • the second data associated with the first data may be transmitted on a transmission path different from that of the first data.
  • the second data associated with the first data may be recorded on a recording medium (or another recording area of the same recording medium) different from the first data.
  • this "association" may be a part of the data, not the entire data.
  • the 3D data and the metadata corresponding to the 3D data may be associated with each other in any unit such as a plurality of samples, one sample, or a part of the sample.
  • the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.
  • the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units).
  • the configurations described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit).
  • a configuration other than the above may be added to the configuration of each device (or each processing unit).
  • a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). ..
  • the above-mentioned program may be executed in any device.
  • the device may have necessary functions (functional blocks, etc.) so that necessary information can be obtained.
  • each step of one flowchart may be executed by one device, or may be shared and executed by a plurality of devices.
  • one device may execute the plurality of processes, or the plurality of devices may share and execute the plurality of processes.
  • a plurality of processes included in one step can be executed as processes of a plurality of steps.
  • the processes described as a plurality of steps can be collectively executed as one step.
  • the program executed by the computer may have the following characteristics. For example, the processing of the steps of writing a program may be performed in chronological order in the order described herein. Further, the processes of the steps for writing the program may be executed in parallel. Further, the processing of the step for writing the program may be executed individually at a required timing such as when it is called. That is, as long as there is no contradiction, the processes of each step may be executed in an order different from the above-mentioned order. Further, the processing of the step for describing this program may be executed in parallel with the processing of other programs. Further, the processing of the step for writing this program may be executed in combination with the processing of another program.
  • a plurality of technologies related to this technology can be independently implemented independently as long as there is no contradiction.
  • any plurality of the present technologies can be used in combination.
  • some or all of the techniques described in any of the embodiments may be combined with some or all of the techniques described in other embodiments.
  • a part or all of any of the above-mentioned techniques may be carried out in combination with other techniques not described above.
  • the present technology can also have the following configurations.
  • a scalable decoding information generator that generates scalable decoding information regarding the scalable decoding of the G-PCC content based on the dependency between the first slice and the second slice.
  • An information processing device including a content file generation unit that generates a content file for storing the G-PCC content and stores the scalable decoding information in the metadata area of the content file.
  • the scalable decoding information includes slice configuration information relating to the slice configuration for each sample.
  • the content file generation unit sets a subsample for each slice, and stores the slice configuration information in the codec spiritual parameters of the subsample information box of the metadata area.
  • Information processing device sets a subsample for each slice, and stores the slice dependency information in the codec spiritual parameters of the subsample information box of the metadata area.
  • Information processing equipment (6)
  • the slice dependency information includes the reference source geometry slice identification information and the reference destination geometry slice identification information.
  • the reference source geometry slice identification information is identification information of a geometry slice that is a reference source in the dependency relationship between the first slice and the second slice.
  • the information processing apparatus according to (6) or (7), which stores identification information.
  • the information processing apparatus according to any one of (4) to (8), wherein the slice dependency information includes attribute geometry slice identification information which is identification information of the geometry slice referred to by the attribute slice.
  • the content file generation unit sets a subsample for each slice, and stores the attribute geometry slice identification information in the codec spiritual parameters of the subsample information box of the metadata area (9).
  • the slice dependency information includes any of (4) to (8), which is non-scalable coded attribute geometry slice identification information which is identification information of the geometry slice referred to by the attribute slice to which the non-scalable coding is applied. Information processing device described in Crab.
  • the non-scalable coded attribute geometry slice identification information includes the identification information of the geometry slice including the geometry having the maximum depth information among the geometry slices referred to by the attribute slice (11).
  • Information processing equipment (13)
  • the content file generation unit sets a subsample for each slice, and stores the non-scalable coded attribute geometry slice identification information in the codec spiritual parameters of the subsample information box of the metadata area.
  • the information processing apparatus according to (11) or (12).
  • (14) The information according to any one of (11) to (13), wherein the non-scalable coded attribute geometry slice identification information includes a non-scalable coded flag indicating whether non-scalable coding has been applied to the attribute slice. Processing equipment.
  • the content file generation unit sets a subsample for each slice, and stores the non-scalable coding flag in the codec spiritual parameters of the subsample information box of the metadata area (14).
  • the payload type of the slice dependency information of the independently decodable independent geometry slice is different from the payload type of the slice dependency information of the dependent geometry slice that refers to the other geometry slice at the time of decoding (10).
  • the slice configuration information includes geometry slice depth information related to the depth information of the geometry included in the geometry slice.
  • the geometry slice depth information includes the minimum depth information indicating the minimum value of the depth information in the geometry slice.
  • the information processing apparatus according to (17) or (18), wherein the geometry slice depth information includes maximum depth information indicating the maximum value of the depth information in the geometry slice.
  • the content file generation unit sets a subsample for each slice, and stores the geometry slice depth information in the codec spiritual parameters of the subsample information box of the metadata area (17) to ( The information processing apparatus according to any one of 19).
  • the slice composition information further includes slice dependency information indicating the dependency between the first slice and the second slice.
  • the content file generation unit sets the flags of the subsample information box that stores the geometry slice depth information to a value different from the flags of the subsample information box that stores the slice dependency information (20).
  • the scalable decoding information includes track configuration information relating to the configuration of a track that stores the G-PCC content in slice units in the content file. ..
  • the track configuration information includes track depth information relating to the depth information of the geometry of the geometry of all the slices included in the track corresponding to the track configuration information.
  • the track depth information includes track minimum depth information indicating the minimum value of the depth information in the track.
  • the track depth information includes track maximum depth information indicating the maximum value of the depth information in the track.
  • the minimum value of the depth information in each sample included in the track matches the minimum value of the depth information in the track, and the depth in each sample included in the track.
  • the track configuration information includes track dependency information indicating a dependency between the first track and the second track.
  • the information processing apparatus includes dependent information indicating another track including the slice necessary for decoding the dependent slice contained in the track.
  • the dependent information indicates all the other tracks including the slice required for decoding the dependent slice.
  • the information processing apparatus indicates the other track including the slice referenced from the dependent slice.
  • the track dependency information includes independent information indicating other tracks including other slices that require an independent slice included in the track at the time of decoding, according to any one of (28) to (31). Information processing equipment.
  • the information processing apparatus indicates the other track including the other slice that refers to the independent slice at the time of decoding.
  • the content file generation unit stores the track dependency information as a track reference in the metadata area.
  • Depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice, and the said in the G-PCC content.
  • the scalable decoding information regarding the scalable decoding of the G-PCC content is generated.
  • the scalable decoding information is information related to the scalable decoding of the G-PCC content, and the depth information indicating the quality hierarchy level of the geometry contained in the slice in the G-PCC content and the first-first in the G-PCC content.
  • An information processing device that is information generated based on a dependency between a slice and the second slice.
  • the information processing apparatus includes slice configuration information relating to the configuration of the slice for each sample.
  • the extraction unit is from the content file based on the slice configuration information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice.
  • the information processing apparatus according to (52), wherein any slice of the G-PCC content is extracted.
  • the information processing apparatus according to (52) or (53), wherein the slice configuration information includes slice dependency information indicating the dependency between the first slice and the second slice.
  • the extraction unit is the content file based on the slice dependency information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice.
  • the information processing apparatus wherein any slice of the G-PCC content is extracted from the G-PCC content.
  • the slice dependency information includes the reference source geometry slice identification information and the reference destination geometry slice identification information.
  • the reference source geometry slice identification information is identification information of a geometry slice that is a reference source in the dependency relationship between the first slice and the second slice.
  • the information processing according to (54) or (55), wherein the referenced geometry slice identification information is identification information of a geometry slice to be referenced in the dependency between the first slice and the second slice.
  • the reference source geometry slice identification information and the reference destination geometry slice identification information are the same (56). ).
  • the information processing device When the geometry slice corresponding to the reference source geometry slice identification information is an independent geometry slice that can be independently decoded, the reference source geometry slice identification information and the reference destination geometry slice identification information are the same (56). ).
  • the information processing device When the geometry slice corresponding to the reference source geometry slice identification information is an independent geometry slice that can be independently decoded, the reference source geometry slice identification information and the reference
  • the extraction unit receives the reference source geometry slice identification information and the reference destination geometry stored in the codec-spirit parameters of the subsample information box of the metadata area of the subsample set for each slice.
  • the information processing apparatus according to (56) or (57), which extracts any said slice of the G-PCC content from the content file based on the slice identification information.
  • the information processing apparatus according to any one of (54) to (58), wherein the slice dependency information includes attribute geometry slice identification information which is identification information of the geometry slice referred to by the attribute slice.
  • the extraction unit is based on the attribute geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice.
  • the slice dependency information includes non-scalable coded attribute geometry slice identification information which is identification information of the geometry slice referred to by the attribute slice to which the non-scalable coding is applied. Information processing device described in Crab.
  • the non-scalable coded attribute geometry slice identification information includes the identification information of the geometry slice including the geometry having the maximum depth information among the geometry slices referred to by the attribute slice (61). Information processing equipment.
  • the extraction unit uses the non-scalable coded attribute geometry slice identification information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice.
  • the information processing apparatus which extracts any said slice of the G-PCC content from the content file based on the above.
  • Processing equipment. (65)
  • the extraction unit is based on the non-scalable coding flag stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice.
  • the information processing apparatus according to (64), wherein any slice of the G-PCC content is extracted from the content file.
  • the payload type of the slice dependency information of an independently decodable independent geometry slice is different from the payload type of the slice dependency information of a dependent geometry slice that references the other geometry slice during decoding (60). ) To (65). (67) The information processing apparatus according to any one of (52) to (66), wherein the slice configuration information includes geometry slice depth information relating to the depth information of the geometry included in the geometry slice. (68) The information processing apparatus according to (67), wherein the geometry slice depth information includes the minimum depth information indicating the minimum value of the depth information in the geometry slice. (69) The information processing apparatus according to (67) or (68), wherein the geometry slice depth information includes maximum depth information indicating the maximum value of the depth information in the geometry slice.
  • the extraction unit is based on the geometry slice depth information stored in the codec spiritual parameters of the subsample information box of the metadata area of the subsample set for each slice.
  • the information processing apparatus according to any one of (67) to (69), wherein any slice of the G-PCC content is extracted from a file.
  • the slice composition information further includes slice dependency information indicating the dependency between the first slice and the second slice.
  • the information according to (70), wherein the flags of the subsample information box that stores the geometry slice depth information and the flags of the subsample information box that stores the slice dependency information are set to different values. Processing equipment.
  • the scalable decoding information includes track configuration information relating to the configuration of a track that stores the G-PCC content in slice units in the content file. .. (73)
  • the track configuration information includes track depth information relating to the depth information of the geometry of the geometry of all the slices included in the track corresponding to the track configuration information.
  • the track depth information includes track minimum depth information indicating a minimum value of the depth information in the track.
  • the track depth information includes track maximum depth information indicating the maximum value of the depth information in the track.
  • the minimum value of the depth information in each sample included in the track matches the minimum value of the depth information in the track, and the depth in each sample included in the track.
  • the information processing apparatus according to any one of (73) to (75), which includes a match flag indicating whether the maximum value of information matches the maximum value of the depth information in the track.
  • the extraction unit extracts any slice of the G-PCC content from the content file based on the track depth information stored in the depth information box of the sample entry in the metadata area.
  • the information processing apparatus according to any one of (72) to (77) wherein the track configuration information includes track dependency information indicating a dependency between the first track and the second track.
  • the information processing apparatus includes dependent information indicating another track including the slice necessary for decoding the dependent slice contained in the track.
  • the dependent information indicates all the other tracks including the slice required for decoding the dependent slice.
  • the information processing apparatus indicates the other track including the slice referenced from the dependent slice.
  • the track dependency information includes independent information indicating other tracks including other slices that require an independent slice included in the track at the time of decoding, according to any one of (78) to (81). Information processing equipment.
  • the information processing apparatus indicates the other track including the other slice that refers to the independent slice at the time of decoding.
  • the extraction unit extracts any slice of the G-PCC content from the content file based on the track dependency information stored as a track reference in the metadata area (78) to.
  • the information processing apparatus according to any one of (83).
  • (85) From the content file based on the scalable decoding information stored in the metadata area of the content file that stores the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice.
  • G-PCC Geometry-based Point Cloud Compression
  • the scalable decoding information is information related to the scalable decoding of the G-PCC content, and the depth information indicating the quality hierarchy level of the geometry contained in the slice in the G-PCC content and the first-first in the G-PCC content.
  • An information processing method that is information generated based on a dependency between a slice and the second slice.
  • the adaptation set configuration information is information processing device that is information about the configuration of an adaptation set that describes information about the track of the content file.
  • the adaptation set configuration information includes adaptation set depth information relating to the depth information of the geometry of the geometry of all the slices included in the track corresponding to the adaptation set.
  • the adaptation set depth information includes the adaptation set minimum depth information indicating the minimum value of the depth information in the track.
  • the adaptation set depth information includes adaptation set maximum depth information indicating the maximum value of the depth information in the track.
  • the sample minimum depth information of each sample included in the track matches the adaptation set minimum depth information
  • the sample maximum depth information of each sample included in the track is the adaptation set maximum.
  • the sample minimum depth information indicates the minimum value of the depth information in the sample.
  • the sample maximum depth information indicates the maximum value of the depth information in the sample.
  • the adaptation set minimum depth information indicates the minimum value of the depth information in the track.
  • the information processing apparatus according to any one of (102) to (104), wherein the adaptation set maximum depth information indicates the maximum value of the depth information in the track.
  • the information processing apparatus according to any one of (102) to (104), wherein the adaptation set maximum depth information indicates the maximum value of the depth information in the track.
  • the adaptation set configuration information includes representation dependency information indicating a dependency between a first representation and a second representation.
  • the representation dependency information includes dependent information indicating other representations necessary for decoding the representation corresponding to the representation dependency information.
  • the information processing apparatus wherein the dependent information indicates all the other representations necessary for decoding the representation corresponding to the dependent information.
  • the representation dependency information includes any of (106) to (109) including independent information indicating other representations for which a representation corresponding to the representation dependency information is required at the time of decoding. The information processing device described. (111) The information processing apparatus according to (110), wherein the independent information indicates all the other representations for which the representation corresponding to the independent information is required at the time of decoding.
  • the information processing apparatus according to any one of (106) to (111), wherein the control file generation unit stores the representation dependency information in the control file as a representation association ID and an association type.
  • Depth information indicating the quality hierarchy level of the geometry contained in each slice in the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice, and the said in the G-PCC content.
  • Adaptation set configuration information is generated based on the dependency between the first slice and the second slice.
  • a control file for controlling the reproduction of the content file for storing the G-PCC content is generated, and the adaptation set configuration information is stored in the control file.
  • the content file stores the G-PCC content in a track in slice units.
  • the adaptation set configuration information is information processing method that is information regarding the configuration of an adaptation set that describes information about the track of the content file.
  • the control file that controls the reproduction of the content file that stores the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice in the track in slice units is analyzed, and the control file is described.
  • An analysis unit that identifies the adaptation set required to obtain an arbitrary slice of the G-PCC content based on the adaptation set configuration information stored in.
  • An acquisition unit that acquires the track corresponding to the adaptation set specified by the analysis unit of the content file, and an acquisition unit.
  • a decoding unit for decoding the slice of the G-PCC content stored in the track acquired by the acquisition unit is provided.
  • the adaptation set configuration information is information about the configuration of the adaptation set that describes the information about the track of the content file, and is depth information indicating the quality hierarchy level of the geometry contained in the slice in the G-PCC content and the said.
  • An information processing device that is information generated based on the dependency between the first slice and the second slice in the G-PCC content.
  • the adaptation set configuration information includes adaptation set depth information relating to the depth information of the geometry of the geometry of all the slices included in the track corresponding to the adaptation set.
  • the adaptation set depth information includes the adaptation set minimum depth information indicating the minimum value of the depth information in the track.
  • the adaptation set depth information includes adaptation set maximum depth information indicating the maximum value of the depth information in the track.
  • the sample minimum depth information of each sample included in the track matches the adaptation set minimum depth information
  • the sample maximum depth information of each sample included in the track is the adaptation set maximum. Includes a match flag to indicate if it matches the depth information
  • the sample minimum depth information indicates the minimum value of the depth information in the sample.
  • the sample maximum depth information indicates the maximum value of the depth information in the sample.
  • the adaptation set minimum depth information indicates the minimum value of the depth information in the track.
  • the information processing apparatus according to any one of (152) to (154), wherein the adaptation set maximum depth information indicates the maximum value of the depth information in the track.
  • the adaptation set configuration information includes representation dependency information indicating a dependency between a first representation and a second representation.
  • the representation dependency information includes dependent information indicating other representations necessary for decoding the representation corresponding to the representation dependency information.
  • the dependent information indicates all the other representations necessary for decoding the representation corresponding to the dependent information.
  • the dependent information indicates the other representation referred to from the representation corresponding to the dependent information.
  • the representation dependency information includes any of (156) to (159) including independent information indicating other representations for which a representation corresponding to the representation dependency information is required at the time of decoding.
  • the information processing device described. (161) The information processing apparatus according to (160), wherein the independent information indicates all the other representations for which the representation corresponding to the independent information is required at the time of decoding.
  • the analysis unit identifies the adaptation set based on the representation association ID and the representation dependency information stored as the association type in the control file (156) to (161).
  • the information processing device described. (163) The control file that controls the reproduction of the content file that stores the G-PCC (Geometry-based Point Cloud Compression) content including the first slice and the second slice in the track in slice units is analyzed, and the control file is described.
  • G-PCC Geometry-based Point Cloud Compression
  • the adaptation set configuration information is information about the configuration of the adaptation set that describes the information about the track of the content file, and is depth information indicating the quality hierarchy level of the geometry contained in the slice in the G-PCC content and the said.
  • An information processing method that is information generated based on the dependency between the first slice and the second slice in the G-PCC content.
  • 300 file generation device 311 extraction unit, 312 coding unit, 313 bitstream generation unit, 314 scalable decoding information generation unit, 315 file generation unit, 321 geometry coding unit, 322 attribute coding unit, 323 metadata generation unit, 400 playback device, 401 control unit, 411 file acquisition unit, 412 playback processing unit, 413 presentation processing unit, 421 file processing unit, 422 decoding unit, 423 presentation information generation unit, 431 bitstream extraction unit, 441 geometry decoding unit, 442 Attribute decoding unit, 451 point cloud construction unit, 452 presentation processing unit, 600 file generation device, 615 content file generation unit, 616 MPD generation unit, 700 playback device, 711 file acquisition unit, 712 MPD analysis unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
PCT/JP2021/032594 2020-09-08 2021-09-06 情報処理装置および方法 Ceased WO2022054744A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US18/040,820 US12341981B2 (en) 2020-09-08 2021-09-06 Information processing device and method
JP2022547574A JP7746995B2 (ja) 2020-09-08 2021-09-06 情報処理装置および方法
CN202180053432.7A CN116157838B (zh) 2020-09-08 2021-09-06 信息处理装置和方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063075361P 2020-09-08 2020-09-08
US63/075,361 2020-09-08

Publications (1)

Publication Number Publication Date
WO2022054744A1 true WO2022054744A1 (ja) 2022-03-17

Family

ID=80847060

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/032594 Ceased WO2022054744A1 (ja) 2020-09-08 2021-09-06 情報処理装置および方法

Country Status (4)

Country Link
US (1) US12341981B2 (https=)
JP (1) JP7746995B2 (https=)
CN (1) CN116157838B (https=)
WO (1) WO2022054744A1 (https=)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115379189A (zh) * 2022-08-19 2022-11-22 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法及相关设备
JP2025501125A (ja) * 2021-12-21 2025-01-17 北京字節跳動網絡技術有限公司 点群コーディングのための方法、装置及び媒体
JP2025510565A (ja) * 2022-04-05 2025-04-15 キヤノン株式会社 ポイントクラウドデータのマルチトラックカプセル化を改善するための方法、装置、およびコンピュータプログラム

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116781913A (zh) * 2022-03-11 2023-09-19 腾讯科技（深圳）有限公司 点云媒体的编解码方法及相关产品

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020008106A1 (en) * 2018-07-02 2020-01-09 Nokia Technologies Oy An apparatus, a method and a computer program for video coding and decoding
WO2020241723A1 (ja) * 2019-05-28 2020-12-03 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
US20210029187A1 (en) * 2019-07-03 2021-01-28 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus and point cloud data reception method
WO2021049333A1 (ja) * 2019-09-11 2021-03-18 ソニー株式会社 情報処理装置、情報処理方法、再生処理装置及び再生処理方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2453652B1 (en) * 2009-07-06 2019-09-04 Huawei Technologies Co., Ltd. Transmission method, receiving method and device for scalable video coding files
US9521418B2 (en) * 2011-07-22 2016-12-13 Qualcomm Incorporated Slice header three-dimensional video extension for slice header prediction
US9848202B2 (en) * 2012-12-28 2017-12-19 Electronics And Telecommunications Research Institute Method and apparatus for image encoding/decoding
US20150264404A1 (en) * 2014-03-17 2015-09-17 Nokia Technologies Oy Method and apparatus for video coding and decoding
WO2020012073A1 (en) * 2018-07-11 2020-01-16 Nokia Technologies Oy Method and apparatus for storage and signaling of compressed point clouds
WO2020070379A1 (en) * 2018-10-03 2020-04-09 Nokia Technologies Oy Method and apparatus for storage and signaling of compressed point clouds
KR102695285B1 (ko) * 2018-10-09 2024-08-14 파나소닉 인텔렉츄얼 프로퍼티 코포레이션 오브 아메리카 삼차원 데이터 부호화 방법, 삼차원 데이터 복호 방법, 삼차원 데이터 부호화 장치, 및 삼차원 데이터 복호 장치
CN117061767A (zh) * 2018-12-28 2023-11-14 索尼集团公司 信息处理装置和信息处理方法
US11558643B2 (en) * 2020-04-08 2023-01-17 Qualcomm Incorporated Secondary component attribute coding for geometry-based point cloud compression (G-PCC)
CA3169708A1 (en) * 2020-06-24 2021-12-30 Zte Corporation Three-dimensional content processing methods and apparatus
WO2022035256A1 (ko) 2020-08-12 2022-02-17 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020008106A1 (en) * 2018-07-02 2020-01-09 Nokia Technologies Oy An apparatus, a method and a computer program for video coding and decoding
WO2020241723A1 (ja) * 2019-05-28 2020-12-03 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
US20210029187A1 (en) * 2019-07-03 2021-01-28 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus and point cloud data reception method
WO2021049333A1 (ja) * 2019-09-11 2021-03-18 ソニー株式会社 情報処理装置、情報処理方法、再生処理装置及び再生処理方法

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2025501125A (ja) * 2021-12-21 2025-01-17 北京字節跳動網絡技術有限公司 点群コーディングのための方法、装置及び媒体
JP7827358B2 (ja) 2021-12-21 2026-03-10 北京字節跳動網絡技術有限公司 点群コーディングのための方法、装置及び媒体
JP2025510565A (ja) * 2022-04-05 2025-04-15 キヤノン株式会社 ポイントクラウドデータのマルチトラックカプセル化を改善するための方法、装置、およびコンピュータプログラム
JP7825066B2 (ja) 2022-04-05 2026-03-05 キヤノン株式会社 ポイントクラウドデータのマルチトラックカプセル化を改善するための方法、装置、およびコンピュータプログラム
CN115379189A (zh) * 2022-08-19 2022-11-22 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法及相关设备
CN115379189B (zh) * 2022-08-19 2023-12-12 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法及相关设备

Also Published As

Publication number Publication date
US20230291918A1 (en) 2023-09-14
CN116157838B (zh) 2025-09-30
US12341981B2 (en) 2025-06-24
CN116157838A (zh) 2023-05-23
JPWO2022054744A1 (https=) 2022-03-17
JP7746995B2 (ja) 2025-10-01

Similar Documents

Publication Publication Date Title
JP7563385B2 (ja) 情報処理装置、情報処理方法、再生処理装置及び再生処理方法
US11532103B2 (en) Information processing apparatus and information processing method
JP7726209B2 (ja) 情報処理装置および方法
JP7746995B2 (ja) 情報処理装置および方法
US11516453B2 (en) Information processing apparatus, information processing method, and program for point cloud sample processing
JP7768130B2 (ja) 情報処理装置および方法
US12432370B2 (en) Information processing device and information processing method
GB2509953A (en) Displaying a Region of Interest in a Video Stream by Providing Links Between Encapsulated Video Streams
JP7331852B2 (ja) 画像処理装置および方法
WO2021117859A1 (ja) 画像処理装置および方法
WO2023176419A1 (ja) 情報処理装置および方法
US12587678B2 (en) Information processing apparatus and method thereof
WO2022059495A1 (ja) 情報処理装置および方法
WO2022138221A1 (ja) 情報処理装置および方法
WO2021193088A1 (ja) 画像処理装置および方法
WO2024143466A1 (ja) 情報処理装置および方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21866703

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022547574

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21866703

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 18040820

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 202180053432.7

Country of ref document: CN