WO2022068672A1 - 点云数据处理方法、装置、存储介质及电子装置 - Google Patents

点云数据处理方法、装置、存储介质及电子装置 Download PDF

Info

Publication number
WO2022068672A1
WO2022068672A1 PCT/CN2021/119964 CN2021119964W WO2022068672A1 WO 2022068672 A1 WO2022068672 A1 WO 2022068672A1 CN 2021119964 W CN2021119964 W CN 2021119964W WO 2022068672 A1 WO2022068672 A1 WO 2022068672A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
item
data
static
scene
Prior art date
Application number
PCT/CN2021/119964
Other languages
English (en)
French (fr)
Inventor
白雅贤
黄成�
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to US18/246,868 priority Critical patent/US20230360304A1/en
Priority to EP21874333.4A priority patent/EP4224871A4/en
Priority to CA3194294A priority patent/CA3194294A1/en
Publication of WO2022068672A1 publication Critical patent/WO2022068672A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/85406—Content authoring involving a specific file format, e.g. MP4 format
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/36—Level of detail
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/56—Particle system, point based geometry or rendering

Definitions

  • the embodiments of the present application relate to the field of communications, and in particular, to a point cloud data processing method, device, storage medium, and electronic device.
  • a three-dimensional object can be directly digitized by a lidar or a camera matrix, and a three-dimensional object can be represented by independent points and surfaces, that is, a three-dimensional point cloud ( Point Cloud).
  • Point Cloud is a set of discrete points that are randomly distributed in space and express the spatial structure and surface properties of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may also have color, material or other information depending on the application scenario.
  • Point clouds can be captured by multiple cameras and depth sensors, and the number of points can reach thousands to billions. There is no connection and order between points and points, and can be sorted at will. Therefore, point clouds are flexible and convenient in their expression. , high data accuracy, aroused widespread concern in the industry.
  • point clouds are relatively easy to capture and render, and the application scenarios include: autonomous driving, real-time inspection, cultural heritage, 6DoF immersive real-time communication, etc.
  • point cloud can be divided into two categories: static point cloud and dynamic point cloud.
  • static point clouds such as large-scale static point clouds such as cities and streets
  • users usually only need to pay attention to part of the point cloud data when applying point cloud data, and do not need to obtain the complete point cloud objects, for example:
  • the scene is selected according to the geographical area and viewed randomly;
  • the digital cultural heritage supports the selection according to the perspective, and the virtual tour;
  • the user only pays attention to the point cloud data in some areas or the point cloud data containing some details.
  • the decoder usually needs to traverse the complete point cloud code stream before identifying the user. Part of the required point cloud data, but for large point cloud objects, such as urban scenes, map navigation, etc., this solution is inefficient and time-consuming; and in the existing static point cloud storage solutions, the entire object is usually used as the storage unit , which is not conducive to parallel decoding and has low flexibility.
  • the embodiments of the present application provide a point cloud data processing method, device, storage medium, and electronic device, so as to at least solve the problem in the related art that it is usually necessary to traverse the complete point cloud data before identifying part of the point cloud data required by the user, resulting in analysis of Inefficient and time-consuming.
  • a method for processing point cloud data comprising:
  • a partial spatial region of the 3D scene is rendered according to the decoded data.
  • a method for processing point cloud data comprising:
  • Part of the level of detail of the 3D scene is rendered from the decoded data.
  • a point cloud data processing device comprising:
  • a first determining module configured to determine static geometrically encoded point cloud data including spatial region information of the 3D scene, wherein the static geometrically encoded point cloud data is represented by point cloud item data;
  • a first decoding module configured to decode part of the item data corresponding to part of the spatial region of the 3D scene in the static geometrically encoded point cloud data
  • the first rendering module is configured to render a partial spatial region of the 3D scene according to the decoded data.
  • a point cloud data processing device comprising:
  • a second determination module configured to determine static geometry-encoded point cloud data including hierarchical detail information of the 3D scene, wherein the static geometry-encoded point cloud data is represented by cloud item data;
  • a second decoding module configured to decode part of the item data corresponding to part of the level details of the 3D scene in the static geometrically encoded point cloud data
  • the second rendering module is configured to render some level details of the 3D scene according to the decoded data.
  • a computer-readable storage medium is also provided, and a computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute any one of the above methods when running steps in the examples.
  • an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any one of the above Steps in Method Examples.
  • the present application can solve the problem of low analysis efficiency and long time consuming in the related art that it is usually necessary to traverse the complete point cloud data before identifying part of the point cloud data required by the user. This makes it possible to obtain part of the required project data without traversing the complete point cloud data, which improves the decoding efficiency, shortens the decoding time, and shortens the user's rendering waiting time.
  • Fig. 1 is the hardware structure block diagram of the mobile terminal of the point cloud data processing method of the embodiment of the present application
  • FIG. 2 is a flowchart 1 of a method for processing point cloud data according to an embodiment of the present application
  • FIG. 3 is a second flowchart of a method for processing point cloud data according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a single-item storage structure of a static point cloud according to the present embodiment
  • FIG. 5 is a schematic diagram of a multi-item storage structure of a static point cloud according to the present embodiment
  • Fig. 6 is the schematic diagram of the static point cloud storage based on Subsample according to the present embodiment.
  • FIG. 7 is a schematic diagram of subsample division based on a spatial region according to the present embodiment.
  • Fig. 8 is the schematic diagram of the static point cloud storage based on Tile Item according to the present embodiment.
  • FIG. 9 is a schematic diagram of static point cloud storage based on 3D region property and Subsample according to the present embodiment.
  • FIG. 10 is a schematic diagram of static point cloud storage based on Region Item according to the present embodiment.
  • FIG. 11 is a schematic diagram of subsample division based on hierarchical information according to the present embodiment.
  • Fig. 12 is a schematic diagram of static point cloud storage based on Layer Item according to the present embodiment.
  • FIG. 13 is a schematic diagram of static point cloud storage based on spatial regions and hierarchical details according to the present embodiment
  • FIG. 14 is a schematic diagram of a static point cloud space region marking according to the present embodiment.
  • FIG. 15 is a schematic diagram of associated storage of a static point cloud and a dynamic point cloud according to the present embodiment
  • FIG. 16 is a block diagram 1 of a point cloud data processing apparatus according to the present embodiment.
  • FIG. 17 is a second block diagram of a point cloud data processing apparatus according to the present embodiment.
  • FIG. 1 is a block diagram of the hardware structure of a mobile terminal of a point cloud data processing method according to an embodiment of the present application.
  • the mobile terminal may include one or more (only shown in FIG. 1 ).
  • a processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data, wherein the above-mentioned mobile terminal may also include a communication device for communication Function transmission device 106 and input and output device 108.
  • a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.
  • a memory 104 for storing data
  • the above-mentioned mobile terminal may also include a communication device for communication Function transmission device 106 and input and output device 108.
  • FIG. 1 is only a schematic diagram, which does not limit the structure of the above-mentioned mobile terminal.
  • the mobile terminal may also include more or fewer components than those shown in FIG. 1 , or have a different configuration than that shown in FIG. 1 .
  • the memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the data processing methods in the embodiments of the present application. A functional application and data processing are implemented, namely, the above-mentioned method is implemented.
  • Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory located remotely from the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • Transmission means 106 are used to receive or transmit data via a network.
  • the specific example of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal.
  • the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet.
  • the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet in a wireless manner.
  • RF Radio Frequency
  • FIG. 2 is a flowchart 1 of a point cloud data processing method according to an embodiment of the present application. As shown in FIG. 2 , the The process includes the following steps:
  • Step S202 determining static geometrically encoded point cloud data that includes spatial region information of the 3D scene, wherein the static geometrically encoded point cloud data is represented by point cloud item data;
  • the point cloud item data is composed of geometric item data and one or more attribute item data, the geometric item data and attribute item data are divided into multiple levels, and the point cloud item data of each level is It consists of a uniformly distributed set of points.
  • Step S204 decoding part of item data corresponding to part of the spatial region of the 3D scene in the static geometrically encoded point cloud data
  • Step S206 rendering a partial spatial region of the 3D scene according to the decoded data.
  • the project data makes it possible to obtain part of the required project data without traversing the complete point cloud data, which improves the decoding efficiency, shortens the decoding time, and shortens the user's rendering waiting time.
  • the point cloud item data is determined by the item type in the metadata data box, wherein the item type at least includes: point cloud item, point cloud block item, and point cloud space area item.
  • the above step S204 may specifically include: in the case that the item type is the point cloud item, determining the corresponding item according to the characteristics of the sub-sample item associated with the point cloud item in the metadata data box. For the partial item data of one or more blocks, further, determine the partial item data corresponding to one or more blocks according to the first subsample data type and block identifier in the subsample item characteristics, The partial item data is decoded.
  • the above step S204 may specifically include: in the case that the item type is the point cloud item, according to the sub-sample item characteristics associated with the point cloud item in the metadata data box, and The 3D space area item characteristic determines the part of the item data corresponding to one or more space areas, and further, according to the second subsample data type and the space area identifier in the subsample item characteristic, it is determined corresponding to one or more space areas.
  • the part of the item data of , wherein the 3D space area item characteristics include: the number of space areas, the description information of the space area, the number of blocks contained in the space area, the corresponding identifiers of the blocks contained in the space area, the description of the space area
  • the information includes: space area identifier, anchor point coordinates, length, width, and height of the space area, and the partial item data is decoded.
  • the above-mentioned step S204 may specifically include: in the case that the item type is a point cloud segmented item, identifying the point cloud data including the 3D static scene according to the point cloud full cluster group; according to the point cloud configuration The set of point cloud segments in the item property, and one or more point cloud segment items in the metadata data box determine the partial item data corresponding to the one or more segments, and decode the partial item data.
  • the point cloud tile entry includes a single tile described in a point cloud tile set, wherein each tile of the one or more tiles represents an independently decodable region of the point cloud space. subset of data.
  • the above step S204 may specifically include: in the case that the item type is a point cloud space area item, identifying point cloud data including the 3D static scene according to the point cloud full cluster group; one or more point cloud space area items in the information data box, the space area associated with the point cloud space item describes the item characteristics, determines the part of the item data corresponding to the one or more point cloud space areas, decodes the part of the item data, wherein, each point cloud space region in the one or more point cloud space regions corresponds to a single space region described by the characteristics of the space region description item.
  • the characteristics of the spatial area description item include: a spatial area identifier, an anchor point coordinate, and the length, width, and height of the spatial area.
  • the point cloud full-cluster group includes: the number of point cloud subsets accessed by the most divided part of the complete point cloud object, the type of point cloud subsets, and the identifiers of the point cloud subsets, wherein,
  • the point cloud subsets include: point cloud subsets divided according to blocks, and point cloud subsets divided according to spatial regions.
  • the above step S202 may specifically include: in the case that the item type is a point cloud space area item, determining part of the item data for indicating the marked scene according to the area marker group, and decoding the part of the item data.
  • the area marker group includes at least a marker type and a marker text, and the part of the point cloud items indicating the marker scene is described by the marker type and the marker text.
  • one or more media resources associated with the partial item data are determined and acquired, and specifically, one or more media resources associated with the partial item data may be determined and acquired according to a multi-entity playback group. while decoding the partial item data, decoding the one or more media resources; while rendering the partial spatial region of the 3D scene according to the decoded data, decoding the one or more media resources media resources to play.
  • the media resources include at least: video, audio, and dynamic point cloud.
  • FIG. 3 is a second flowchart of the method for processing point cloud data according to an embodiment of the present application. As shown in FIG. 3 , the process includes the following steps :
  • Step S302 determining static geometrically encoded point cloud data that includes hierarchical detail information of the 3D scene, wherein the static geometrically encoded point cloud data is represented by cloud item data;
  • the point cloud item data is composed of geometric item data and one or more attribute item data, the geometric item data and attribute item data are divided into multiple levels, and each level is composed of a uniformly distributed Group point set composition.
  • Step S304 decoding part of the item data corresponding to part of the level details of the 3D scene in the static geometrically encoded point cloud data
  • Step S306 rendering some level details of the 3D scene according to the decoded data.
  • the project data makes it possible to obtain part of the required project data without traversing the complete point cloud data, which improves the decoding efficiency, shortens the decoding time, and shortens the user's rendering waiting time.
  • the point cloud item data is determined by the item type in the metadata data box, and the item type at least includes: point cloud item and point cloud level item.
  • the above-mentioned step S304 may specifically include: in the case that the item type is a point cloud item, determining the corresponding item according to the characteristics of the sub-sample item associated with the point cloud item in the metadata data box. or partial item data of multiple levels, further, the partial item data corresponding to one or more levels can be determined according to the third subsample data type and level value in the subsample item property, and the partial item data can be decoded.
  • the above step S304 may specifically include: in the case that the item type is a point cloud level item, identifying point cloud data including all details of the 3D static scene according to the point cloud full cluster group; according to the One or more point cloud level items in the metadata data box, and the level detail item properties associated with the point cloud level items determine partial item data corresponding to the one or more levels, decode the partial item data, wherein the Each of the one or more levels corresponds to a single level of level detail item characteristic description; the level detail item characteristic includes at least: point cloud level value, level point count.
  • the point cloud full-cluster group includes: the number of point cloud subsets accessed by the most divided part of the complete point cloud object, the type of point cloud subsets, and the identifiers of the point cloud subsets, wherein, The subset of point clouds includes subsets of point clouds divided according to the level of detail.
  • one or more media resources associated with the partial item data are determined and acquired, and specifically, one or more media resources associated with the partial item data may be determined and acquired according to a multi-entity playback group. while decoding the part of the project data, decoding the one or more media resources; while rendering part of the level details of the 3D scene according to the decoded data, decoding the one or more media resources media resources to play.
  • the media resources include at least: video, audio, and dynamic point cloud.
  • This embodiment can use the ISO International Organization for Standardization (International Organization for Standardization) basic media file format to describe 3D point cloud compressed data that does not contain time information, support the description of 3D point cloud space area information and level information, and then provide point cloud Part of the data access mechanism, so that the decoder can select part of the 3D point cloud data for analysis, decoding, and rendering according to the user space position, viewing direction and other information, and improve the processing efficiency of static point clouds.
  • Partial access of 3D point cloud includes at least the following two ways: partial access based on spatial area; progressive access based on level details.
  • items in the ISOBMFF media file format and related data boxes are used to store static 3D point cloud data and metadata information required for partial access, which can meet the needs of partial access to 3D point clouds in various scenarios. The semantics are further described in the Examples.
  • the implementation of this embodiment is based on the ISO (basic media file format to store the spatial position information, hierarchical information, and relationship information of the static 3D point cloud in the media file.
  • the basic media file format can refer to ISO/IEC JTC1/SC29/WG11 motion.
  • the MPEG-4 Part 12 ISO Base Media File Format formulated by the Picture Experts Group (Moving Picture Experts Group, MPEG for short) is operated.
  • the static point cloud compression data format can refer to ISO/IEC JTC1/SC29/WG11 Moving Picture Experts Group (MPEG ) formulated MPEG-I Part 9:Geometry-based Point Cloud Compression (G-PCC) is based on the geometry-coded point cloud compression technology to operate.
  • All data in the ISOBMFF media file is packed in a data box, and its type and size can be described in the header of the data box; if a box supports nesting (one box contains another subbox), the data A box is called a container box that supports nesting.
  • Static data (such as images) in ISOBMFF media files are usually stored in the "item" format.
  • the metadata box (Meta box) can store the item's description information, metadata, and can also contain specific payloads.
  • the metadata box supports Integrated storage of multiple static media objects, such as image collections.
  • Static point cloud data supports two basic encapsulation methods:
  • FIG. 4 is a schematic diagram of the single-item storage structure of the static point cloud according to the present embodiment, as shown in FIG. 4 , encapsulates all the geometric data and attribute data of the G-PCC point cloud with a single item. , Sequence Parameter Set (SPS for short), Attribute Parameter Set (APS), Geometry Parameter Set (Geometry Parameter Set, GPS for short) and other general parameters such as Tile Inventory
  • SPS Sequence Parameter Set
  • APS Attribute Parameter Set
  • Geometry Parameter Set Geometry Parameter Set
  • GPS GPS for short
  • Tile Inventory The metadata information is described in the item property (Item Property), the type is "gpcc", and is associated with the point cloud item (G-PCC Item).
  • FIG. 5 is a schematic diagram of the multi-item storage structure of the static point cloud according to the present embodiment.
  • the geometric data and attribute data of the G-PCC point cloud are packaged in multiple In Item, the complete point cloud contains a point cloud geometry item (G-PCC Geometry Item) and one or more point cloud attribute items (G-PCC Attribute Item), as shown in Figure 2.
  • Parameter information such as point cloud SPS, APS, GPS and general metadata information such as Tile Inventory are described in Item Property, which are associated with multiple items such as geometric data and attribute data.
  • Item Reference Box (iref) refers to one or more property items.
  • This embodiment supports two basic static point cloud encapsulation formats, Single-item and Multiple-item.
  • the Single-item encapsulation format is used as an example to expand the definition, which is only used to explain this application and is not used to limit it. this application
  • Example 1 Partial access to static point clouds based on spatial regions
  • This embodiment describes a method for accessing a static 3D point cloud part based on a spatial area, and a method for describing point cloud spatial area information in a media file.
  • the point cloud data corresponding to each spatial region supports independent decoding and rendering.
  • the point cloud data is stored in multiple point cloud tile items (G-PCC Tile Items) according to different spatial areas, and corresponds to the area division in the tile set.
  • the terminal can select one or more tile items to complete decoding, rendering etc.
  • the complete point cloud data is stored in a single Item, and the 3D space area division of the point cloud data is described by the 3D space area item property (3D region item property) and the subsample item property.
  • the terminal can select one or more area subsamples ( Region Sub-sample) to complete decoding, rendering and other operations;
  • G-PCC Region Item Store the point cloud data in multiple point cloud space area items (G-PCC Region Item) according to different space areas.
  • Each area item describes the space area range through the space area item property (Region Description item property), and the terminal can choose one or multiple area items to complete decoding, rendering, etc.
  • the terminal parsing process includes:
  • the terminal identifies items in the metadata box containing static point cloud data according to one of the following item types:
  • Point cloud projects: gpcc, gpcg, gpca;
  • Point cloud segmentation projects: gpct, gptg, gpta;
  • Point cloud space area projects: gpcr, gprg, gpra;
  • the terminal reads the point cloud configuration item property (G-PCC Configuration Item Property) associated with the point cloud item, reads the parameter information and configuration information such as SPS, APS, GPS, Tile Inventory, etc., and completes the initialization of the decoder;
  • the terminal determines the spatial area information and data range of the static point cloud data according to one of the following methods:
  • Method 1 Read the subsample item property (Subsample Item Property), if flags is 1, and data_type is 0, read the G-PCC Tile data range and tile_id indicated by each Sub-sample;
  • Method 2 Determine the G-PCC Tile data range and its tile_id according to the point cloud block item
  • Method 3 Read the 3D space area item property (3D Region Item Property), analyze the coordinates, range and other information of each 3D space area, read the subsample item property (Subsample Item Property), if flags is 1, and data_type is 1, Read the G-PCC 3D region data range and its region_id indicated by each Sub-sample;
  • Method 4 Determine the G-PCC 3D region data range and its region_id according to the point cloud space area item;
  • the terminal calculates according to user requirements and spatial region information, and determines one or more tiles or 3D regions to be decoded;
  • the terminal reads part of the G-PCC data, and inputs it into the decoder to complete decoding;
  • the renderer renders a partial spatial area of the 3D scene.
  • Method 1 Subsample describes the division of spatial regions
  • FIG. 6 is a schematic diagram of the static point cloud storage based on Subsample according to the present embodiment.
  • the complete point cloud data is stored in a single Item, and the space of the point cloud data is described by the Sub-sample item property attribute information.
  • Regional division The minimum part of the G-PCC point cloud data obtained by the terminal according to the spatial area is one G-PCC Tile. Tile spatial location, range and other information are described in G-PCC Configuration Item Property, and Subsample Item Property is used to describe the spatial area division of point cloud data in G-PCC Item, taking the basic package format of Single-item as an example, the storage based on Subsample The structure is shown in Figure 6.
  • the storage structure based on Subsample is similar to Figure 6, the difference is that the static point cloud access entry is G-PCC Geometry Item of type 'gpcg', which is the same as G-PCC Configuration Item Property, Subsample Item Property two item properties are associated, G-PCC Geometry Item refers to one or more G-PCC Attribute Item property items of type 'gpca' through the item reference data box (Item Reference Box, iref) of type 'gpca', each A property item is associated with a Subsample Item Property.
  • G-PCC Geometry Item refers to one or more G-PCC Attribute Item property items of type 'gpca' through the item reference data box (Item Reference Box, iref) of type 'gpca', each A property item is associated with a Subsample Item Property.
  • FIG. 7 is a schematic diagram of the subsample division based on the space area according to the present embodiment.
  • the G-PCC Subsample Item Property is as follows:
  • a data_type indicates the spatial region type corresponding to a single sub-sample
  • a data_type of 0 indicates that the sub-sample corresponds to a G-PCC tile
  • a data_type of 0 indicates that the sub-sample corresponds to a 3D region, that is, multiple G-PCC tiles
  • tile_id indicating the G-PCC tile identifier corresponding to the sub-sample, which is consistent with the tile identifier in Tile Inventory
  • 3D_region_id indicating the G-PCC 3D region identifier corresponding to the sub-sample, which is consistent with the identifier in the 3D Region Item Property
  • FIG. 8 is a schematic diagram of static point cloud storage based on Tile Items according to this embodiment.
  • this embodiment describes a method for accessing parts of static 3D point clouds based on multiple items, and stores point cloud data in different spatial areas.
  • multiple Region Items are described as complete static 3D point clouds through the point cloud full cluster group data box (CompletePointCloudGroupBox), the group type is 'cppc', and the group contains geometric data.
  • CompletePointCloudGroupBox point cloud full cluster group data box
  • the group type is 'cppc'
  • the group contains geometric data.
  • the mapping relationship between Tile Item and Tile_id in Tile Inventory Taking the basic package format of Single-item as an example, the static point cloud storage structure based on Tile item is shown in Figure 8.
  • the storage structure based on Tile Item is similar to Figure 8, the difference is that the static point cloud access entry is the G-PCC Geometry Tile Item of type 'gptg', which is the same as the G-PCC Configuration Item Property item. Attribute association, G-PCC Geometry Tile Item refers to one or more attribute items of type 'gpta' through the item reference data box (Item Reference Box, iref) of type 'gpca'.
  • the 'gpct' type represents G-PCC Tile Item, which consists of G-PCC part of the item data.
  • Each Tile Item represents a subset of independently decoded point clouds belonging to a block, which corresponds to the Tile description in Tile Inventory.
  • the 'gptg' type represents the G-PCC Geometry Tile Item, which is composed of part of the G-PCC item data, and each Item represents the point cloud geometric data belonging to a certain block;
  • the 'gpta' type represents the G-PCC Attribute Tile Item, which consists of part of the G-PCC item data, and each Item represents a type of point cloud attribute data belonging to a certain block;
  • partial_gpcc_num which indicates the maximum number of partial access subsets that can be divided in the complete point cloud object
  • tile_enable a value of 1 indicates that the group describes the combination of Tile Items, otherwise the value is 0;
  • region_enable a value of 1 indicates that this group describes the combination of Region Items, otherwise the value is 0;
  • a value of 1 indicates that this group describes the combination of Layer Items, otherwise the value is 0;
  • entry_id indicating the index of a subset G-PCC Item (Tile Item, Region Item or Layer Item) in EntityToGroupBox;
  • tile_id describing the tile_id corresponding to the Tile Item
  • 3DSpatialRegionStruct() which describes the spatial region information corresponding to the Region Item
  • lod_value indicating the level of detail of the Layer Item data, the higher the value, the higher the level of the corresponding sub-sample data and the richer the details
  • tile_enable is 1, and region_enable and scalable_enable are 0.
  • FIG. 9 is a schematic diagram of static point cloud storage based on 3D region property and Subsample according to the present embodiment.
  • the part of G-PCC point cloud data obtained by the terminal according to the spatial region is at least one G-PCC 3D Region, That is, a space area, all 3D Region information is described by 3D Region Item Property, including the number of space areas, coordinates, range, etc. Tile spatial location, range and other information are described in G-PCC Configuration Item Property, and Subsample Item Property is used to describe the spatial division of point cloud data in G-PCC Item, taking the basic package format of Single-item as an example, based on 3D Region and The static point cloud storage structure of Subsample is shown in Figure 9.
  • G-PCC Geometry Item refers to one or more property items of type 'gpca' through the item reference data box (Item Reference Box, iref) of type 'gpca'.
  • 3d_region_id indicating the 3D spatial region identifier
  • region_dx, region_dy, region_dz indicating the 3D space region x, y, z direction range
  • num_regions indicating the number of 3D space regions described in the Item Property
  • num_tiles indicating the number of G-PCC tiles contained in each 3D space
  • tile_id indicating the G-PCC tile identifier, consistent with the tile identifier in Tile Inventory
  • Region Item describes the division of space
  • FIG. 10 is a schematic diagram of static point cloud storage based on Region Item according to the present embodiment.
  • the static 3D point cloud partial access method based on multiple items stores point cloud data of different spatial regions in independent G -In the PCC Region Item, the entire spatial area division of the point cloud data is described by the 3D region item property attribute information.
  • Multiple Region Items containing geometric data are described as complete static 3D point clouds by the CompletePointCloudGroupBox, and the group type is 'cppc'.
  • the static point cloud storage structure based on Region Item is shown in Figure 10.
  • G-PCC Geometry Region Item refers to one or more attribute items of type 'gpra' through the item reference data box (Item Reference Box, iref) of type 'gpca'.
  • the 'gpcr' type represents a G-PCC Region Item, which consists of G-PCC part of the item data, and each Region Item represents a subset of independently decodable point clouds belonging to a certain spatial area.
  • Each 'gpcr' type Item should be associated with a Regin Description Item to provide spatial region description information.
  • the 'gprg' type represents the G-PCC Geometry Region Item, which is composed of G-PCC part of the item data, and each Item represents the point cloud geometric data belonging to a certain spatial area;
  • the 'gpra' type represents the G-PCC Attribute Region Item, which consists of part of the G-PCC item data, and each Item represents a type of point cloud attribute data belonging to a certain spatial area;
  • RegionDescriptionInfoBox extends FullBox('3drg',0,0) ⁇ 3DSpatialRegionStruct();
  • CompletePointCloudGroupBox is the same as that of Embodiment 2. This embodiment describes the spatial region division of the point cloud based on Region Item. In CompletePointCloudGroupBox, region_enable is 1, and tile_enable and scalable_enable are 0.
  • Example 2 Partial access to static point cloud based on LoD (level of detail)
  • This embodiment describes a method for progressively accessing a static 3D point cloud based on Level of Details (LoD), and a method for describing point cloud level information in a media file.
  • its geometric data and attribute data can be divided into multiple levels according to the geometric positions of the points, and each level is composed of a set of uniformly distributed points. The higher the level, the denser the point set, and the richer the details of the point cloud objects that can be observed; the lower the level, the sparser the point set, and the more blurry the details of the point cloud objects that can be observed.
  • the terminal obtains the point cloud compressed data, it can select the appropriate level for decoding and rendering according to the user's needs, so as to ensure the user experience and avoid the waste of decoder resources caused by rendering unnecessary details.
  • G-PCC Layer Item Store the point cloud data in multiple point cloud level items (G-PCC Layer Item) according to the different LoD levels.
  • Each Layer Item describes the level value and number of points through the level detail item property (LoD Description item property), and the terminal can Select one or more Layer Items to complete decoding, rendering, etc.
  • the terminal parsing process includes:
  • the terminal identifies the item containing static point cloud data in the metadata box according to one of the following item types:
  • Point cloud projects: gpcc, gpcg, gpca;
  • Point cloud level projects: gpcl, gplg, gpla;
  • the terminal reads the point cloud configuration item property (G-PCC Configuration Item Property) associated with the point cloud item, reads the parameter information and configuration information such as SPS, APS, GPS, etc., identifies the LoD classification level and other related parameter information, and completes the decoding device initialization;
  • the terminal determines the level detail information of the static point cloud data and the range of each level point cloud data according to one of the following methods:
  • Method 1 Read Subsample information (Subsample Item Property), where flags is 2, read the level value, number of points and data range of LoD single-level point cloud data indicated by each Sub-sample;
  • Method 2 Determine the point cloud data range, LoD level value and number of points of each level according to the point cloud level item and LoD Description Item Property;
  • the terminal calculates according to user requirements, LoD level value and point information, and determines one or more LoD levels to be decoded;
  • the terminal reads the G-PCC data of the partial level, and inputs it into the decoder to complete the decoding;
  • the renderer renders a 3D static scene containing some details.
  • This embodiment describes a method for progressively accessing a static 3D point cloud based on level details, and a method for describing point cloud level information in a media file.
  • the complete point cloud data is stored in a single Item, and the hierarchical division of the point cloud data is described by the Sub-sample item property information.
  • the multi-level static point cloud storage structure based on Subsample is shown in Figure 6.
  • the storage structure based on Subsample is similar to Figure 6, the difference is that the static point cloud access entry is G-PCC Geometry Item of type 'gpcg', which is different from G-PCC Configuration Item Property, Subsample Item Property is associated with two item properties.
  • G-PCC Geometry Item refers to one or more property items of type 'gpca' through the item reference data box (Item Reference Box, iref) of type 'gpca'.
  • FIG. 11 is a schematic diagram of the subsample division based on hierarchical information according to the present embodiment.
  • the G-PCC Subsample Item Property is as follows:
  • lod_value indicating the level of detail of the G-PCC sub-sample data, the higher the value, the higher the level of the corresponding sub-sample data and the richer the details
  • point_count indicating that the G-PCC sub-sample contains the number of points
  • Fig. 12 is a schematic diagram of static point cloud storage based on Layer Item according to this embodiment. As shown in Fig. 12, this embodiment describes another method for accessing static 3D point cloud parts based on hierarchical details. Cloud data is stored in an independent G-PCC Layer Item. The layer value of the Layer Item, the number of points included, and other information are described through the LoD Description Item Property attribute information. Multiple Layer Items containing geometric data are combined into a complete static 3D through the type 'gpcc' point cloud. Taking the basic package format of Single-item as an example, the static point cloud storage structure based on Layer Item is shown in Figure 12.
  • G-PCC Geometry Layer Item refers to one or more attribute items of type 'gpla' through an item reference data box (Item Reference Box, iref) of type 'gpca'.
  • the 'gpcl' type represents G-PCC Layer Item, which consists of G-PCC part of the item data, and each Layer Item represents a subset of independently decodable point clouds belonging to a certain level.
  • Each 'gpcl' type Item should be associated with the LoD Description Item property, providing description information such as the level value and points of the level.
  • the 'gplg' type represents the G-PCC Geometry Layer Item, which consists of part of the G-PCC item data, and each Item represents the point cloud geometric data belonging to a certain level;
  • the 'gpla' type represents the G-PCC Attribute Layer Item, which consists of part of the G-PCC item data, and each Item represents a type of point cloud attribute data belonging to a certain level;
  • RegionDescriptionInfoBox extends FullBox(",0,0) ⁇
  • lod_value indicating the level of detail of the G-PCC sub-sample data, the higher the value, the higher the level of the corresponding sub-sample data and the richer the details
  • point_count indicating that the G-PCC sub-sample contains the number of points
  • CompletePointCloudGroupBox is the same as that of Embodiment 1. This embodiment describes the hierarchical division of point clouds based on Layer Item. In CompletePointCloudGroupBox, scalable_enable is 1, and tile_enable and region_enable are 0.
  • FIG. 13 is a schematic diagram of static point cloud storage based on spatial regions and hierarchical details according to the present embodiment. Taking the basic package format of Single-item as an example, the static point cloud storage structure based on spatial regions and hierarchical details is shown in Fig. 13 , Store the point cloud data of different spatial areas in a separate G-PCC Region Item, and describe the level information of the point cloud data in each area through the Subsample Item Property attribute information.
  • the three G-PCC Region Items correspond to layers 3 and 8 respectively. , 12 layers are divided into three levels, and multiple Region Items are combined into a complete static 3D point cloud through the type 'gpcc'.
  • Example 4 Partial access to static point clouds based on creator intent
  • This embodiment also provides a method for accessing the static 3D point cloud part based on the creator's intention.
  • static 3D point clouds such as cities, streets, and large buildings
  • the content creator can access the point clouds in the 3D point cloud collection, encoding, and packaging stages.
  • Data is labeled, such as floor divisions for large buildings, room divisions, block divisions for street scenes, and more.
  • static 3D point cloud scenes can be displayed in association with other media objects, such as street scenes or different areas of a digital museum with different audio explanations, and dynamic point cloud objects such as different people corresponding to different floors in large buildings.
  • this implementation can be divided into two implementation manners:
  • Area marking that is, a combination of multiple Items
  • the static point cloud is displayed in association with the dynamic media content, that is, the combination of Item and Track.
  • FIG. 14 is a schematic diagram of region marking in static point cloud space according to this embodiment.
  • a region marking group data box (RegionMarkingGroupBox) is used to combine multiple point cloud data subsets of different regions, and provide content descriptions Information, taking the basic package format of Single-item as an example, the static point cloud storage structure that supports spatial region marking is shown in Figure 14.
  • the complete point cloud object can be divided into 5 Region Items according to different spatial regions, corresponding to two rooms respectively.
  • the scene is marked with an EntitytoGroupBox of type 'pcrm', and describes the scene information of the two rooms respectively.
  • RegionMarkingGroupBox extends EntityToGroupBox('pcrm') ⁇
  • region_type provides some item data tag types corresponding to the EntitytoGroup group
  • region_description a null-terminated UTF-8 string that provides the textual description of the point cloud's markers
  • FIG. 15 is a schematic diagram of the associated storage of static point clouds and dynamic point clouds according to the present embodiment.
  • Multiple entity playout group data boxes (MultipleEntityPlayoutGroupBox) are used to combine multiple point cloud data subsets in different regions and provide content description information.
  • MultipleEntityPlayoutGroupBox the associated storage structure of static point cloud and dynamic point cloud is shown in Figure 15.
  • the static point cloud objects contained in the Meta box can be divided into G-PCC Region Items according to different spatial areas: Region 1, Region 2.
  • G-PCC Region 1 is associated with the dynamic point cloud object in the Movie box. The associated information is described by MultipleEntityPlayoutGroupBox, indicating that G-PCC Region 1 needs to be displayed together with the point cloud data contained in track 1.
  • G-PCC Region 2 is associated with dynamic point cloud objects and audio in Movie box. The associated information is described by MultipleEntityPlayoutGroupBox, indicating that G-PCC Region 2 needs to be displayed together with the point cloud data contained in track 2 and the audio data contained in track 3.
  • FIG. 16 is a block diagram 1 of the apparatus for processing point cloud data according to this embodiment. As shown in FIG. 16 , the apparatus includes:
  • the first determining module 162 is configured to determine static geometrically encoded point cloud data including spatial region information of the 3D scene, wherein the static geometrically encoded point cloud data is represented by point cloud item data;
  • the first decoding module 164 is configured to decode part of the item data corresponding to the part of the spatial region of the 3D scene in the static geometrically encoded point cloud data;
  • the first rendering module 166 is configured to render a partial spatial region of the 3D scene according to the decoded data.
  • the point cloud item data passes the item type in the metadata data box, wherein the item type at least includes: point cloud item, point cloud block item, and point cloud space area item.
  • the first determining module includes:
  • the first determination submodule is configured to, in the case that the item type is the point cloud item, determine the corresponding one or more items according to the characteristics of the subsample items associated with the point cloud item in the metadata data box.
  • the partial item data is chunked, and the partial item data is decoded.
  • the first determination sub-module is further configured as
  • Partial item data corresponding to one or more segments is determined according to the first subsample data type and segment identifier in the subsample item properties.
  • the first determining module includes:
  • the second determination sub-module is configured to, in the case that the item type is the point cloud item, according to the sub-sample item characteristics associated with the point cloud item in the metadata data box, and the 3D space area item characteristics
  • the partial item data corresponding to one or more spatial regions is determined, and the partial item data is decoded.
  • the second determination sub-module is further configured as
  • the partial item data corresponding to one or more spatial regions is determined according to the second subsample data type and the spatial region identifier in the subsample item properties.
  • the 3D space area item properties include:
  • the number of space areas, the description information of the space area, the number of blocks contained in the space area, and the corresponding identifiers of the blocks contained in the space area are referred to as the number of space areas, the description information of the space area, the number of blocks contained in the space area, and the corresponding identifiers of the blocks contained in the space area.
  • the spatial region description information includes: a spatial region identifier, anchor point coordinates, and the length, width, and height of the spatial region.
  • the first determining module includes:
  • the first identification sub-module is configured to identify the point cloud data including the 3D static scene according to the full cluster group of the point cloud when the project type is a point cloud block project;
  • the third determination sub-module is configured to determine, according to the point cloud sub-block set in the point cloud configuration item characteristics, and one or more point cloud sub-block items in the metadata data box, the partial item data, decoding the partial item data.
  • the point cloud tile entry includes a single tile described in a point cloud tile set, wherein each tile of the one or more tiles represents an independently decodable region of the point cloud space. subset of data.
  • the first determining module includes:
  • the second identification sub-module is configured to identify the point cloud data including the 3D static scene according to the full cluster group of the point cloud when the item type is a point cloud space area item;
  • the fourth determination sub-module is set to describe item characteristics according to one or more point cloud space area items in the metadata data box, and the space area associated with the point cloud space item, and determine the corresponding one or more point cloud space areas. part of the item data, decoding the part of the item data.
  • each of the one or more point cloud spatial regions corresponds to a single spatial region described by the spatial region description item characterization.
  • the characteristics of the spatial area description item include: a spatial area identifier, anchor point coordinates, and the length, width, and height of the spatial area.
  • the point cloud full cluster group includes:
  • the first determining module includes:
  • the fifth determination submodule is configured to determine, according to the area marker group, part of the item data for indicating the marked scene when the item type is a point cloud space area item, and to decode the part of the item data.
  • the area marker group includes at least a marker type and a marker text, and the part of the point cloud items indicating the marker scene is described by the marker type and the marker text.
  • the apparatus further includes:
  • a first determining resource module configured to determine and acquire one or more media resources associated with the partial project data
  • a first decoding resource module configured to decode the one or more media resources while decoding the partial item data
  • the first playing module is configured to play the one or more media resources while rendering the partial spatial region of the 3D scene according to the decoded data.
  • the first determining resource module is further set to
  • One or more media resources associated with the partial item data are determined and acquired according to the multi-entity play group.
  • the media resources include at least: video, audio, and dynamic point cloud.
  • the point cloud item data consists of geometric item data and one or more attribute item data.
  • FIG. 17 is a block diagram 2 of the apparatus for processing point cloud data according to this embodiment. As shown in FIG. 17 , the apparatus includes:
  • the second determining module 172 is configured to determine static geometrically encoded point cloud data including hierarchical detail information of the 3D scene, wherein the static geometrically encoded point cloud data is represented by cloud item data;
  • the second decoding module 174 is configured to decode part of the item data corresponding to part of the level details of the 3D scene in the static geometrically encoded point cloud data;
  • the second rendering module 176 is configured to render some level details of the 3D scene according to the decoded data.
  • the point cloud item data is determined by the item type in the metadata data box, and the item type at least includes: point cloud item and point cloud level item.
  • the second determining module includes:
  • the sixth determination sub-module is configured to, in the case that the item type is a point cloud item, according to the characteristics of the sub-sample item associated with the point cloud item in the metadata data box, to determine the item corresponding to one or more levels. partial item data, decoding the partial item data.
  • the sixth determination sub-module is further configured as
  • the portion of item data corresponding to one or more levels is determined according to the third sub-sample data type and level value in the sub-sample item property.
  • the second determining module includes:
  • the second identification sub-module is configured to identify the point cloud data containing all the details of the 3D static scene according to the whole cluster group of the point cloud when the item type is a point cloud level item;
  • the seventh determination sub-module is configured to determine part of the item data corresponding to one or more levels according to one or more point cloud level items in the metadata data box and the level detail item properties associated with the point cloud level items, The partial item data is decoded.
  • each of the one or more levels corresponds to a single level of level detail item property description.
  • the level detail item characteristics include at least: point cloud level value, level point number.
  • the point cloud full cluster group includes:
  • the apparatus further includes:
  • a second determining resource module configured to determine and acquire one or more media resources associated with the part of the project data
  • a second decoding resource module configured to decode the one or more media resources while decoding the partial item data
  • the second playing module is configured to play the one or more media resources while rendering some level details of the 3D scene according to the decoded data.
  • the second determining resource module includes:
  • the eighth determination submodule is configured to determine and acquire one or more media resources associated with the partial item data according to the multi-entity playback group.
  • the media resources include at least: video, audio, and dynamic point cloud.
  • the point cloud item data is composed of geometric item data and one or more attribute item data, the geometric item data and attribute item data are divided into multiple levels, and each level is composed of evenly distributed A set of point sets.
  • the above modules can be implemented by software or hardware, and the latter can be implemented in the following ways, but not limited to this: the above modules are all located in the same processor; or, the above modules can be combined in any combination The forms are located in different processors.
  • Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute the steps in any one of the above method embodiments when running.
  • the above-mentioned computer-readable storage medium may include, but is not limited to, a USB flash drive, a read-only memory (Read-Only Memory, referred to as ROM for short), and a random access memory (Random Access Memory, referred to as RAM for short) , mobile hard disk, magnetic disk or CD-ROM and other media that can store computer programs.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • Embodiments of the present application further provide an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
  • the above-mentioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.
  • modules or steps of the present application can be implemented by a general-purpose computing device, and they can be centralized on a single computing device, or distributed in a network composed of multiple computing devices
  • they can be implemented in program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, can be performed in a different order than shown here.
  • the described steps, or they are respectively made into individual integrated circuit modules, or a plurality of modules or steps in them are made into a single integrated circuit module to realize.
  • the present application is not limited to any particular combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Computer Security & Cryptography (AREA)
  • Processing Or Creating Images (AREA)
  • Image Generation (AREA)

Abstract

本申请实施例提供了一种点云数据处理方法、装置、存储介质及电子装置，其中，该方法包括：确定包含3D场景的空间区域信息的静态几何编码点云数据，其中，该静态几何编码点云数据通过点云项目数据表示；解码该静态几何编码点云数据中对应该3D场景的部分空间区域的部分项目数据；根据解码后的数据渲染该3D场景的部分空间区域，可以解决相关技术中通常需要遍历完整点云数据之后才能识别出用户所需的部分点云数据，导致解析效率低、耗时长的问题，根据用户需求确定所需的部分项目数据，使得无需遍历完整的点云数据即可获取到所需的部分项目数据，提高了解码效率，缩短了解码时间，也缩短了用户的渲染等待时间。

Description

点云数据处理方法、装置、存储介质及电子装置
相关申请的交叉引用
本公开基于2020年9月30日提交的发明名称为“点云数据处理方法、装置、存储介质及电子装置”的中国专利申请202011063014.X，并且要求该专利申请的优先权，通过引用将其所公开的内容全部并入本公开。
技术领域
本申请实施例涉及通信领域，具体而言，涉及一种点云数据处理方法、装置、存储介质及电子装置。
背景技术
沉浸媒体技术中对三维世界的描述方式包括多种，除了VR全景视频外，还可以将激光雷达或者摄像机矩阵将一个三维物体直接数字化，以独立的点和面表示三维物体，即三维点云(Point Cloud)。点云是空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集。点云中的每个点至少具有三维位置信息，根据应用场景不同，还可能具有色彩、材质或其他信息。点云可以通过多个摄像头和深度传感器捕获，其中点数可以达到几千到几十亿，点和点之间没有联系、没有顺序，可以进行随意的排序，因此点云以其灵活方便的表达形式、较高的数据精度，引起了业界的广泛关注。
与其他容积媒体格式相比，点云相对容易捕获和渲染，应用场景包括：自动驾驶，实时巡检、文化遗产、6DoF沉浸式实时通信等。根据点云时间信息表示，可以将点云分为静态点云及动态点云两大类别。对于静态点云的应用场景，如城市、街道等大型静态点云，用户在应用点云数据时通常只需关注部分点云数据，而无需获取完整点云对象，例如：
1)地理信息系统等大型复杂场景点云数据，根据地理区域选择场景、随机查看；数字文化遗产支持根据视点透视选择，虚拟游览；
2)大型复杂场景、地图支持由远及近、由粗到细查看；数字文化遗产支持渐进式渲染、画面缩放、细节查看；
3)针对多个点云对象关联应用的场景，根据对象选择部分3D点云进行解码、渲染。
针对上述应用场景，用户只关注部分区域的点云数据或包含部分细节的点云数据，而现有针对静态点云的技术方案中，解码器通常需要遍历完整点云码流之后才能识别出用户所需的部分点云数据，而对于大型点云对象，例如城市场景、地图导航等，此方案解析效率低、耗时长；并且现有的静态点云存储方案中，通常以整个对象作为存储单位，不利于并行解码，灵活度较低。
针对相关技术中通常需要遍历完整点云数据之后才能识别出用户所需的部分点云数据，导致解析效率低、耗时长的问题，尚未提出解决方案。
发明内容
本申请实施例提供了一种点云数据处理方法、装置、存储介质及电子装置，以至少解决 相关技术中通常需要遍历完整点云数据之后才能识别出用户所需的部分点云数据，导致解析效率低、耗时长的问题。
根据本申请的一个实施例，提供了一种点云数据处理方法，所述方法包括：
确定包含3D场景的空间区域信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过点云项目数据表示；
解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据；
根据解码后的数据渲染所述3D场景的部分空间区域。
根据本申请的另一个实施例，还提供了一种点云数据处理方法，所述方法包括：
确定包含3D场景的层级细节信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过云项目数据表示；
解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据；
根据解码后的数据渲染所述3D场景的部分层级细节。
根据本申请的另一个实施例，还提供了一种点云数据处理装置，所述装置包括：
第一确定模块，设置为确定包含3D场景的空间区域信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过点云项目数据表示；
第一解码模块，设置为解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据；
第一渲染模块，设置为根据解码后的数据渲染所述3D场景的部分空间区域。
根据本申请的另一个实施例，还提供了一种点云数据处理装置，所述装置包括：
第二确定模块，用于确定包含3D场景的层级细节信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过云项目数据表示；
第二解码模块，用于解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据；
第二渲染模块，用于根据解码后的数据渲染所述3D场景的部分层级细节。
根据本申请的又一个实施例，还提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
根据本申请的又一个实施例，还提供了一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行上述任一项方法实施例中的步骤。
通过本申请，可以解决相关技术中通常需要遍历完整点云数据之后才能识别出用户所需的部分点云数据，导致解析效率低、耗时长的问题，根据用户需求确定所需的部分项目数据，使得无需遍历完整的点云数据即可获取到所需的部分项目数据，提高了解码效率，缩短了解码时间，也缩短了用户的渲染等待时间。
附图说明
图1是本申请实施例的点云数据处理方法的移动终端的硬件结构框图；
图2是根据本申请实施例的点云数据处理方法的流程图一；
图3是根据本申请实施例的点云数据处理方法的流程图二；
图4是根据本实施例的静态点云单项目存储结构的示意图；
图5是根据本实施例的静态点云多项目存储结构的示意图；
图6是根据本实施例的基于Subsample的静态点云存储的示意图；
图7是根据本实施例的基于空间区域的Subsample划分的示意图；
图8是根据本实施例的基于Tile Item的静态点云存储的示意图；
图9是根据本实施例的基于3D region property与Subsample的静态点云存储的示意图；
图10是根据本实施例的基于Region Item的静态点云存储的示意图；
图11是根据本实施例的基于层级信息的Subsample划分的示意图；
图12是根据本实施例的基于Layer Item的静态点云存储的示意图；
图13是根据本实施例的基于空间区域以及层级细节的静态点云存储的示意图；
图14是根据本实施例的静态点云空间区域标记的示意图；
图15是根据本实施例的静态点云与动态点云关联存储的示意图；
图16是根据本实施例的点云数据处理装置的框图一；
图17是根据本实施例的点云数据处理装置的框图二。
具体实施方式
下文中将参考附图并结合实施例来详细说明本申请的实施例。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
本申请实施例中所提供的方法实施例可以在移动终端、计算机终端或者类似的运算装置中执行。以运行在移动终端上为例，图1是本申请实施例的点云数据处理方法的移动终端的硬件结构框图，如图1所示，移动终端可以包括一个或多个(图1中仅示出一个)处理器102(处理器102可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)和用于存储数据的存储器104，其中，上述移动终端还可以包括用于通信功能的传输设备106以及输入输出设备108。本领域普通技术人员可以理解，图1所示的结构仅为示意，其并不对上述移动终端的结构造成限定。例如，移动终端还可包括比图1中所示更多或者更少的组件，或者具有与图1所示不同的配置。
存储器104可用于存储计算机程序，例如，应用软件的软件程序以及模块，如本申请实施例中的数据处理方法对应的计算机程序，处理器102通过运行存储在存储器104内的计算机程序，从而执行各种功能应用以及数据处理，即实现上述的方法。存储器104可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器104可进一步包括相对于处理器102远程设置的存储器，这些远程存储器可以通过网络连接至移动终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输装置106用于经由一个网络接收或者发送数据。上述的网络具体实例可包括移动终端的通信供应商提供的无线网络。在一个实例中，传输装置106包括一个网络适配器(Network Interface Controller，简称为NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置106可以为射频(Radio Frequency，简称为RF)模块，其用于通过无线方式与互联网进行通讯。
在本实施例中提供了一种运行于上述移动终端或网络架构的点云数据处理方法，图2是根据本申请实施例的点云数据处理方法的流程图一，如图2所示，该流程包括如下步骤：
步骤S202，确定包含3D场景的空间区域信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过点云项目数据表示；
本实施例中，所述点云项目数据由几何项目数据、一种或多种属性项目数据构成，所述几何项目数据和属性项目数据均划分为多个层级，每个层级的点云项目数据由分布均匀的一组点集构成。
步骤S204，解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据；
步骤S206，根据解码后的数据渲染所述3D场景的部分空间区域。
通过上述步骤S202至S206，可以解决相关技术中通常需要遍历完整点云数据之后才能识别出用户所需的部分点云数据，导致解析效率低、耗时长的问题，根据用户需求确定所需的部分项目数据，使得无需遍历完整的点云数据即可获取到所需的部分项目数据，提高了解码效率，缩短了解码时间，也缩短了用户的渲染等待时间。
本实施例中，所述点云项目数据通过元信息数据盒中的项目类型确定，其中，所述项目类型至少包括：点云项目、点云分块项目、点云空间区域项目。
在一实施例中，上述步骤S204具体可以包括：在所述项目类型为所述点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性确定对应于一个或多个分块的所述部分项目数据，进一步的，根据所述子样本项目特性中第一子样本数据类型和分块标识符确定对应于一个或多个分块的部分项目数据，解码所述部分项目数据。
在一实施例中，上述步骤S204具体可以包括：在所述项目类型为所述点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性，以及3D空间区域项目特性确定对应于一个或多个空间区域的所述部分项目数据，进一步的，根据子样本项目特性中第二子样本数据类型和空间区域标识符确定对应于一个或多个空间区域的所述部分项目数据，其中，所述3D空间区域项目特性包括：空间区域数量，空间区域描述信息，空间区域包含的分块数量，空间区域包含的分块对应标识符，所述空间区域描述信息包括：空间区域标识符，锚点坐标，空间区域的长、宽、高，解码所述部分项目数据。
在一实施例中，上述步骤S204具体可以包括：在所述项目类型为点云分块项目的情况下，根据点云全集群组识别包含所述3D静态场景的点云数据；根据点云配置项目特性中的点云分块集、所述元信息数据盒中一个或多个点云分块项目确定对应于一个或多个分块的所述部分项目数据，解码所述部分项目数据。在一实施例中，所述点云分块项目包含点云分块集中描述的单个分块，其中，所述一个或多个分块中的每个分块表示点云空间区域中可独立解码的数据子集。
在一实施例中，上述步骤S204具体可以包括：在所述项目类型为点云空间区域项目的情况下，根据点云全集群组识别包含所述3D静态场景的点云数据；根据所述元信息数据盒中一个或多个点云空间区域项目，与点云空间项目关联的空间区域描述项目特性，确定对应于一个或多个点云空间区域的部分项目数据，解码所述部分项目数据，其中，所述一个或多个点云空间区域中的每个点云空间区域对应于空间区域描述项目特性描述的单个空间区域。在一实施例中，所述空间区域描述项目特性包括：空间区域标识符，锚点坐标，空间区域的长、 宽、高。
在一实施例中，所述点云全集群组包括：完整点云对象中最多划分的部分访问的点云子集的数量、点云子集的类型、点云子集的标识符，其中，所述点云子集包括：根据分块划分的点云子集、根据空间区域划分的点云子集。
在一实施例中，上述步骤S202具体可以包括：在所述项目类型为点云空间区域项目的情况下，根据区域标记群组确定用于指示标记场景的部分项目数据，解码所述部分项目数据。其中，所述区域标记群组至少包括标记类型、标记文本，所述指示标记场景的部分点云项目通过所述标记类型、所述标记文本来描述。
在一实施例中，确定并获取与所述部分项目数据相关联的一个或多个媒体资源，具体的，可以根据多实体播放群组确定并获取与所述部分项目数据相关联的一个或多个媒体资源；在解码所述部分项目数据同时，对所述一个或多个媒体资源进行解码；在根据解码后的数据渲染所述3D场景的部分空间区域的同时，对所述一个或多个媒体资源进行播放。在一实施例中，所述媒体资源至少包括：视频、音频、动态点云。
根据本申请的另一个实施例，还提供了一种点云数据处理方法，图3是根据本申请实施例的点云数据处理方法的流程图二，如图3所示，该流程包括如下步骤：
步骤S302，确定包含3D场景的层级细节信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过云项目数据表示；
本实施例中，所述点云项目数据由几何项目数据、一种或多种属性项目数据构成，所述几何项目数据和属性项目数据均划分为多个层级，每个层级由分布均匀的一组点集构成。
步骤S304，解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据；
步骤S306，根据解码后的数据渲染所述3D场景的部分层级细节。
通过上述步骤S302至S306，可以解决相关技术中通常需要遍历完整点云数据之后才能识别出用户所需的部分点云数据，导致解析效率低、耗时长的问题，根据用户需求确定所需的部分项目数据，使得无需遍历完整的点云数据即可获取到所需的部分项目数据，提高了解码效率，缩短了解码时间，也缩短了用户的渲染等待时间。
本实施例中，所述点云项目数据通过元信息数据盒中的项目类型确定，所述项目类型至少包括：点云项目、点云层级项目。
在一实施例中，上述步骤S304具体可以包括：在所述项目类型为点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性确定对应于一个或多个层级的部分项目数据，进一步的，可以根据子样本项目特性中第三子样本数据类型和层级值确定对应于一个或多个层级的部分项目数据，解码所述部分项目数据。
在一实施例中，上述步骤S304具体可以包括：在所述项目类型为点云层级项目的情况下，根据点云全集群组识别包含所述3D静态场景全部细节的点云数据；根据所述元信息数据盒中一个或多个点云层级项目，以及与点云层级项目关联的层级细节项目特性确定对应于一个或多个层级的部分项目数据，解码所述部分项目数据，其中，所述一个或多个层级中的每个层级对应于层级细节项目特性描述的单个层级；所述层级细节项目特性至少包括：点云层级值，层级点数。
在一实施例中，所述点云全集群组包括：完整点云对象中最多划分的部分访问的点云子集的数量、点云子集的类型、点云子集的标识符，其中，所述点云子集包括根据层级细节划分的点云子集。
在一实施例中，确定并获取与所述部分项目数据相关联的一个或多个媒体资源，具体的，可以根据多实体播放群组确定并获取与所述部分项目数据相关联的一个或多个媒体资源；在解码所述部分项目数据同时，对所述一个或多个媒体资源进行解码；在根据解码后的数据渲染所述3D场景的部分层级细节的同时，对所述一个或多个媒体资源进行播放。
本实施例中，所述媒体资源至少包括：视频、音频、动态点云。
本实施例，能够利用ISO International Organization for Standardization，国际标准化组织)基本媒体文件格式对不包含时间信息的3D点云压缩数据进行描述，支持3D点云空间区域信息及层级信息描述，进而提供点云数据的部分访问机制，以使解码器能够根据用户空间位置、观看方向等信息选取部分3D点云数据进行解析、解码、渲染，提高静态点云处理效率。3D点云的部分访问至少包括以下两种方式：基于空间区域的部分访问；基于层级细节的渐进式访问。本实施例中利用ISOBMFF媒体文件格式中的项目(Item)以及相关数据盒存储静态3D点云数据及部分访问所需的元数据信息，能够满足多种场景下3D点云部分访问的需要，具体语义在实施例中进一步描述。
本实施例实现方式是基于ISO(基本媒体文件格式将静态3D点云的空间位置信息、层级信息、关联关系信息存储在媒体文件中。基本媒体文件格式可参照ISO/IEC JTC1/SC29/WG11运动图像专家组(Moving Picture Experts Group，简称MPEG)制定的MPEG-4 Part 12 ISO Base Media File Format来操作。其中静态点云压缩数据格式可参照ISO/IEC JTC1/SC29/WG11运动图像专家组(MPEG)制定的MPEG-I Part 9:Geometry-based Point Cloud Compress ion(G-PCC)基于几何编码的点云压缩技术来操作。
ISOBMFF媒体文件中所有数据都装在数据盒(box)中，并可以在数据盒头部描述其类型和大小；如果一个box支持嵌套(一个box中包含另一个子box)，则将该数据盒称为支持嵌套的数据盒(container box)。ISOBMFF媒体文件中的静态数据(例如图像)通常以”item”格式存储，元信息数据盒(Meta box)可以存放项目的描述信息、元数据，也可以包含具体的载荷，元信息数据盒中支持多个静态媒体对象的集成存储，例如图集(image collection)。
静态点云数据支持两种基本封装方式：
1)单项目(Single-item)：图4是根据本实施例的静态点云单项目存储结构的示意图，如图4所示，以单个项目封装G-PCC点云的全部几何数据和属性数据，序列参数集(Sequence Parameter Set，简称为SPS)、属性参数集(Attribute Parameter Set，APS)、几何参数集(Geometry Parameter Set，简称为GPS)等参数信息以及分块集(Tile Inventory)等通用元数据信息在项目特性(Item Property)中描述，类型为“gpcc”，与点云项目(G-PCC Item)关联。
2)多项目(Multiple-item)：图5是根据本实施例的静态点云多项目存储结构的示意图，如图5所示，G-PCC点云的几何数据及属性数据分别封装在多个Item中，即完整点云包含一个点云几何项目(G-PCC Geometry Item)与一个或多个点云属性项目(G-PCC Attribute Item)，如图2。点云SPS、APS、GPS等参数信息以及Tile Inventory等通用元数据信息在Item Property中描述，与多个几何数据、属性数据等多个Item关联，点云的几何项目通过类型 为‘gpcg’的项目引用数据盒(Item Reference Box，iref)引用一个或多个属性项目。
本实施例均支持Single-item、Multiple-item两种静态点云基本封装格式，实施例进一步详细说明中以Single-item封装格式为例进行扩展定义，仅仅用以解释本申请，并不用于限定本申请
示例一：基于空间区域的静态点云部分访问
本实施例描述基于空间区域的静态3D点云部分访问方法，以及点云空间区域信息在媒体文件的描述方法。对于支持基于空间区域部分访问的静态G-PCC点云压缩数据，每个空间区域对应的点云数据均支持独立解码、渲染。
在本实施例场景中，支持四种实施方式：
将完整的点云数据存储在单个Item中，通过分块集提供的包围盒(Bonding Box)信息、子样本项目特性(Sub-sample item property)描述点云数据的Tile划分，终端可以根据用户需求灵活选择一个或多个分块子样本(Tile Sub-sample)完成解码、渲染等操作；
将点云数据根据空间区域不同存储在多个点云分块项目(G-PCC Tile Item)中，与分块集中的区域划分一一对应，终端可以选择一个或多个分块项目完成解码、渲染等。
将完整的点云数据存储在单个Item中，通过3D空间区域项目特性(3D region item property)、子样本项目特性描述点云数据的3D空间区域划分，终端可以选择一个或多个区域子样本(Region Sub-sample)完成解码、渲染等操作；
将点云数据根据空间区域不同存储在多个点云空间区域项目(G-PCC Region Item)中，每个区域项目通过空间区域项目特性(Region Description item property)描述空间区域范围，终端可以选择一个或多个区域项目完成解码、渲染等。
终端解析流程包括：
1)终端根据以下项目类型之一识别元信息数据盒中的包含静态点云数据的项目：
点云项目：gpcc、gpcg、gpca；
点云分块项目：gpct、gptg、gpta；
点云空间区域项目：gpcr、gprg、gpra；
2)终端读取与点云项目关联的点云配置项目特性(G-PCC Configuration Item Property)，读取SPS、APS、GPS、Tile Inventory等参数信息、配置信息，完成解码器初始化；
3)终端根据以下方式之一确定静态点云数据的空间区域信息及数据范围：
方式1：读取子样本项目特性(Subsample Item Property)，若flags为,1，且data_type为0，读取每个Sub-sample指示的G-PCC Tile数据范围及其tile_id；
方式2：根据点云分块项目确定G-PCC Tile数据范围及其tile_id；
方式3：读取3D空间区域项目特性(3D Region Item Property)，解析各个3D空间区域坐标、范围等信息，读取子样本项目特性(Subsample Item Property)，若flags为1，且data_type为1，读取每个Sub-sample指示的G-PCC 3D region数据范围及其region_id；
方式4：根据点云空间区域项目确定G-PCC 3D region数据范围及其region_id；
4)终端根据用户需求与空间区域信息进行计算，确定待解码的一个或多个tile或3D region；
5)终端读取部分G-PCC数据，输入到解码器中完成解码；
6)渲染器渲染3D场景的部分空间区域。
方式1：Subsample描述空间区域划分
图6是根据本实施例的基于Subsample的静态点云存储的示意图，如图6所示，将完整的点云数据存储在单个Item中，通过Sub-sample item property属性信息描述点云数据的空间区域划分。终端根据空间区域获取的部分G-PCC点云数据最小为一个G-PCC Tile。Tile空间位置、范围等信息在G-PCC Configuration Item Property中描述，同时利用Subsample Item Property描述G-PCC Item中点云数据的空间区域划分，以Single-item基本封装格式为例，基于Subsample的存储结构如图6所示。
对于Multiple-item基本封装格式，基于Subsample的存储结构与图6相似，不同之处在于静态点云访问入口为类型‘gpcg’的G-PCC Geometry Item，并与G-PCC Configuration Item Property、Subsample Item Property两个项目特性关联，G-PCC Geometry Item通过类型为‘gpca’的项目引用数据盒(Item Reference Box，iref)引用一个或多个类型为‘gpca’的G-PCC Attribute Item属性项目，每个属性项目与一个Subsample Item Property关联。
图7是根据本实施例的基于空间区域的Subsample划分的示意图，如图7所示，G-PCC Subsample Item Property如下：
Figure PCTCN2021119964-appb-000001
Figure PCTCN2021119964-appb-000002
当Subsample Item Property指示G-PCC Item空间区域划分时，flags为1，codec_specific_parameters扩展语法如下：
Figure PCTCN2021119964-appb-000003
语义：
data_type，指示单个sub-sample对应的空间区域类型，data_type为0表示sub-sample对应G-PCC tile，data_type为0表示sub-sample对应3D region，即多个G-PCC tile；
tile_id，指示sub-sample对应的G-PCC tile标识符，与Tile Inventory中tile标识符一致；
3D_region_id，指示sub-sample对应的G-PCC 3D region标识符，与3D Region Item Property中标识符一致；
方式2：Tile Item描述空间区域划分
图8是根据本实施例的基于Tile Item的静态点云存储的示意图，如图8所示，本实施例描述基于多Item的静态3D点云部分访问方法，将不同空间区域的点云数据存储在独立的G-PCC Tile Item中，多个Region Item通过点云全集群组数据盒(CompletePointCloudGroupBox)描述为完整静态3D点云，群组类型为‘cppc’，同时群组中描述包含几何数据的Tile Item与Tile Inventory中Tile_id的映射关系。以Single-item基本封装格式为例，基于Tile item的静态点云存储结构如图8所示。
对于Multiple-item基本封装格式，基于Tile Item的存储结构与图8相似，不同之处在于静态点云访问入口为类型‘gptg’的G-PCC Geometry Tile Item，并与G-PCC Configuration Item Property项目特性关联，G-PCC Geometry Tile Item通过类型为‘gpca’的项目引用数据盒(Item Reference Box，iref)引用一个或多个类型为‘gpta’的属性项目。
G-PCC Tile Item
‘gpct’类型表示G-PCC Tile Item，由G-PCC部分项目数据组成，每个Tile Item表示属于某一分块的可独立解码点云子集，与Tile Inventory中的Tile描述一一对应。
‘gptg’类型表示G-PCC Geometry Tile Item，由G-PCC部分项目数据组成，每个Item表示属于某一分块的点云几何数据；
‘gpta’类型表示G-PCC Attribute Tile Item，由G-PCC部分项目数据组成，每个Item表示属于某一分块的一类点云属性数据；
Figure PCTCN2021119964-appb-000004
partial_gpcc_num，表示完整点云对象中最多可以划分的部分访问子集数量；
tile_enable，值为1表示此群组描述Tile Item的组合关系，否则取值为0；
region_enable，值为1表示此群组描述Region Item的组合关系，否则取值为0；
scalable_enable，值为1表示此群组描述Layer Item的组合关系，否则取值为0；
entry_id，表示某一子集G-PCC Item(Tile Item、Region Item或Layer Item)在EntityToGroupBox中的索引；
tile_id，描述Tile Item对应的tile_id；
3DSpatialRegionStruct()，描述Region Item对应的空间区域信息；
lod_value，指示Layer Item数据的层级细节高低，取值越高代表对应sub-sample数据层级越高，细节越丰富；
本实施方式是基于Tile Item描述点云的空间区域划分，CompletePointCloudGroupBox中tile_enable为1，region_enable、scalable_enable为0。
方式3：3D region property与Subsample描述空间区域划分
图9是根据本实施例的基于3D region property与Subsample的静态点云存储的示意图，如图9所示，终端根据空间区域获取的部分G-PCC点云数据最小为一个G-PCC 3D Region，即一个空间区域，全部的3D Region信息通过3D Region Item Property描述，包括空间区 域数量、坐标、范围等。Tile空间位置、范围等信息在G-PCC Configuration Item Property中描述，同时利用Subsample Item Property描述G-PCC Item中点云数据的空间区域划分，以Single-item基本封装格式为例，基于3D Region与Subsample的静态点云存储结构如图9所示。
对于Multiple-item基本封装格式，基于Subsample的存储结构与图9相似，不同之处在于静态点云访问入口为类型‘gpcg’的G-PCC Geometry Item，并与G-PCC Configuration Item Property、Subsample Item Property、3D Region Item Property三个项目特性关联，G-PCC Geometry Item通过类型为‘gpca’的项目引用数据盒(Item Reference Box，iref)引用一个或多个类型为‘gpca’的属性项目。
Figure PCTCN2021119964-appb-000005
语义：
3d_region_id，指示3D空间区域标识符；
anchor_x、anchor_y、anchor_z，指示3D空间区域原点坐标；
region_dx、region_dy、region_dz，指示3D空间区域x，y，z方向范围；
num_regions，指示Item Property中描述的3D空间区域数量；
num_tiles，指示每个3D空间中包含的G-PCC tile数量；
tile_id，指示G-PCC tile标识符，与Tile Inventory中tile标识符一致；
方式4：Region Item描述空间区域划分
图10是根据本实施例的基于Region Item的静态点云存储的示意图，如图10所示，基于多Item的静态3D点云部分访问方法，将不同空间区域的点云数据存储在独立的G-PCC Region Item中，通过3D region item property属性信息描述点云数据的全部空间区域划分，多个包含几何数据的Region Item通过CompletePointCloudGroupBox描述为完整静态3D点云，群组类型为‘cppc’。以Single-item基本封装格式为例，基于Region Item的静态点云存储结构如图10所示。
对于Multiple-item基本封装格式，基于Region Item的存储结构与图10相似，不同之处在于静态点云访问入口为类型‘gprg’的G-PCC Geometry Region Item，并与G-PCC Configuration Item Property项目特性关联，G-PCC Geometry Region Item通过类型为‘gpca’的项目引用数据盒(Item Reference Box，iref)引用一个或多个类型为‘gpra’的属性项目。
G-PCC Region Item
‘gpcr’类型表示G-PCC Region Item，由G-PCC部分项目数据组成，每个Region Item表示属于某一空间区域的可独立解码点云子集。每个’gpcr’类型的Item都应关联Regin Description Item，提供空间区域描述信息。
‘gprg’类型表示G-PCC Geometry Region Item，由G-PCC部分项目数据组成，每个Item表示属于某一空间区域的点云几何数据；
‘gpra’类型表示G-PCC Attribute Region Item，由G-PCC部分项目数据组成，每个Item表示属于某一空间区域的一类点云属性数据；
Region Description Item Property
Box type:'rdip'
Property type:Descriptive item property
Container:ItemPropertyContainerBox
Mandatory(per item):No
Quantity(per item):Zero or one for a G-PCC Region item
aligned(8)class RegionDescriptionInfoBox extends FullBox('3drg',0,0){3DSpatialRegionStruct()；
}
CompletePointCloudGroupBox
CompletePointCloudGroupBox定义同实施方式2，本实施方式是基于Region Item描述点云的空间区域划分，CompletePointCloudGroupBox中region_enable为1，tile_enable、scalable_enable为0。
示例二：基于LoD(层级细节)的静态点云部分访问
本实施例描述基于层级细节(Level of Details，LoD)的静态3D点云渐进访问方法，以及点云层级信息在媒体文件的描述方法。对于支持渐进访问的静态G-PCC点云压缩数据，其几何数据和属性数据都可以根据点的几何位置划分为多个层级，每个层级都由分布均匀的一组点集构成。层级越高，点集越稠密，能观察到的点云对象细节就越丰富；层级越低，点集越稀疏，能观察到的点云对象细节较模糊。终端获取点云压缩数据后，可以根据用户需求选择适当层级进行解码、渲染，从而在保证用户体验的同时，避免了因渲染不必要的细节而造成的解码器资源浪费。
在本实施例场景中，支持两种实施方式：
1)将完整的点云数据存储在单个Item中，通过SPS、APS参数集中描述的LoD相关参数以及Sub-sample item property属性信息描述点云数据的层级划分，终端可以根据用户需求灵活选择一个或多个LoD Sub-sample完成解码、渲染等操作；
2)将点云数据根据LoD层级不同存储在多个点云层级项目(G-PCC Layer Item)中，每个Layer Item通过层级细节项目特性(LoD Description item property)描述层级值、点数，终端可以选择一个或多个Layer Item完成解码、渲染等。
终端解析流程包括：
1)终端根据以下项目类型之一识别元信息数据盒中包含静态点云数据的项目：
点云项目：gpcc、gpcg、gpca；
点云层级项目：gpcl、gplg、gpla；
2)终端读取与点云项目关联的点云配置项目特性(G-PCC Configuration Item Property)，读取SPS、APS、GPS等参数信息、配置信息，识别LoD划分层级等相关参数信息，完成解码器初始化；
3)终端根据以下方式之一确定静态点云数据的层级细节信息以及每个层级点云数据的范围：
方式1：读取Subsample信息(Subsample Item Property)，其中flags为2，读取每个Sub-sample指示的LoD单个层级点云数据的层级值、点数以及数据范围；
方式2：根据点云层级项目以及LoD Description Item Property确定每个层级的点云数据范围以及LoD层级值、点数；
4)终端根据用户需求、LoD层级值和点数信息进行计算，确定待解码的一个或多个LoD层级；
5)终端读取部分层级的G-PCC数据，输入到解码器中完成解码；
6)渲染器渲染包含部分细节的3D静态场景。
方式1：Subsample描述LoD层级划分
本实施例描述基于层级细节的静态3D点云渐进访问方法，以及点云层级信息在媒体文件的描述方法。对于支持渐进访问的静态G-PCC点云压缩数据，其完整的点云数据存储在单个Item中，通过Sub-sample item property属性信息描述点云数据的层级划分。以Single-item基本封装格式为例，基于Subsample的多层级静态点云存储结构如图6所示。
对于Multiple-item基本封装格式，基于Subsample的存储结构与图6相似，不同之处在于静态点云访问入口为类型‘gpcg’的G-PCC Geometry Item，并与G-PCC Configuration Item Property、Subsample Item Property两个项目特性关联，G-PCC Geometry Item通过 类型为‘gpca’的项目引用数据盒(Item Reference Box，iref)引用一个或多个类型为‘gpca’的属性项目。
图11是根据本实施例的基于层级信息的Subsample划分的示意图，如图11所示，G-PCC Subsample Item Property如下：
当Subsample Item Property指示G-PCC Item空间区域划分时，flags为2，codec_specific_parameters扩展语法如下：
Figure PCTCN2021119964-appb-000006
语义：
lod_value，指示G-PCC sub-sample数据的层级细节高低，取值越高代表对应sub-sample数据层级越高，细节越丰富；
point_count，指示G-PCC sub-sample包含点数；
方式2：Layer Item描述LoD层级划分
图12是根据本实施例的基于Layer Item的静态点云存储的示意图，如图12所示，本实施例描述另一种基于层级细节的静态3D点云部分访问方法，将不同空间区域的点云数据存储在独立的G-PCC Layer Item中，通过LoD Description Item Property属性信息描述Layer Item的层级值、包含点数等信息，多个包含几何数据的Layer Item通过类型‘gpcc’组合成为完整静态3D点云。以Single-item基本封装格式为例，基于Layer Item的静态点云存储结构如图12所示。
对于Multiple-item基本封装格式，基于Layer Item的存储结构与图12相似，不同之处在于静态点云访问入口为类型‘gplg’的G-PCC Geometry Layer Item，并与G-PCC Configuration Item Property项目特性关联，G-PCC Geometry Layer Item通过类型为‘gpca’的项目引用数据盒(Item Reference Box，iref)引用一个或多个类型为‘gpla’的属性项目。
G-PCC Layer Item
‘gpcl’类型表示G-PCC Layer Item，由G-PCC部分项目数据组成，每个Layer Item表示属于某一层级的可独立解码点云子集。每个‘gpcl’类型的Item都应关联LoD Description Item property，提供该层级的层级值、点数等描述信息。
‘gplg’类型表示G-PCC Geometry Layer Item，由G-PCC部分项目数据组成，每个Item表示属于某一层级的点云几何数据；
‘gpla’类型表示G-PCC Attribute Layer Item，由G-PCC部分项目数据组成，每个Item表示属于某一层级的一类点云属性数据；
LoD Description Item Property
Box type:'lodd'
Property type:Descriptive item property
Container:ItemPropertyContainerBox
Mandatory(per item):No
Quantity(per item):Zero or one for a G-PCC Layer item
aligned(8)class RegionDescriptionInfoBox extends FullBox(”,0,0){
unsigned int(8)lod_value；
unsigned int(24)point_count；
}
语义：
lod_value，指示G-PCC sub-sample数据的层级细节高低，取值越高代表对应sub-sample数据层级越高，细节越丰富；
point_count，指示G-PCC sub-sample包含点数；
CompletePointCloudGroupBox
CompletePointCloudGroupBox定义同实施例1，本实施方式是基于Layer Item描述点云的层级划分，CompletePointCloudGroupBox中scalable_enable为1，tile_enable、region_enable为0。
示例三：空间区域及LoD结合的部分访问
本实施例描述基于空间区域以及层级细节的静态3D点云部分访问方法，其中不同空间区域根据点云密集程度不同可以划分为数量不等的细节层级。图13是根据本实施例的基于空间区域以及层级细节的静态点云存储的示意图，以Single-item基本封装格式为例，基于空间区域以及层级细节的静态点云存储结构如图13所示，将不同空间区域的点云数据存储在独立的G-PCC Region Item中，通过Subsample Item Property属性信息描述每个区域点云数据的层级信息，三个G-PCC Region Item分别对应3层、8层、12层三种层级划分，多个Region Item通过类型‘gpcc’组合成为完整静态3D点云。
示例四：基于创作者意图的静态点云部分访问
本实施例还提供基于创作者意图的静态3D点云部分访问方法，对于城市、街道、大型建筑等静态3D点云，内容创作者可以在3D点云采集、编码、封装阶段对其中的点云数据进行标记，如大型建筑的楼层划分、房间划分，街道场景的街区划分等等。此外，静态3D点云场景可以与其他媒体对象关联展示，例如街道场景或数字博物馆的不同区域提供不同音频讲解，大型建筑中不同楼层对应不同的人物等动态点云对象。
针对以上两种场景，本实施可分为两种实施方式：
区域标记，即多个Item组合；
静态点云与动态媒体内容关联展示，即Item与轨道(Track)组合。
方式1：区域标记
图14是根据本实施例的静态点云空间区域标记的示意图，本实施方式中，通过区域标记群组数据盒(RegionMarkingGroupBox)将多个不同区域的点云数据子集进行组合，并提供内容描述信息，以Single-item基本封装格式为例，支持空间区域标记的静态点云存储结构如图14所示，完整的点云对象可以根据空间区域不同划分为5个Region Item，分别对应两个房间场景，通过类型为‘pcrm’的实体群组数据盒(EntitytoGroupBox)进行标记，并分别 描述两个房间场景信息。
RegionMarkingGroupBox
Box Types:'pcrm'
Container:GroupsListBox
Mandatory:No
Quantity:Zero or more
aligned(8)class RegionMarkingGroupBox extends EntityToGroupBox('pcrm'){
unsigned int(8)region_type；
string region_description；
}
语义：
region_type，提供EntitytoGroup群组对应的部分项目数据标记类型；
region_description，以空字符结尾的UTF-8字符串，提供点云的标记的文本描述信息；
方式2：静态点云与动态点云共同渲染
图15是根据本实施例的静态点云与动态点云关联存储的示意图，通过多实体播放群组数据盒(MultipleEntityPlayoutGroupBox)将多个不同区域的点云数据子集进行组合，并提供内容描述信息，以Single-item基本封装格式为例，静态点云与动态点云关联存储结构如图15所示。
Meta box中包含的静态点云对象可以根据空间区域不同划分为个G-PCC Region Item：Region 1、Region 2。G-PCC Region 1与电影数据盒(Movie box)中动态点云对象关联，通过MultipleEntityPlayoutGroupBox描述关联信息，表示G-PCC Region 1需要与track 1包含的点云数据共同展示。G-PCC Region 2与Movie box中动态点云对象、音频关联，通过MultipleEntityPlayoutGroupBox描述关联信息，表示G-PCC Region 2需要与track 2包含的点云数据、track 3包含的音频数据共同展示。
MultipleEntityPlayoutGroupBox
Box Types:'mepl'
Container:GroupsListBox
Mandatory:No
Quantity:Zero or more
aligned(8)class MultipleEntityPlayoutGroupBox extends EntityToGroupBox('mepl'){
}
根据本申请的另一个实施例，还提供了一种点云数据处理装置，图16是根据本实施例的点云数据处理装置的框图一，如图16所示，所述装置包括：
第一确定模块162，设置为确定包含3D场景的空间区域信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过点云项目数据表示；
第一解码模块164，设置为解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据；
第一渲染模块166，设置为根据解码后的数据渲染所述3D场景的部分空间区域。
在一实施例中，所述点云项目数据通过元信息数据盒中的项目类型，其中，所述项目类型至少包括：点云项目、点云分块项目、点云空间区域项目。
在一实施例中，所述第一确定模块包括：
第一确定子模块，设置为在所述项目类型为所述点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性确定对应于一个或多个分块的所述部分项目数据，解码所述部分项目数据。
在一实施例中，所述第一确定子模块，还设置为
根据所述子样本项目特性中第一子样本数据类型和分块标识符确定对应于一个或多个分块的部分项目数据。
在一实施例中，所述第一确定模块包括：
第二确定子模块，设置为在所述项目类型为所述点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性，以及3D空间区域项目特性确定对应于一个或多个空间区域的所述部分项目数据，解码所述部分项目数据。
在一实施例中，所述第二确定子模块，还设置为
根据子样本项目特性中第二子样本数据类型和空间区域标识符确定对应于一个或多个空间区域的所述部分项目数据。
在一实施例中，所述3D空间区域项目特性包括：
空间区域数量，空间区域描述信息，空间区域包含的分块数量，空间区域包含的分块对应标识符。
在一实施例中，所述空间区域描述信息包括：空间区域标识符，锚点坐标，空间区域的长、宽、高。
在一实施例中，所述第一确定模块包括：
第一识别子模块，设置为在所述项目类型为点云分块项目的情况下，根据点云全集群组识别包含所述3D静态场景的点云数据；
第三确定子模块，设置为根据点云配置项目特性中的点云分块集、所述元信息数据盒中一个或多个点云分块项目确定对应于一个或多个分块的所述部分项目数据，解码所述部分项目数据。
在一实施例中，所述点云分块项目包含点云分块集中描述的单个分块，其中，所述一个或多个分块中的每个分块表示点云空间区域中可独立解码的数据子集。
在一实施例中，所述第一确定模块包括：
第二识别子模块，设置为在所述项目类型为点云空间区域项目的情况下，根据点云全集群组识别包含所述3D静态场景的点云数据；
第四确定子模块，设置为根据所述元信息数据盒中一个或多个点云空间区域项目，与点云空间项目关联的空间区域描述项目特性，确定对应于一个或多个点云空间区域的部分项目数据，解码所述部分项目数据。
在一实施例中，所述一个或多个点云空间区域中的每个点云空间区域对应于空间区域描 述项目特性描述的单个空间区域。
在一实施例中，所述空间区域描述项目特性包括：空间区域标识符，锚点坐标，空间区域的长、宽、高。
在一实施例中，所述点云全集群组包括：
完整点云对象中最多划分的部分访问的点云子集的数量、点云子集的类型、点云子集的标识符，其中，所述点云子集包括：根据分块划分的点云子集、根据空间区域划分的点云子集。
在一实施例中，所述第一确定模块包括：
第五确定子模块，设置为在所述项目类型为点云空间区域项目的情况下，根据区域标记群组确定用于指示标记场景的部分项目数据，解码所述部分项目数据。
在一实施例中，所述区域标记群组至少包括标记类型、标记文本，所述指示标记场景的部分点云项目通过所述标记类型、所述标记文本来描述。
在一实施例中，所述装置还包括：
第一确定资源模块，设置为确定并获取与所述部分项目数据相关联的一个或多个媒体资源；
第一解码资源模块，设置为在解码所述部分项目数据同时，对所述一个或多个媒体资源进行解码；
第一播放模块，设置为在根据解码后的数据渲染所述3D场景的部分空间区域的同时，对所述一个或多个媒体资源进行播放。
在一实施例中，所述第一确定资源模块，还设置为
根据多实体播放群组确定并获取与所述部分项目数据相关联的一个或多个媒体资源。
在一实施例中，所述媒体资源至少包括：视频、音频、动态点云。
在一实施例中，所述点云项目数据由几何项目数据、一种或多种属性项目数据构成。
根据本申请的另一个实施例，还提供了一种点云数据处理装置，图17是根据本实施例的点云数据处理装置的框图二，如图17所示，所述装置包括：
第二确定模块172，设置为确定包含3D场景的层级细节信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过云项目数据表示；
第二解码模块174，设置为解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据；
第二渲染模块176，设置为根据解码后的数据渲染所述3D场景的部分层级细节。
在一实施例中，所述点云项目数据通过元信息数据盒中的项目类型确定，所述项目类型至少包括：点云项目、点云层级项目。
在一实施例中，所述第二确定模块包括：
第六确定子模块，设置为在所述项目类型为点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性确定对应于一个或多个层级的部分项目数据，解码所述部分项目数据。
在一实施例中，所述第六确定子模块，还设置为
根据子样本项目特性中第三子样本数据类型和层级值确定对应于一个或多个层级的部分 项目数据。
在一实施例中，所述第二确定模块包括：
第二识别子模块，设置为在所述项目类型为点云层级项目的情况下，根据点云全集群组识别包含所述3D静态场景全部细节的点云数据；
第七确定子模块，设置为根据所述元信息数据盒中一个或多个点云层级项目，以及与点云层级项目关联的层级细节项目特性确定对应于一个或多个层级的部分项目数据，解码所述部分项目数据。
在一实施例中，所述一个或多个层级中的每个层级对应于层级细节项目特性描述的单个层级。
在一实施例中，所述层级细节项目特性至少包括：点云层级值，层级点数。
在一实施例中，所述点云全集群组包括：
完整点云对象中最多划分的部分访问的点云子集的数量、点云子集的类型、点云子集的标识符，其中，所述点云子集包括根据层级细节划分的点云子集。
在一实施例中，所述装置还包括：
第二确定资源模块，设置为确定并获取与所述部分项目数据相关联的一个或多个媒体资源；
第二解码资源模块，设置为在解码所述部分项目数据同时，对所述一个或多个媒体资源进行解码；
第二播放模块，设置为在根据解码后的数据渲染所述3D场景的部分层级细节的同时，对所述一个或多个媒体资源进行播放。
在一实施例中，所述第二确定资源模块包括：
第八确定子模块，设置为根据多实体播放群组确定并获取与所述部分项目数据相关联的一个或多个媒体资源。
在一实施例中，所述媒体资源至少包括：视频、音频、动态点云。
在一实施例中，所述点云项目数据由几何项目数据、一种或多种属性项目数据构成，所述几何项目数据和属性项目数据均划分为多个层级，每个层级由分布均匀的一组点集构成。
需要说明的是，上述各个模块是可以通过软件或硬件来实现的，对于后者，可以通过以下方式实现，但不限于此：上述模块均位于同一处理器中；或者，上述各个模块以任意组合的形式分别位于不同的处理器中。
本申请的实施例还提供了一种计算机可读存储介质，该计算机可读存储介质中存储有计算机程序，其中，该计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
在一个示例性实施例中，上述计算机可读存储介质可以包括但不限于：U盘、只读存储器(Read-Only Memory，简称为ROM)、随机存取存储器(Random Access Memory，简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储计算机程序的介质。
本申请的实施例还提供了一种电子装置，包括存储器和处理器，该存储器中存储有计算机程序，该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
在一个示例性实施例中，上述电子装置还可以包括传输设备以及输入输出设备，其中， 该传输设备和上述处理器连接，该输入输出设备和上述处理器连接。
本实施例中的具体示例可以参考上述实施例及示例性实施方式中所描述的示例，本实施例在此不再赘述。
显然，本领域的技术人员应该明白，上述的本申请的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本申请不限制于任何特定的硬件和软件结合。
以上所述仅为本申请的优选实施例而已，并不用于限制本申请，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (36)

  1. 一种点云数据处理方法，所述方法包括：
    确定包含3D场景的空间区域信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过点云项目数据表示；
    解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据；
    根据解码后的数据渲染所述3D场景的部分空间区域。
  2. 根据权利要求1所述的方法，其中，所述点云项目数据通过元信息数据盒中的项目类型确定，其中，所述项目类型至少包括：点云项目、点云分块项目、点云空间区域项目。
  3. 根据权利要求2所述的方法，其中，所述解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据包括：
    在所述项目类型为所述点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性确定对应于一个或多个分块的所述部分项目数据，解码所述部分项目数据。
  4. 根据权利要求3所述的方法，其中，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性数据盒确定对应于一个或多个分块的所述部分项目数据包括：
    根据所述子样本项目特性中第一子样本数据类型和分块标识符确定对应于一个或多个分块的部分项目数据。
  5. 根据权利要求2所述的方法，其中，所述解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据包括：
    在所述项目类型为所述点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性，以及3D空间区域项目特性确定对应于一个或多个空间区域的所述部分项目数据，解码所述部分项目数据。
  6. 根据权利要求5所述的方法，其中，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性，以及3D空间区域项目特性确定对应于一个或多个空间区域的所述部分项目数据包括：
    根据子样本项目特性中第二子样本数据类型和空间区域标识符确定对应于一个或多个空间区域的所述部分项目数据。
  7. 根据权利要求5所述的方法，其中，所述3D空间区域项目特性包括：
    空间区域数量，空间区域描述信息，空间区域包含的分块数量，空间区域包含的分块对应标识符。
  8. 根据权利要求7所述的方法，其中，所述空间区域描述信息包括：空间区域标识符，锚点坐标，空间区域的长、宽、高。
  9. 根据权利要求2所述的方法，其中，所述解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据包括：
    在所述项目类型为所述点云分块项目的情况下，根据点云全集群组识别包含所述3D场景的点云项目数据；
    根据点云配置项目特性中的点云分块集、所述元信息数据盒中一个或多个点云分块项目确定对应于一个或多个分块的所述部分项目数据，解码所述部分项目数据。
  10. 根据权利要求9所述的方法，其中，所述点云分块项目包含点云分块集中描述的单个分块，其中，所述一个或多个分块中的每个分块表示点云空间区域中可独立解码的数据子集。
  11. 根据权利要求2所述的方法，其中，所述解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据包括：
    在所述项目类型为所述点云空间区域项目的情况下，根据点云全集群组识别包含所述3D静态场景的点云数据；
    根据所述元信息数据盒中一个或多个点云空间区域项目，与点云空间项目关联的空间区域描述项目特性，确定对应于一个或多个点云空间区域的所述部分项目数据，解码所述部分项目数据。
  12. 根据权利要求11所述的方法，其中，所述一个或多个点云空间区域中的每个点云空间区域对应于空间区域描述项目特性描述的单个空间区域。
  13. 根据权利要求11所述的方法，其中，所述空间区域描述项目特性包括：空间区域标识符，锚点坐标，空间区域的长、宽、高。
  14. 根据权利要求9至13中任一项所述的方法，其中，所述点云全集群组包括：
    完整点云对象中最多划分的部分访问的点云子集的数量、点云子集的类型、点云子集的标识符，其中，所述点云子集包括：根据分块划分的点云子集、根据空间区域划分的点云子集。
  15. 根据权利要求2所述的方法，其中，所述解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据包括：
    在所述项目类型为所述点云空间区域项目的情况下，根据区域标记群组确定用于指示标记场景的所述部分项目数据，解码所述部分项目数据。
  16. 根据权利要求15所述的方法，其中，所述区域标记群组至少包括标记类型、标记文本，所述指示标记场景的部分点云项目通过所述标记类型、所述标记文本来描述。
  17. 根据权利要求1至13、15至16中任一项所述的方法，其中，所述方法还包括：
    确定并获取与所述部分项目数据相关联的一个或多个媒体资源；
    在解码所述部分项目数据的同时，对所述一个或多个媒体资源进行解码；
    在根据解码后的数据渲染所述3D场景的部分空间区域的同时，对所述一个或多个媒体资源进行播放。
  18. 根据权利要求17所述的方法，其中，所述确定并获取与所述部分项目数据相关联的一个或多个媒体资源包括：
    根据多实体播放群组确定并获取与所述部分项目数据相关联的一个或多个媒体资源。
  19. 根据权利要求17所述的方法，其中，所述媒体资源至少包括：视频、音频、动态点云。
  20. 根据权利要求1至13、15、16、18、19中任一项所述的方法，其中，所述点云项目数据由几何项目数据、一种或多种属性项目数据构成。
  21. 一种点云数据处理方法，所述方法包括：
    确定包含3D场景的层级细节信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过云项目数据表示；
    解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据；
    根据解码后的数据渲染所述3D场景的部分层级细节。
  22. 根据权利要求21所述的方法，其中，所述点云项目数据通过元信息数据盒中的项目类型确定，所述项目类型至少包括：点云项目、点云层级项目。
  23. 根据权利要求22所述的方法，其中，所述解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据包括：
    在所述项目类型为所述点云项目的情况下，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性确定对应于一个或多个层级的所述部分项目数据，解码所述部分项目数据。
  24. 根据权利要求23所述的方法，其中，根据所述元信息数据盒中与所述点云项目关联的子样本项目特性数据盒确定对应于一个或多个层级的所述部分项目数据包括：
    根据子样本项目特性中第三子样本数据类型和层级值确定对应于一个或多个层级的所述部分项目数据。
  25. 根据权利要求22所述的方法，其中，所述解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据包括：
    在所述项目类型为所述点云层级项目的情况下，根据点云全集群组识别包含所述3D静态场景全部细节的点云项目数据；
    根据所述元信息数据盒中一个或多个点云层级项目，以及与点云层级项目关联的层级细节项目特性确定对应于一个或多个层级的所述部分项目数据，解码所述部分项目数据。
  26. 根据权利要求25所述的方法，其中，所述一个或多个层级中的每个层级对应于层级细节项目特性描述的单个层级。
  27. 根据权利要求25所述的方法，其中，所述层级细节项目特性至少包括：点云层级值，层级点数。
  28. 根据权利要求23至27中任一项所述的方法，其中，所述点云全集群组包括：
    完整点云对象中最多划分的部分访问的点云子集的数量、点云子集的类型、点云子集的标识符，其中，所述点云子集包括根据层级细节划分的点云子集。
  29. 根据权利要求21至27中任一项所述的方法，其中，所述方法还包括：
    确定并获取与所述部分项目数据相关联的一个或多个媒体资源；
    在解码所述部分项目数据的同时，对所述一个或多个媒体资源进行解码；
    在根据解码后的数据渲染所述3D场景的部分层级细节的同时，对所述一个或多个媒体资源进行播放。
  30. 根据权利要求29所述的方法，其中，所述确定并获取与所述部分项目数据相关联的一个或多个媒体资源包括：
    根据多实体播放群组确定并获取与所述部分项目数据相关联的一个或多个媒体资源。
  31. 根据权利要求30所述的方法，其中，所述媒体资源至少包括：视频、音频、动态点云。
  32. 根据权利要求21至27、30至31中任一项所述的方法，其中，所述点云项目数据由几何项目数据、一种或多种属性项目数据构成，所述几何项目数据和所述属性项目数据均划分为多个层级，每个层级由分布均匀的一组点集构成。
  33. 一种点云数据处理装置，所述装置包括：
    第一确定模块，设置为确定包含3D场景的空间区域信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过点云项目数据表示；
    第一解码模块，设置为解码所述静态几何编码点云数据中对应所述3D场景的部分空间区域的部分项目数据；
    第一渲染模块，设置为根据解码后的数据渲染所述3D场景的部分空间区域。
  34. 一种点云数据处理装置，所述装置包括：
    第二确定模块，设置为确定包含3D场景的层级细节信息的静态几何编码点云数据，其中，所述静态几何编码点云数据通过云项目数据表示；
    第二解码模块，设置为解码所述静态几何编码点云数据中对应所述3D场景的部分层级细节的部分项目数据；
    第二渲染模块，设置为根据解码后的数据渲染所述3D场景的部分层级细节。
  35. 一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行所述权利要求1至20、21至32任一项中所述的方法。
  36. 一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行所述权利要求1至20、21至32任一项中所述的方法。
PCT/CN2021/119964 2020-09-30 2021-09-23 点云数据处理方法、装置、存储介质及电子装置 WO2022068672A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US18/246,868 US20230360304A1 (en) 2020-09-30 2021-09-23 Point cloud data processing method and apparatus, and storage medium and electronic apparatus
EP21874333.4A EP4224871A4 (en) 2020-09-30 2021-09-23 METHOD AND APPARATUS FOR PROCESSING POINT CLOUD DATA, AND STORAGE MEDIUM AND ELECTRONIC APPARATUS
CA3194294A CA3194294A1 (en) 2020-09-30 2021-09-23 Point cloud data processing method and apparatus, and storage medium and electronic apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011063014.XA CN112492385A (zh) 2020-09-30 2020-09-30 点云数据处理方法、装置、存储介质及电子装置
CN202011063014.X 2020-09-30

Publications (1)

Publication Number Publication Date
WO2022068672A1 true WO2022068672A1 (zh) 2022-04-07

Family

ID=74926131

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/119964 WO2022068672A1 (zh) 2020-09-30 2021-09-23 点云数据处理方法、装置、存储介质及电子装置

Country Status (5)

Country Link
US (1) US20230360304A1 (zh)
EP (1) EP4224871A4 (zh)
CN (1) CN112492385A (zh)
CA (1) CA3194294A1 (zh)
WO (1) WO2022068672A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115102932A (zh) * 2022-06-09 2022-09-23 腾讯科技（深圳）有限公司 点云媒体的数据处理方法、装置、设备、存储介质及产品

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112492385A (zh) * 2020-09-30 2021-03-12 中兴通讯股份有限公司 点云数据处理方法、装置、存储介质及电子装置
CN115150384B (zh) * 2021-03-31 2023-08-18 腾讯科技（深圳）有限公司 点云编解码方法、装置、计算机可读介质及电子设备
CN117529924A (zh) * 2021-06-30 2024-02-06 索尼集团公司 信息处理装置和方法
CN115618027A (zh) * 2021-07-12 2023-01-17 腾讯科技（深圳）有限公司 一种数据处理方法、装置、计算机及可读存储介质
CN115733576B (zh) * 2021-08-26 2024-06-25 腾讯科技（深圳）有限公司 点云媒体文件的封装与解封装方法、装置及存储介质

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020012967A1 (ja) * 2018-07-11 2020-01-16 ソニー株式会社 画像処理装置および方法
CN111179394A (zh) * 2019-11-25 2020-05-19 苏州智加科技有限公司 点云场景渲染方法、装置和设备
WO2020137642A1 (ja) * 2018-12-28 2020-07-02 ソニー株式会社 情報処理装置および情報処理方法
US20200217937A1 (en) * 2019-01-08 2020-07-09 Apple Inc. Point cloud compression using a space filling curve for level of detail generation
WO2020190093A1 (ko) * 2019-03-20 2020-09-24 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법
US20200302632A1 (en) * 2019-03-21 2020-09-24 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus, and point cloud data reception method
WO2020189976A1 (ko) * 2019-03-16 2020-09-24 엘지전자 주식회사 포인트 클라우드 데이터 처리 장치 및 방법
WO2020189895A1 (ko) * 2019-03-21 2020-09-24 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법
CN112492385A (zh) * 2020-09-30 2021-03-12 中兴通讯股份有限公司 点云数据处理方法、装置、存储介质及电子装置
CN112700550A (zh) * 2021-01-06 2021-04-23 中兴通讯股份有限公司 三维点云数据处理方法、装置、存储介质及电子装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11200701B2 (en) * 2019-03-19 2021-12-14 Nokia Technologies Oy Method and apparatus for storage and signaling of static point cloud data
US11968393B2 (en) * 2019-03-20 2024-04-23 Lg Electronics Inc. Point cloud data transmitting device, point cloud data transmitting method, point cloud data receiving device, and point cloud data receiving method
CN111684494A (zh) * 2019-05-23 2020-09-18 深圳市大疆创新科技有限公司 点云的实时显示方法、装置和计算机存储介质

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020012967A1 (ja) * 2018-07-11 2020-01-16 ソニー株式会社 画像処理装置および方法
WO2020137642A1 (ja) * 2018-12-28 2020-07-02 ソニー株式会社 情報処理装置および情報処理方法
US20200217937A1 (en) * 2019-01-08 2020-07-09 Apple Inc. Point cloud compression using a space filling curve for level of detail generation
WO2020189976A1 (ko) * 2019-03-16 2020-09-24 엘지전자 주식회사 포인트 클라우드 데이터 처리 장치 및 방법
WO2020190093A1 (ko) * 2019-03-20 2020-09-24 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법
US20200302632A1 (en) * 2019-03-21 2020-09-24 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus, and point cloud data reception method
WO2020189895A1 (ko) * 2019-03-21 2020-09-24 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법
CN111179394A (zh) * 2019-11-25 2020-05-19 苏州智加科技有限公司 点云场景渲染方法、装置和设备
CN112492385A (zh) * 2020-09-30 2021-03-12 中兴通讯股份有限公司 点云数据处理方法、装置、存储介质及电子装置
CN112700550A (zh) * 2021-01-06 2021-04-23 中兴通讯股份有限公司 三维点云数据处理方法、装置、存储介质及电子装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4224871A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115102932A (zh) * 2022-06-09 2022-09-23 腾讯科技（深圳）有限公司 点云媒体的数据处理方法、装置、设备、存储介质及产品
CN115102932B (zh) * 2022-06-09 2024-01-12 腾讯科技（深圳）有限公司 点云媒体的数据处理方法、装置、设备、存储介质及产品

Also Published As

Publication number Publication date
EP4224871A1 (en) 2023-08-09
CN112492385A (zh) 2021-03-12
EP4224871A4 (en) 2024-03-20
US20230360304A1 (en) 2023-11-09
CA3194294A1 (en) 2022-04-07

Similar Documents

Publication Publication Date Title
WO2022068672A1 (zh) 点云数据处理方法、装置、存储介质及电子装置
CN111213384B (zh) 用于生成定时媒体数据的方法、装置和计算机可读存储介质
Schilling et al. Using glTF for streaming CityGML 3D city models
US20240054682A1 (en) Three-Dimensional Point Cloud Data Processing Method and Apparatus, Storage Medium and Electronic Apparatus
TW202042551A (zh) 沈浸式媒體資料軌道之軌道導出方法和裝置
TW202106001A (zh) 用於沉浸式媒體資料軌道的空間分組以及坐標信令的方法及裝置
WO2022037386A1 (zh) 点云媒体的数据处理方法、装置、设备及存储介质
US11818326B2 (en) Methods and apparatus for signaling viewing regions of various types in immersive media
KR20230136639A (ko) 볼류메트릭 미디어 처리 방법, 장치, 저장 매체 및 전자 장치
US20230254549A1 (en) Method and apparatus for encapsulating annotated region in isobmff tracks
WO2023207119A1 (zh) 沉浸媒体的处理方法、装置、设备及存储介质
KR102624994B1 (ko) 포인트 클라우드 데이터를 위한 파일 형식
WO2024041239A1 (zh) 一种沉浸媒体的数据处理方法、装置、设备、存储介质及程序产品
WO2024041238A1 (zh) 一种点云媒体的数据处理方法及相关设备
WO2021258325A1 (en) Three-dimensional content processing methods and apparatus
KR20220035229A (ko) 볼류메트릭 비디오 콘텐츠를 전달하기 위한 방법 및 장치
WO2022193875A1 (zh) 多视角视频的处理方法、装置、设备及存储介质
WO2022078148A1 (zh) 媒体文件的封装方法、媒体文件的解封装方法及相关设备
CN114556962B (zh) 多视点视频处理方法和装置
WO2023169003A1 (zh) 点云媒体的解码方法、点云媒体的编码方法及装置
US11743441B2 (en) Methods and apparatus for selecting and switching input video tracks using track derivations
WO2023169001A1 (zh) 一种沉浸媒体的数据处理方法、装置、设备及存储介质
CN114598692B (zh) 点云文件传输方法、应用方法、装置、设备及存储介质
WO2023111099A1 (en) Method and apparatus for encapsulating 3d region related annotation in a media file
CN118118694A (zh) 点云封装与解封装方法、装置、介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21874333

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 3194294

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021874333

Country of ref document: EP

Effective date: 20230502