WO2022062860A1 - 一种点云媒体的数据处理方法、装置、设备及存储介质 - Google Patents

一种点云媒体的数据处理方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022062860A1
WO2022062860A1 PCT/CN2021/115689 CN2021115689W WO2022062860A1 WO 2022062860 A1 WO2022062860 A1 WO 2022062860A1 CN 2021115689 W CN2021115689 W CN 2021115689W WO 2022062860 A1 WO2022062860 A1 WO 2022062860A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
sample group
media
indication information
ith
Prior art date
Application number
PCT/CN2021/115689
Other languages
English (en)
French (fr)
Inventor
胡颖
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2022062860A1 publication Critical patent/WO2022062860A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46—Interconnection of networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46—Interconnection of networks
    • H04L12/4633—Interconnection of networks using encapsulation techniques, e.g. tunneling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194—Transmission of image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]

Definitions

  • This application relates to the field of computer technology, in particular to the technical field of point cloud media, and in particular to a data processing method for point cloud media, a data processing device for point cloud media, a data processing device for point cloud media, and a computer Readable storage medium.
  • Point cloud data is often in the form of point cloud media in content production equipment and content consumption. transfer between devices.
  • the transmission process of point cloud media is as follows: after the content production device encodes the point cloud media, it encapsulates the encoded point cloud media to obtain the package file of the point cloud media, and the content production device transmits the package file of the point cloud media to Content consumption device; the content consumption device decapsulates the package file of the point cloud media transmitted by the content production device, and then decodes it, and finally the content consumption device presents the media file. Due to the large amount of point cloud data contained in point cloud media, how to improve the analysis and processing efficiency of point cloud media, so as to bring a better experience for point cloud media consumption, is a problem that the industry has been continuously solving.
  • An embodiment of the present application provides a data processing method for point cloud media, which is executed by a first electronic device, and the data processing method for point cloud media includes:
  • the point cloud media includes N sample groups, and the ith sample group is any one of the N sample groups; the ith sample group includes point cloud objects, and the ith sample group includes point cloud objects.
  • the point cloud object indication information of the i sample group is used to indicate the attributes of the point cloud objects contained in the i-th sample group, N and i are both positive integers and i ⁇ [1,N];
  • An embodiment of the present application provides a data processing method for point cloud media, which is executed by a second electronic device, and the data processing method for point cloud media includes:
  • the point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups; the i-th sample group includes point cloud objects, and the ith sample group
  • the point cloud object indication information of the i sample group is used to indicate the attributes of the point cloud objects contained in the i-th sample group, N and i are both positive integers and i ⁇ [1,N];
  • An embodiment of the present application provides a data processing device for point cloud media, and the data processing device for point cloud media includes:
  • the obtaining unit is configured to obtain the point cloud object indication information of the ith sample group of the point cloud media, the point cloud media includes N sample groups, and the ith sample group is any one of the N sample groups; the ith sample group includes Point cloud object, the point cloud object indication information of the ith sample group is used to indicate the attributes of the point cloud object contained in the ith sample group, N and i are both positive integers and i ⁇ [1, N];
  • the processing unit is configured to parse the point cloud media according to the point cloud object indication information of the ith sample group.
  • An embodiment of the present application provides a data processing device for point cloud media, and the data processing device for point cloud media includes:
  • the processing unit is configured to generate point cloud object indication information of the ith sample group of the point cloud media, the point cloud media includes N sample groups, and the ith sample group is any one of the N sample groups; the ith sample group includes Point cloud object, the point cloud object indication information of the ith sample group is used to indicate the attributes of the point cloud object contained in the ith sample group, N and i are both positive integers and i ⁇ [1, N];
  • the transmission unit is configured to send the point cloud object indication information of the ith sample group to the first electronic device, so that the first electronic device parses the point cloud media according to the point cloud object indication information of the ith sample group.
  • An embodiment of the present application provides a data processing device for point cloud media, and the data processing device for point cloud media includes:
  • a processor adapted to implement computer instructions
  • a computer-readable storage medium where computer instructions are stored in the computer-readable storage medium, and the computer instructions are suitable for being loaded by a processor and executing the above-mentioned data processing method for point cloud media.
  • An embodiment of the present application provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and the computer instructions are suitable for being loaded by a processor and executing the above-mentioned data processing method for point cloud media.
  • Embodiments of the present application provide a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned data processing method for point cloud media.
  • FIG. 1 shows a schematic diagram of the architecture of a data processing system for point cloud media provided by an embodiment of the present application
  • FIG. 2a shows a schematic architectural diagram of a data processing architecture for point cloud media provided by an embodiment of the present application
  • FIG. 2b shows a schematic structural diagram of a sample provided by an embodiment of the present application
  • FIG. 2c shows a schematic structural diagram of a container including multiple file tracks provided by an embodiment of the present application
  • FIG. 2d shows a schematic structural diagram of a sample provided by an embodiment of the present application.
  • FIG. 3 shows a schematic flowchart of a data processing method for point cloud media provided by an embodiment of the present application
  • FIG. 4 shows a schematic flowchart of a data processing method for point cloud media provided by an embodiment of the present application
  • FIG. 5 shows a schematic structural diagram of a data processing apparatus for point cloud media provided by an embodiment of the present application
  • FIG. 6 shows a schematic structural diagram of a data processing apparatus for point cloud media provided by an embodiment of the present application
  • FIG. 7 shows a schematic structural diagram of a data processing device for point cloud media provided by an embodiment of the present application.
  • Point cloud data is a specific recording form of point cloud.
  • the point cloud data of each point in the point cloud can include geometric information (ie, three-dimensional position information) and attribute information.
  • the geometry of each point in the point cloud refers to the Cartesian three-dimensional coordinate data of the point, and the attribute information of each point in the point cloud may include, but is not limited to, at least one of the following: color information, material information, and laser reflection intensity information.
  • each point in the point cloud can have the same amount of attribute information, for example, each point in the point cloud has two types of attribute information: color information and laser reflection intensity; It has three attribute information: color information, material information and laser reflection intensity information.
  • the way to obtain point cloud data may include but not be limited to at least one of the following: 1 computer equipment generation, computer equipment can generate point cloud data according to virtual three-dimensional objects or virtual three-dimensional scenes; 2 three-dimensional (3-Dimensional) , 3D) laser scanning acquisition, through 3D laser scanning, point cloud data of static real-world 3D objects or 3D scenes can be obtained, and millions of point cloud data can be obtained per second; 3 3D photogrammetry acquisition, through 3D photography equipment (ie a set of A camera, or a camera device with multiple lenses and sensors) collects real-world visual scenes to obtain point cloud data of real-world visual scenes, and obtains point cloud data of dynamic real-world three-dimensional objects or three-dimensional scenes through 3D photography ;4 Obtain the point cloud data of biological tissues and organs through medical equipment. In the medical field, biological tissues can be obtained through medical equipment such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT), and electromagnetic positioning information. Organ point cloud data.
  • MRI Magnetic Resonance Imaging
  • CT
  • point cloud media refers to point cloud media files formed by point cloud data.
  • the point cloud media includes multiple media frames, and each media frame in the point cloud media is composed of point cloud data.
  • Point cloud media can express the spatial structure and surface properties of 3D objects or 3D scenes flexibly and conveniently, so it is widely used.
  • the main application scenarios of point cloud media can be classified into two categories: the first category is machine perception point cloud, such as autonomous navigation system (Autonomous Navigation System, ANS), real-time inspection system, geographic information system (Geography Information System, GIS) , visual sorting robot, rescue and disaster relief robot, etc.; the second category is human eye perception point cloud, such as digital cultural heritage, free viewpoint broadcasting, Computer Aided Design (CAD), 3D immersive communication, 3D immersive interaction, biological 3D reconstruction of tissues and organs, etc.
  • machine perception point cloud such as autonomous navigation system (Autonomous Navigation System, ANS), real-time inspection system, geographic information system (Geography Information System, GIS) , visual sorting robot, rescue and disaster relief robot, etc.
  • GIS Geographic Information System
  • visual sorting robot eography Information System
  • rescue and disaster relief robot etc.
  • human eye perception point cloud such as digital cultural heritage, free viewpoint broadcasting, Computer Aided Design (CAD), 3D immersive communication, 3D immersive interaction, biological 3D reconstruction of tissues and organs, etc
  • Point cloud objects refer to objects existing in the media frames of point cloud media, that is, special objects identified from the media frames of point cloud media and under certain application scenarios.
  • the types of point cloud objects can include any of the following: Types: scene anomalies (also known as scene anomalies, such as abnormal high-voltage wire nodes, abnormal street lights, etc. detected in the real-time inspection scene), scene indication objects (such as traffic lights, cameras identified in the high-precision map scene) etc.), target objects (for example, the organisms waiting for rescue identified in the rescue and disaster relief scenes, such as people, dogs, etc.).
  • scene anomalies also known as scene anomalies, such as abnormal high-voltage wire nodes, abnormal street lights, etc. detected in the real-time inspection scene
  • scene indication objects such as traffic lights, cameras identified in the high-precision map scene
  • target objects for example, the organisms waiting for rescue identified in the rescue and disaster relief scenes, such as people, dogs, etc.
  • the i-th sample group including point cloud objects in the point cloud media is indicated, and the point cloud object indication information of the i-th sample group is generated.
  • the point cloud object indication information of the ith sample group is used to indicate the attributes of the point cloud objects contained in the ith sample group, and the attributes include at least one of the following: the number of point cloud objects, the type of point cloud objects, the Description information, the application scenario to which the point cloud object belongs, and the priority of the point cloud object.
  • various point cloud objects in the point cloud media and the attributes of the point cloud objects can be indicated according to the point cloud object indication information of the i-th sample group of the point cloud media, which makes the point cloud
  • the technical standard can support more abundant application scenarios; at the same time, according to the attributes indicated by the point cloud object indication information, the transmission strategy of point cloud media can be flexibly determined, which can effectively improve the transmission efficiency of point cloud media under certain network conditions; This effectively improves the analysis and processing efficiency of the point cloud media by the first electronic device, thereby bringing a better experience to the consumption of the point cloud media.
  • FIG. 1 shows a schematic diagram of the architecture of a point cloud media data processing system provided by an embodiment of the present application.
  • the first electronic device is used as a content consumption device
  • the second electronic device is used as a content production device.
  • the point cloud media data processing system 10 includes a content consumption device 101 and a content production device 102 .
  • the content production device 102 refers to a computer device used by a point cloud media provider (for example, a point cloud media content producer), and the computer device may be a terminal, a server, a mobile platform, or the like, which is capable of encoding and encapsulating point cloud media. device of.
  • the content consumption device 101 refers to a computer device used by a user (eg, a user) of point cloud media, and the computer device may be a terminal, a mobile platform, or other device capable of decapsulating and decoding point cloud media.
  • the above-mentioned terminal may be a personal computer (Personal Computer, PC), a smart mobile device (such as a smart phone), a tablet computer, a notebook computer, a desktop computer, a smart TV, a smart watch, a virtual reality (Virtual Reality, VR) device (such as VR) helmet, VR glasses), etc.
  • the movable platform can be a drone (Unmanned Aerial Vehicle, UAV), a robot, etc., which is not limited.
  • the content production device 102 and the content consumption device 101 may be directly or indirectly connected through wired communication or wireless communication, which is not limited in this embodiment of the present application.
  • FIG. 2a shows a schematic diagram of the data processing architecture of a point cloud media provided by an embodiment of the present application.
  • the data processing system of the point cloud media shown in FIG. 1 and the data of the point cloud media shown in FIG. 2a will be combined below.
  • Processing architecture, the data processing solution for point cloud media provided by the embodiments of this application is introduced.
  • the data processing process of point cloud media includes the data processing process on the side of the content production device and the data processing process on the side of the content consumption device.
  • the specific processing process is as follows:
  • the acquisition method of point cloud data can be divided into two methods: collecting real-world visual scenes through a capture device and generating through a computer device.
  • the capture device may be a hardware component provided in the content production device, for example, the capture device is a camera, a sensor, or the like of a terminal.
  • the capture device may also be a hardware device connected to the content production device, such as a camera connected to a server.
  • the capture device is used to provide point cloud data acquisition services for the content production device, and the capture device may include, but is not limited to, any of the following: a camera device, a sensor device, and a scanning device; wherein, the camera device may include an ordinary camera, a stereo camera, Light field cameras, etc.; sensing devices may include laser devices, radar devices, etc.; scanning devices may include 3D laser scanning devices, etc.
  • the number of capture devices can be multiple, and these capture devices are deployed in some specific positions in the real space to capture point cloud data from different angles in the space at the same time, and the captured point cloud data are synchronized in time and space.
  • the computer device may generate point cloud data according to the virtual three-dimensional object and the virtual three-dimensional scene. Due to the different acquisition methods of point cloud data, the corresponding compression coding methods of point cloud data acquired by different methods may also be different.
  • the content production device may use Geometry-Based Point Cloud Compression (GPCC) encoding or traditional video encoding-based Video-Based Point Cloud Compression (VPCC) encoding
  • GPCC Geometry-Based Point Cloud Compression
  • VPCC Video-Based Point Cloud Compression
  • the obtained point cloud data is encoded by the method to obtain an encoded bit stream of the point cloud data, such as a GPCC bit stream corresponding to the GPCC encoding method or a VPCC bit stream corresponding to the VPCC encoding method.
  • the media frame may be used as a unit, that is, the media frame may be encoded to obtain an encoded bit stream.
  • the content production device uses a file track to encapsulate the GPCC bit stream obtained by the encoding process.
  • the so-called file track refers to the encapsulation container of the GPCC bitstream.
  • the GPCC bitstream can be encapsulated in a single file track or multiple file tracks.
  • the GPCC bitstream is encapsulated in a single file track and multiple file tracks.
  • the GPCC bitstream is encapsulated in a single file track.
  • the GPCC bitstream is required to be declared and represented according to the transport rules of the single file track.
  • the GPCC bit stream can be encapsulated into a single file track by the International Organization for Standardization Base Media File Format (ISOBMFF).
  • ISOBMFF International Organization for Standardization Base Media File Format
  • each sample (Sample) packaged in a single file track contains one or more GPCC components
  • the so-called sample refers to the package structure set of one or more point clouds, and is the package unit in the process of point cloud media packaging
  • Point cloud media contains multiple samples
  • one sample usually represents one media frame of point cloud media
  • each sample is encapsulated by one or more Type-Length-Value ByteStream Format (TLV) encapsulation structures composition.
  • Fig. 2b shows a schematic structural diagram of a sample provided by an embodiment of the present application. As shown in Fig. 2b, when a single file track is transmitted, the sample in the file track consists of a GPCC parameter set TLV, a geometric bitstream TLV and an attribute The bitstream TLV consists of the samples packed into a single file track.
  • the GPCC bitstream is encapsulated in multiple file tracks.
  • each sample in the file track contains at least one TLV package structure that carries a single GPCC component, and the TLV package structure Do not contain both the GPCC geometry bitstream and the GPCC attribute bitstream.
  • Fig. 2c shows a schematic structural diagram of an encapsulation container including multiple file tracks provided by an embodiment of the present application. As shown in Fig.
  • the encapsulation package 1 transmitted in the file track 1 includes the GPCC geometric bit stream, and does not include GPCC attribute bitstream; Packet 2 transmitted in file track 2 contains the GPCC attribute bitstream and does not contain the GPCC geometry bitstream. Since the content consumption device decodes the GPCC geometry bitstream first, and the decoding of the GPCC attribute bitstream depends on the geometry information obtained by decoding the GPCC geometry bitstream, different GPCC component bitstreams are encapsulated in different files. track, enabling content consumption devices to access the file track carrying the GPCC geometry bitstream before the GPCC attribute bitstream.
  • FIG. 2d shows a schematic structural diagram of a sample provided by an embodiment of the present application.
  • FIG. 2d shows The sample in the file track is composed of the GPCC parameter set TLV and the geometric bitstream TLV, and the sample does not contain the attribute bitstream TLV, and the sample is encapsulated in any one of the multiple file tracks.
  • the obtained point cloud data is encoded and packaged by the content production device to form an encapsulation file (for example, several sample groups) of the point cloud media.
  • the encapsulation file of the point cloud media may be the entire media file, or Can be a media fragment in a media file.
  • the content production device can use the media presentation description information (that is, the description signaling file) (Media Presentation Description, MPD) according to the file format requirements of the point cloud media to record the metadata of the encapsulated file of the point cloud media, where the metadata is the same as the one.
  • the metadata may include description information of the media content, description information of the window, and signaling information related to the presentation of the media content.
  • the content production device sends the MPD to the content consumption device, so that the content consumption device requests to obtain the package file of the point cloud media according to the relevant description information in the MPD.
  • the content production device sends the package file and MPD of the point cloud media to the content consumption device through a transmission mechanism, wherein the transmission mechanism is such as Dynamic Adaptive Streaming over HTTP (DASH), Smart Media Transport, SMT) and so on.
  • DASH Dynamic Adaptive Streaming over HTTP
  • SMT Smart Media Transport
  • the content consumption device may obtain the package file of the point cloud media through the MPD sent by the content production device.
  • the file decapsulation process on the content consumption device side is inverse to the file encapsulation process on the content production device side. GPCC bitstream or VPCC bitstream).
  • the decoding process on the content consumption device is opposite to the encoding process on the content production device. For example, the content consumption device decodes the encoded bit stream to restore point cloud data.
  • the content consumption device decodes point cloud data (such as media containing point cloud data) obtained by decoding the encoded bitstream according to metadata related to rendering and viewport in the MPD (such as the viewing (window) direction of the current user) frame) for rendering, and the rendering is completed to realize the presentation of the visual scene corresponding to the point cloud data.
  • point cloud data such as media containing point cloud data
  • the real-world visual scene is sampled by the capture device to obtain point cloud data corresponding to the real-world visual scene;
  • the point cloud data is encoded to obtain GPCC bit stream or VPCC bit stream (both of these bit streams include geometric bit stream and attribute bit stream); then encapsulate the GPCC bit stream or VPCC bit stream to obtain the encapsulation of point cloud media files (including media files or media clips);
  • content production equipment can also encapsulate metadata into media files or media clips, and send the encapsulated files of point cloud media to Content consumption device.
  • the content consumption device For the content consumption device, first receive the point cloud media encapsulation file sent by the content production device; then decapsulate the point cloud media encapsulation file to obtain an encoded bit stream (such as GPCC bit stream or VPCC bit stream) and metadata ; Then parse the metadata in the encoded bit stream (that is, decode the encoded bit stream to obtain point cloud data); finally, based on the viewing (window) direction of the current user, the decoded point cloud data is rendered and displayed in the In the content consumption device, for example, it is displayed in the human-computer interaction interface provided by the content consumption device. It should be noted that the viewing (window) direction of the current user is determined by the head tracking and/or visual tracking function.
  • the audio of the current user's viewing (window) direction can also be decoded and optimized through an audio decoder.
  • the collected point cloud data is encoded and encapsulated by the content production equipment to realize the effective storage of the point cloud data; Transmission, publishing and sharing; the content consumption device decapsulates, decodes and consumes the encapsulated files of point cloud media, so that the real-world visual scene can be presented in the content consumption device.
  • the content production device needs to encode the point cloud media and encapsulate it into a package file of the point cloud media before sending it to the content consumption device.
  • the point cloud media can only be rendered and presented after the encapsulated file is decapsulated and decoded.
  • the data processing system for point cloud media provided by the embodiments of the present application supports a data box (Box), such as an ISOBMFF data box.
  • a data box refers to a data block including metadata or an object including metadata, that is, a data box.
  • point cloud media contains metadata of point cloud media; point cloud media can be associated with multiple data boxes, for example, point cloud media includes N sample groups, point cloud media is associated with N data boxes, and the ith sample group corresponds to the ith data box.
  • the embodiment of the present application expands the data box supported by the data processing system of point cloud media, and encapsulates the point cloud object indication information of the ith sample group including the point cloud object in the data box; the point cloud object of the ith sample group indicates the The information is the point cloud object indication group entry (PointCloudObjectIndicationGroupEntry) class in the data box, and the point cloud object indication information of the i-th sample group contains at least one of the following fields: object scene field (object_sceiario), object priority field (object_priority), The object count field (object_count), the object type field (object_type), and the object description field (object_description); the object scene field is used to indicate the application scene to which the point cloud objects contained in the i-th sample group belong, and the object priority field is
  • the object quantity field is used to indicate the number of point cloud objects contained in the i th sample group
  • the object type field is used to indicate the type of point cloud objects contained in the i th sample group
  • the object description field is used to Indicates the description information of the point cloud objects contained in the i-th sample group.
  • the object scene field object_scenario indicates the application scene to which the point cloud object contained in the i-th sample group belongs.
  • the value of the object scene field is different.
  • the example corresponding relationship between the value of the object scene field and the application scene is shown in Table 2. When the value of the object scene field is 0, it indicates the i-th sample group.
  • the application scene to which the point cloud objects contained in the sample group belong is a high-precision map scene; when the value of the object scene field is 1, it indicates that the application scene to which the point cloud objects contained in the i-th sample group belong is a real-time inspection scene; When the value of the scene field is 2, it indicates that the application scene of the point cloud object contained in the i-th sample group belongs to the rescue and disaster relief scene; it should be noted that the object scene field also has other extended values, that is, when the point When cloud objects exist in other application scenarios except the above three application scenarios (ie, high-precision map scenarios, real-time inspection scenarios, and rescue and disaster relief scenarios), it is only necessary to expand the value of the object scenario field to apply other applications. scene to indicate.
  • the object priority field object_priority indicates the priority of the i-th sample group.
  • the i-th sample group contains M point cloud objects, and M is a positive integer.
  • the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority. level, then the priority of the i-th sample group is equal to the priority of the point cloud objects contained in the i-th sample group; when the value of M is greater than 1, the i-th sample group contains M point cloud objects, M point cloud objects Each point cloud object in , corresponds to a priority, and the priority of the i-th sample group is equal to the highest priority among the M priorities (that is, the priorities corresponding to the M point cloud objects).
  • the object number field object_count indicates the number of point cloud objects contained in the i-th sample group. Assuming that the i-th sample group contains M point cloud objects, the value of the object quantity field is M. For example, the i-th sample group contains 1 point cloud object, then the object quantity field is 1; the i-th sample group If it contains 10 point cloud objects, the value of the object quantity field is 10.
  • the object type field object_type indicates the type of the point cloud object contained in the i-th sample group. Assuming that the ith sample group contains M point cloud objects, the point cloud object indication information of the ith sample group contains M object type fields, and the M object type fields are used to respectively indicate the types of the M point cloud objects, for example
  • the mth point cloud object is any one of the M point cloud objects, and the mth object type field is used to indicate the type of the mth point cloud object.
  • the value of the object type field corresponding to different types of point cloud objects is different. An example correspondence between the value of the object type field and the type of the point cloud object is shown in Table 3.
  • the value of the object type field is 0 , indicating that the type of the point cloud object is the scene exception (also known as the scene exception object); when the value of the object type field is 1, it indicates that the type of the point cloud object is the scene indicator object; when the value of the object type field is 2 , it indicates that the type of the point cloud object is the target object; it should be noted that there are other extended values in the object type field, that is, when the type of the point cloud object is other than the above three types (ie, the scene exception, the scene When indicating other types other than the object and target object), it is only necessary to extend the value of the object type field to indicate other types.
  • the object description field object_description indicates the description information of the point cloud object contained in the i-th sample group.
  • the point cloud object indication information of the ith sample group contains M object description fields, and the M object description fields are used to respectively indicate the description information of the M point cloud objects.
  • the mth point cloud object is any one of the M point cloud objects
  • the value of the mth object description field is a null-terminated 8-bit (Unicode Transformation Format-8, UTF-8) string, used for Indicates the description information of the mth point cloud object.
  • the content production device After the content production device indicates the i-th sample group that contains point cloud objects in the point cloud media, it generates point cloud object indication information of the i-th sample group, and the point cloud object indication information of the i-th sample group is used for Indicates the attributes of the point cloud objects contained in the i-th sample group (for example, priority, application scenario, type, etc.); the content consumption device can indicate the point cloud according to the point cloud object indication information of the i-th sample group of the point cloud media Various point cloud objects in the media and the attributes of the point cloud objects, so as to analyze the point cloud media, which enables the point cloud technical standards to support more abundant application scenarios; and according to the attributes indicated by the point cloud object indication information of the sample group, It can flexibly determine the transmission strategy of point cloud media, effectively improve the transmission efficiency of point cloud media under certain network conditions, and also effectively improve the analysis and processing efficiency of point cloud media by content consumption equipment; During the transmission of cloud media to the content consumption device, if the transmission network congestion is
  • FIG. 3 shows a schematic flowchart of a data processing method for point cloud media provided by an embodiment of the present application.
  • the content consumption device 101 is executed, the data processing method of the point cloud media includes the following steps S301 to S302:
  • Step S301 obtaining the point cloud object indication information of the ith sample group of the point cloud media, the point cloud media includes N sample groups, the ith sample group is any one of the N sample groups, and the ith sample group includes the point cloud.
  • Object, N and i are both positive integers and i ⁇ [1, N].
  • Step S302 parse the point cloud media according to the point cloud object indication information of the i-th sample group.
  • the point cloud object indication information (PointCloudObjectIndicationGroupEntry) of the ith sample group is used to indicate attributes of the point cloud objects included in the ith sample group, wherein the attributes may include at least one of the following: Quantity, type of point cloud objects, description information of point cloud objects, application scenarios to which point cloud objects belong, and priority of point cloud objects.
  • the point cloud object indication information of the ith sample group includes an object priority field (object_priority), and the object priority field is used to indicate the priority of the ith sample group.
  • object_priority object priority field
  • the priority of the ith sample group can be determined according to the priority of the point cloud objects contained in the ith sample group.
  • the ith sample group contains M point cloud objects
  • M is a positive integer
  • M 1
  • the priority of the ith sample group is equal to the priority of the point cloud object contained in the ith sample group
  • the ith sample group contains M point cloud objects
  • each point cloud object in the M point cloud objects corresponds to a priority
  • the priority of the ith sample group is equal to M priorities highest priority in .
  • the larger the value of the object priority field in the lower the priority of the i-th sample group.
  • the first electronic device can prioritize the resolution point The sample group with higher priority in cloud media, that is, the higher the priority of the i-th sample group, the higher the parsing order of the i-th sample group in parsing; the lower the priority of the i-th sample group, the higher The sample group is parsed later in the parsing order.
  • the sample group with higher priority in the point cloud media can be analyzed first, and then the sample group with lower priority in the point cloud media can be analyzed.
  • the jth sample group is any one of the N sample groups except the ith sample group, j is a positive integer and j ⁇ [1,N], the jth sample group includes point cloud objects, and the jth sample group
  • the value of the object priority field contained in the point cloud object indication information of the ith sample group is greater than the value of the object priority field contained in the point cloud object indication information of the ith sample group, then the priority of the ith sample group is higher than that of the jth sample group The priority of the sample group.
  • the i-th sample group is first analyzed according to the point cloud object indication information of the i-th sample group, and then the j-th sample group is analyzed according to the point cloud object indication information of the j-th sample group.
  • the priority of a sample group that includes point cloud objects in the point cloud media is higher than the priority of a sample group that does not include point cloud objects in the point cloud media.
  • the sample groups in the point cloud media that contain the point cloud objects are analyzed first, and then the sample groups that do not contain the point cloud objects in the point cloud media are analyzed.
  • the jth sample group is any one of the N sample groups except the ith sample group, j is a positive integer and j ⁇ [1, N], and the jth sample group does not contain point cloud objects, the ith sample If the group contains point cloud objects, the priority of the i-th sample group is higher than that of the j-th sample group.
  • the i-th sample is preferentially analyzed according to the point cloud object indication information of the i-th sample group. group, and then analyze the jth sample group.
  • the point cloud object indication information of the ith sample group includes an object count field (object_count), and the object count field is used to indicate the number of point cloud objects contained in the ith sample group; let the ith sample group be contains M point cloud objects, the value of the object quantity field is M.
  • object_count object count field
  • the point cloud object indication information of the ith sample group includes an object scene field (object_sceiario), and the object scene field is used to indicate the application scene to which the point cloud object contained in the ith sample group belongs.
  • object_sceiario object scene field
  • the value of the object scene field is different.
  • the object scene field in the point cloud object indication information of the ith sample group is read, and the point cloud in the ith sample group is determined according to the value of the object scene field.
  • the application scenarios may include at least one of the following: map scenarios (such as high-precision map scenarios), real-time inspection scenarios, and rescue and disaster relief scenarios.
  • map scenarios such as high-precision map scenarios
  • real-time inspection scenarios real-time inspection scenarios
  • rescue and disaster relief scenarios For example, according to the value 0 of the object scene field in the point cloud object indication information of the ith sample group, it is determined that the application scene to which the point cloud object in the ith sample group belongs is a high-precision
  • the point cloud object indication information of the ith sample group includes an object type field (object_type), and the object type field is used to indicate the type of the point cloud object included in the ith sample group.
  • object_type object type field
  • the ith sample group contains M point cloud objects, where M is a positive integer
  • the point cloud object indication information of the ith sample group contains M object type fields
  • the M object type fields are used to respectively indicate M point clouds The type of the object.
  • the value of the object type field corresponding to different types of point cloud objects is different.
  • the mth object type field is used to indicate the type of the mth point cloud object, m is a positive integer and m ⁇ [1, M].
  • the mth object type field in the point cloud object indication information of the ith sample group is read, and the ith sample group is determined according to the value of the mth object type field.
  • the type of the m-th point cloud object within; wherein, the type may include any of the following: scene anomalies, scene indicating objects, and target objects. For example, according to the value 0 of the m th object type field in the point cloud object indication information of the ith sample group, it is determined that the type of the m th point cloud object in the ith sample group is a scene abnormality.
  • the point cloud object indication information of the ith sample group includes an object description field (object_description), and the object description field is used to indicate description information of the point cloud objects included in the ith sample group.
  • object_description object description field
  • the ith sample group contains M point cloud objects, where M is a positive integer
  • the point cloud object indication information of the ith sample group contains M object description fields
  • the M object description fields are used to respectively indicate M point clouds Description of the object.
  • the value of the mth object description field can be a null-terminated 8-bit string, which is used to indicate the description information of the mth point cloud object.
  • m is a positive integer and m ⁇ [1,M].
  • the mth object description field in the point cloud object indication information of the ith sample group is read, and the ith sample group is determined according to the value of the mth object description field.
  • the value of the mth object description field alarm determines the description information of the mth point cloud object in the ith sample group as alarm information, and respond to the alarm information, such as triggering the local alarm system
  • the value traffic light (traffic light) of the description field of the mth object determine the description information of the mth point cloud object in the ith sample group as highlighted information, and respond to the highlighted information, such as highlighting in the point cloud media Display the traffic light
  • SOS SOS
  • the point cloud media includes multiple media frames, the multiple media frames are encapsulated into N sample groups, and each sample group includes at least one media frame (here, including at least one media frame refers to encapsulation There is at least one media frame), the point cloud object in the ith sample group exists in the media frame in the ith sample group, and all the media frames in the ith sample group constitute a set that can be independently encoded and decoded.
  • the first electronic device acquires a description signaling file (MPD) sent by the second electronic device, and the description signaling file includes at least one package file description information of the point cloud media; when describing the target package in the signaling file
  • MPD description signaling file
  • the description information of the target package file may include description information of the media content contained in the target package file, description information of the window, and signaling information related to the presentation of the media content contained in the target package file.
  • the sample group and the point cloud object indication information of the i-th sample group The first electronic device obtains the point cloud object indication information of the ith sample group from the received target package file, and independently decodes the ith sample group according to the point cloud object indication information of the ith sample group to obtain the ith sample group at least one media frame within.
  • the above-mentioned independent decoding of the i-th sample group according to the point cloud object indication information of the i-th sample group can be implemented in this way to obtain at least one media frame in the i-th sample group: according to the i-th sample group
  • the point cloud object indication information of the i-th sample group decapsulates the i-th sample group to obtain an encoded bit stream; performs decoding processing on the encoded bit-stream to obtain at least one media frame in the i-th sample group.
  • the samples in the i-th sample group are samples obtained by encapsulating the encoded bit stream, where the encoded bit stream may be a GPCC bit stream or a VPCC bit stream.
  • the ith sample group can be decapsulated according to the point cloud object indication information of the ith sample group to obtain the coded bit stream, and then the coded bit stream can be decoded to obtain at least one media frame in the ith sample group. , in this way, the restoration of point cloud data can be realized.
  • the encoded bit stream includes a geometric bit stream and an attribute bit stream; the geometric bit stream and the attribute bit stream are used to be encapsulated together into the same sample in the ith sample group, or used to be encapsulated into the ith sample group respectively In different samples within; the above-mentioned decoding processing of the coded bit stream can be implemented in this way to obtain at least one media frame in the i-th sample group: decoding the geometric bit stream is performed to obtain the i-th sample group. geometric information of at least one media frame; decoding the attribute bitstream according to the geometric information of at least one media frame in the i-th sample group to obtain attribute information of at least one media frame in the i-th sample group.
  • the geometric bit stream may be decoded first to obtain the geometric information of at least one media frame in the i-th sample group, that is, the geometric information of points in the point cloud data included in the at least one media frame . Then, decode the attribute bitstream according to the geometric information of at least one media frame in the ith sample group to obtain attribute information of at least one media frame in the ith sample group, that is, in the point cloud data included in the at least one media frame In this way, the decoding process of the encoded bit stream is completed, and the complete information of at least one media frame in the i-th sample group is obtained.
  • the encoded bit stream can be encapsulated into a single file track, that is, the geometric bit stream and the attribute bit stream are used to jointly encapsulate into the same sample; the encoded bit stream can also be encapsulated into multiple file tracks, that is, the geometric bit stream and the attribute bit stream.
  • the bitstream is used to pack separately into different samples.
  • the method further includes: performing any one of the following processing: obtaining the viewing direction of the current user from the description signaling file, and performing the following operations on at least one of the i-th sample groups according to the viewing direction of the current user Perform rendering processing on the media frame; perform tracking processing on the current user to obtain the viewing direction of the current user, and perform rendering processing on at least one media frame in the i-th sample group according to the viewing direction of the current user; wherein, the tracking processing includes head tracking processing and at least one of visual tracking processing.
  • the at least one media frame may be rendered according to the viewing (window) direction of the current user, that is, the point cloud data included in the at least one media frame may be rendered to achieve For the presentation of the visual scene corresponding to the point cloud data, in this way, the accuracy of the rendering can be ensured and the viewing needs of the current user can be met.
  • the ways of obtaining the viewing direction of the current user include but are not limited to the following two: 1) obtaining the viewing direction of the current user from the description signaling file; 2) tracking the current user to obtain the viewing direction of the current user, wherein,
  • the tracking processing includes at least one of head tracking processing and visual tracking processing.
  • various point cloud objects in the point cloud media and the attributes of the point cloud objects may be indicated according to the point cloud object indication information of the i-th sample group of the point cloud media,
  • the transmission strategy of the point cloud media can be flexibly determined, effectively improving the point cloud media under certain network conditions.
  • the transmission efficiency is improved, and the analysis and processing efficiency of the point cloud media by the first electronic device can also be effectively improved, thereby bringing a better experience for the consumption of the point cloud media.
  • FIG. 4 shows a schematic flowchart of a data processing method for point cloud media provided by an embodiment of the present application. 102) Execute, the data processing method of the point cloud media comprises the following steps S401 to S402:
  • Step S401 generating point cloud object indication information of the ith sample group of the point cloud media, the point cloud media includes N sample groups, the ith sample group is any one of the N sample groups, and the ith sample group includes the point cloud.
  • Object, N and i are both positive integers and i ⁇ [1, N].
  • the point cloud media includes multiple media frames packed into N sample groups.
  • the second electronic device uses an object recognition algorithm (such as a target recognition algorithm, an image recognition algorithm, an image processing algorithm, etc.) to perform object recognition on each media frame of the point cloud media; when it is recognized that at least one media frame of the point cloud media contains
  • an object recognition algorithm such as a target recognition algorithm, an image recognition algorithm, an image processing algorithm, etc.
  • the second electronic device uses an object recognition algorithm to perform object recognition on each media frame of the point cloud media; when it is recognized that at least one media frame of the point cloud media contains
  • a point cloud object is used, at least one media frame identified as including the point cloud object is encapsulated into the i-th sample group, and all media frames in the i-th sample group form a set that can be independently encoded and decoded; at the same time, the second electronic device
  • the other media frames identified in the point cloud media that do not contain point cloud objects are respectively encapsulated into other sample groups except the i-th sample group among
  • the second electronic device may also encapsulate the media frames that contain point cloud objects in the point cloud media into P sample groups, respectively, and encapsulate the media frames that do not contain point cloud objects in the point cloud media into P sample groups.
  • P is a positive integer greater than 1, and P ⁇ N.
  • the point cloud object indication information of the ith sample group includes an object quantity field, and the object quantity field is used to indicate the number of point cloud objects included in the ith sample group; the second electronic device identifies the ith sample The number of point cloud objects in the group, and the object number field in the point cloud object indication information of the i-th sample group is configured according to the number of point cloud objects in the i-th sample group. For example, the second electronic device recognizes that the ith sample group includes M point cloud objects, and configures the value of the number of objects field in the point cloud object indication information of the ith sample group as M.
  • the ith sample group contains M point cloud objects, and each point cloud object in the M point cloud objects corresponds to a priority, then the M point cloud objects correspond to M priorities in total, and M is a positive integer; the point cloud object indication information of the ith sample group includes an object priority field, and the object priority field is used to indicate the priority of the ith sample group.
  • the second electronic device determines the highest priority among the M priorities, and configures the object priority field in the point cloud object indication information of the i-th sample group according to the highest priority, wherein the value of the object priority field is The smaller the value is, the higher the priority of the i-th sample group is, and the less likely the i-th sample group is to be discarded during transmission; the larger the value of the object priority field, the higher the priority of the i-th sample group. The lower it is, the more likely it is that the i-th sample group will be discarded during transmission.
  • the i-th sample group contains 3 point cloud objects, namely the first point cloud object, the second point cloud object and the third point cloud object.
  • the priority of the first point cloud object is higher than that of the second point cloud object.
  • Priority the priority of the second point cloud object is higher than the priority of the third point cloud object; the higher the priority corresponding to the point cloud object, the smaller the priority value of the point cloud object, the priority corresponding to the first point cloud object
  • the value is 0, the second point cloud object corresponds to the priority value of 1, and the third point cloud object corresponds to the priority value of 2; the highest priority among the three priorities corresponding to the three point cloud objects is the priority of the first point cloud object , then the priority of the i-th sample group is the priority of the first point cloud object, and the second electronic device prioritizes the objects in the point cloud object indication information of the i-th sample group according to the priority value corresponding to the first point cloud object
  • the value of the field is configured as 0.
  • the ith sample group contains M point cloud objects, where M is a positive integer;
  • the point cloud object indication information of the ith sample group contains M object type fields, and the M object type fields are used to respectively Indicates the types of M point cloud objects, and the values of the object type fields corresponding to different types of point cloud objects are different.
  • the mth point cloud object is any one of the M point cloud objects
  • the mth object type field is used to indicate the type of the mth point cloud object, m is a positive integer and m ⁇ [1,M].
  • the second electronic device identifies the type of the mth point cloud object in the ith sample group, and configures the mth object type in the point cloud object indication information of the ith sample group according to the type of the mth point cloud object field. For example, if the second electronic device recognizes that the type of the mth point cloud object in the ith sample group is an abnormal situation in the scene, then according to the type of the mth point cloud object, the mth point cloud object in the point cloud object indication information of the ith sample group is assigned The value of the object type field is configured to be 0.
  • the point cloud object indication information of the ith sample group includes an object scene field, and the object scene field is used to indicate the application scene to which the point cloud object included in the ith sample group belongs; the second electronic device obtains the first The application scene to which the point cloud object in the i sample group belongs, and the object scene field in the point cloud object indication information of the i th sample group is configured according to the application scene to which the point cloud object in the i th sample group belongs.
  • the second electronic device obtains that the application scene to which the point cloud object in the ith sample group belongs is a high-precision map scene, then according to the application scene to which the point cloud object in the ith sample group belongs, the The value of the object scene field in the point cloud object indication information is configured to be 0.
  • the ith sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the ith sample group contains M object description fields, and M object type fields are used to respectively Indicates the description information of M point cloud objects.
  • the mth point cloud object is any one of the M point cloud objects
  • the mth object description field is used to indicate the description information of the mth point cloud object, m is a positive integer and m ⁇ [1, M].
  • the second electronic device acquires the description information of the mth point cloud object in the ith sample group, and configures the mth point cloud object in the point cloud object indication information of the ith sample group according to the description information of the mth point cloud object Object description field.
  • the description information of the m-th point cloud object in the i-th sample group obtained by the second electronic device is alarm information
  • the point cloud object of the i-th sample group is indicated according to the obtained description information of the m-th point cloud object.
  • the value of the mth object description field in the information is configured as alarm.
  • Step S402 Send the point cloud object indication information of the ith sample group to the first electronic device, so that the first electronic device parses the point cloud media according to the point cloud object indication information of the ith sample group.
  • the second electronic device generates a description signaling file, and the description signaling file includes at least one package file description information of the point cloud media; the second electronic device sends the description signaling file to the first electronic device, and Receive an acquisition request sent by the first electronic device, where the acquisition request carries the description information of the selected target package file in the description signaling file; the second electronic device sends the target package file to the first electronic device according to the acquisition request, and the target package file includes The ith sample group and the point cloud object indication information of the ith sample group, so that the first electronic device parses the ith sample group according to the point cloud object indication information of the ith sample group.
  • the second electronic device in the process of transmitting the point cloud media to the first electronic device by the second electronic device, when the second electronic device detects that the discarding condition is satisfied, the second electronic device performs the method according to each sample group contained in the point cloud media. According to the priority indicated by the object priority field in the object indication information, at least part of the sample groups in the point cloud media are discarded according to the priority of each sample group from low to high, and the point cloud for which at least part of the sample groups has been discarded is discarded.
  • the media is repackaged and then sent to the first electronic device, wherein the discarding conditions include at least one of the following: network congestion, limited storage space of the first electronic device (for example, less than a storage space threshold), limited processing capability of the first electronic device (such as less than the processing capacity threshold).
  • the transmission network in which the second electronic device transmits (sends) the point cloud media to the first electronic device includes multiple intermediate nodes, when the first intermediate node (any intermediate node in the transmission network) detects When the network is congested, the first intermediate node discards at least part of the sample groups in the point cloud media according to the priority of each sample group from low to high, and repackages the point cloud media from which at least part of the sample groups have been discarded, and sends it to The second intermediate node (any intermediate node other than the first intermediate node in the transmission network).
  • the transmission network through which the second electronic device transmits the point cloud media to the first electronic device may be a content delivery network (Content Delivery Network, CDN).
  • the point cloud media includes multiple media frames; the second electronic device encodes at least one media frame including the point cloud object among the multiple media frames to obtain an encoded bit stream; and encapsulates the encoded bit stream processing to obtain the i-th sample group.
  • the second electronic device may perform encoding processing on the at least one media frame to obtain an encoded bitstream, such as a GPCC bitstream or a VPCC bitstream. Then, encapsulation processing is performed on the encoded bit stream to obtain the i-th sample group, wherein the encapsulation processing may refer to encapsulation into a single file track or multiple file tracks.
  • the content of the point cloud media can be determined according to the According to the priority indicated by the object priority field in the object indication information of each sample group, at least some sample groups in the point cloud media are discarded according to the priority of each sample group from low to high, and at least some samples that have been discarded are discarded.
  • the point cloud media in the group is repackaged and sent to the first electronic device, thereby saving transmission bandwidth and further improving the transmission efficiency and transmission success rate of the point cloud media.
  • the second electronic device is a drone
  • the application scenario is a real-time inspection scenario.
  • the drone captures point cloud media when overhauling high-voltage wire nodes, and the point cloud media contains multiple media frames.
  • the drone performs object recognition on each media frame contained in the point cloud media, and recognizes that all media frames contained in the point cloud media contain one point cloud object, which is an abnormal high-voltage wire node, and the point cloud object
  • the type of is a scene exception, and the priority value corresponding to the point cloud object is 0.
  • the drone encapsulates all media frames contained in the point cloud media into a sample group (that is, the target sample group), and all the media frames in the target sample group constitute a gather.
  • the UAV also generates point cloud object indication information of the target sample group. Specifically, the UAV converts the object scene in the point cloud object indication information of the target sample group according to the application scene to which the point cloud object in the target sample group belongs.
  • the value of the field is configured as 1; according to the priority value corresponding to the point cloud object in the target sample group, the value of the object priority field in the point cloud object indication information of the target sample group is configured as 0;
  • the number of point cloud objects in the target sample group set the value of the object number field in the point cloud object indication information of the target sample group to 1;
  • the value of the object type field in the point cloud object indication information is configured as 0; according to the obtained description information of the point cloud object in the target sample group, the value of the object description field in the point cloud object indication information of the target sample group is set.
  • the value is configured as alarm.
  • Table 4 shows the correspondence between each field contained in the point cloud object indication information of the target sample group and the value of each field.
  • the UAV encapsulates the point cloud object indication information of the target sample group into the encapsulation file of the point cloud media, and transmits the encapsulation file of the point cloud media to the terminal (for example, the terminal used by the maintenance personnel).
  • the terminal obtains the target sample group and the point cloud object indication information of the target sample group from the package file of the point cloud media, and parses the target sample group according to the point cloud object indication information of the target sample group to obtain all media contained in the target sample group frame.
  • the terminal determines that the description information of the point cloud object of the target sample group is alarm information according to the value of the object description field included in each media frame and the point cloud object indication information of the target sample group obtained by analysis.
  • the terminal responds to the alarm information and triggers the system alarm of the terminal, prompting the maintenance personnel to go to the corresponding location to repair the abnormal high-voltage wire node.
  • the second electronic device is a drone
  • the application scene is a high-precision map scene.
  • the drone captures the high-precision map material to obtain point cloud media, and the point cloud media contains 60 media frames.
  • the drone performs object recognition on each media frame contained in the point cloud media, and it is recognized that the 30 media frames contained in the point cloud media contain 2 point cloud objects, and the other 30 media frames do not contain point cloud objects;
  • the point cloud objects are the first point cloud object and the second point cloud object respectively;
  • the first point cloud object is a traffic light, the type of the first point cloud object is the scene indicator object, and the priority value corresponding to the first point cloud object is 0;
  • the second point cloud object is a vehicle, the type of the second point cloud object is reserved, the priority value corresponding to the second point cloud object is 2, and the priority of the first point cloud object is higher than that of the second point cloud object class.
  • the drone encapsulates 30 media frames containing point cloud objects in the point cloud media into the first sample group, and the other 30 media frames that do not contain point cloud objects in the point cloud media.
  • the frames are packaged into the second sample group, all media frames in the first sample group form a set that can be independently encoded and decoded, and all media frames in the second sample group also form a set that can be independently encoded and decoded.
  • the UAV also generates the point cloud object indication information of the first sample group. Specifically, the UAV converts the point cloud objects of the first sample group according to the application scenario to which the point cloud objects in the first sample group belong.
  • the value of the object scene field in the indication information is configured to be 0; according to the priority value corresponding to the first point cloud object in the first sample group, the object priority in the indication information of the point cloud object of the first sample group is set.
  • the value of the field is configured to be 0; according to the identified number of point cloud objects in the first sample group, the value of the object number field in the point cloud object indication information of the first sample group is configured to be 2; according to The type of the first point cloud object in the first sample group, configure the value of the first object type field in the point cloud object indication information of the first sample group to 1, according to the first sample group Two types of point cloud objects, configure the value of the second object type field in the point cloud object indication information of the first sample group as other; according to the obtained first point cloud object in the first sample group description information, configure the value of the object description field in the first point cloud object indication information of the first sample group as traffic light; according to the acquired description information of the second point cloud object in the first sample group, The value of the object description field in
  • the drone encapsulates the point cloud object indication information of the first sample group into the encapsulation file of the point cloud media, and encapsulates the point cloud media encapsulation file (including the first sample group, the second sample group and the first sample group point cloud object indication information) is transmitted to the terminal.
  • the transmission network in which the UAV transmits the package file of the point cloud media to the terminal includes an intermediate node (ie, the target intermediate node). If the target intermediate node detects that the transmission network is congested, the target intermediate node discards the second sample group according to the priority of the first sample group and the second sample group from low to high, and discards the point cloud media of the second sample group. It is repackaged and sent to the terminal.
  • the terminal obtains the first sample group and the point cloud object indication information of the first sample group from the package file of the point cloud media, and parses the first sample group according to the point cloud object indication information of the first sample group, 30 media frames included in the first sample group are obtained.
  • the terminal determines the description of the first point cloud object of the first sample group according to the 30 media frames obtained by analysis and the value traffic light of the first object description field included in the point cloud object indication information of the first sample group Information is highlighted information.
  • the terminal responds to the highlighted information and highlights the traffic light in the point cloud media.
  • FIG. 5 shows a schematic structural diagram of a data processing apparatus for point cloud media provided by an embodiment of the present application.
  • the data processing apparatus 50 for point cloud media can be used to execute the point cloud media shown in FIG. 3.
  • the corresponding steps in the data processing method, the data processing device 50 of the point cloud media includes the following units:
  • the obtaining unit 501 is configured to obtain the point cloud object indication information of the ith sample group of the point cloud media, the point cloud media includes N sample groups, and the ith sample group is any one of the N sample groups; Including point cloud objects, the point cloud object indication information of the ith sample group is used to indicate the attributes of the point cloud objects contained in the ith sample group, N and i are both positive integers and i ⁇ [1, N]; processing unit 502 , which is configured to parse the point cloud media according to the point cloud object indication information of the i-th sample group.
  • the point cloud object indication information of the ith sample group includes at least one of an object priority field and an object quantity field; wherein, the object priority field is used to indicate the priority of the ith sample group; The object number field is used to indicate the number of point cloud objects contained within the i-th sample group.
  • the jth sample group is any one of the N sample groups except the ith sample group, j is a positive integer and j ⁇ [1,N]; the priority of the ith sample group is higher than that of the ith sample group The priority of the j sample group; the processing unit 502 is further configured to: parse the ith sample group according to the point cloud object indication information of the ith sample group; after parsing the ith sample group, parse the j th sample group.
  • the processing unit 502 is further configured to: when the jth sample group does not include point cloud objects, determine that the priority of the ith sample group is higher than the priority of the jth sample group; The group includes point cloud objects, and the priority indicated by the object priority field contained in the point cloud object indication information of the ith sample group is higher than that indicated by the object priority field contained in the point cloud object indication information of the jth sample group.
  • the priority it is determined that the priority of the i-th sample group is higher than that of the j-th sample group.
  • the point cloud object indication information of the ith sample group includes an object scene field
  • the object scene field is used to indicate the application scene to which the point cloud object included in the ith sample group belongs, and different application scenes correspond to objects Different values of the scene field
  • the processing unit 502 is further configured to: read the object scene field in the point cloud object indication information of the ith sample group, and determine the point cloud in the ith sample group according to the value of the object scene field The application scenario to which the object belongs.
  • the ith sample group contains M point cloud objects, where M is a positive integer;
  • the point cloud object indication information of the ith sample group contains M object type fields, and the M object type fields are used to respectively Indicate the types of the M point cloud objects; the values of the object type fields corresponding to different types of point cloud objects are different;
  • the processing unit 502 is further configured to: read the m th object in the point cloud object indication information of the i th sample group type field, and determine the type of the mth point cloud object in the ith sample group according to the value of the mth object type field; wherein, the mth point cloud object is any one of the M point cloud objects, and the mth object type The field is used to indicate the type of the mth point cloud object;
  • m is a positive integer and m ⁇ [1,M].
  • the ith sample group contains M point cloud objects, where M is a positive integer;
  • the point cloud object indication information of the ith sample group contains M object description fields, and the M object description fields are used to respectively Indicate the description information of the M point cloud objects;
  • the processing unit 502 is further configured to: read the m th object description field in the point cloud object indication information of the i th sample group, and determine the m th object description field according to the value of the m th object description field The description information of the mth point cloud object in the i sample group, and respond to the description information; wherein, the mth point cloud object is any one of the M point cloud objects, and the mth object description field is used to indicate the mth point cloud object description information; m is a positive integer and m ⁇ [1, M];.
  • the point cloud media includes multiple media frames, the multiple media frames are encapsulated into N sample groups, and each sample group includes at least one media frame; the point cloud object in the i-th sample group exists in the media frames in the i-th sample group; all media frames in the i-th sample group form a set that can be independently encoded and decoded;
  • the acquiring unit 501 is further configured to: acquire the description signaling file sent by the second electronic device,
  • the description signaling file includes at least one package file description information of the point cloud media; when the target package file description information in the description signaling file is selected, an acquisition request carrying the target package file description information is sent to the second electronic device, and receive the target package file sent by the second electronic device according to the acquisition request; obtain the point cloud object indication information of the i-th sample group from the target package file;
  • the processing unit 502 is further configured to: independently decode the ith sample group according to the point cloud object indication information of the ith sample group to obtain at least one media frame in the ith sample group.
  • each unit in the data processing apparatus 50 for point cloud media shown in FIG. 5 may be respectively or all merged into one or several other units to form, or some unit(s) among them. It can also be divided into multiple units with smaller functions, which can realize the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the above-mentioned units are divided based on logical functions.
  • the function of one unit may also be implemented by multiple units, or the functions of multiple units may be implemented by one unit.
  • the data processing apparatus 50 for point cloud media may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented by cooperation of multiple units.
  • a general-purpose computing device including a general-purpose computer such as a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), etc.
  • a general-purpose computer may be implemented Run a computer program (including program code) capable of executing the steps involved in the corresponding method as shown in FIG. 3 , to construct a data processing device 50 for point cloud media as shown in FIG. 5 , and to implement the present application
  • the computer program can be recorded on, for example, a computer-readable storage medium, and loaded into the content consumption device 101 of the data processing system of the point cloud media shown in FIG. 1 through the computer-readable storage medium, and executed therein.
  • FIG. 6 shows a schematic structural diagram of a data processing apparatus for point cloud media provided by an embodiment of the present application.
  • the data processing apparatus 60 for point cloud media can be used to execute the point cloud media shown in FIG. 4.
  • the corresponding steps in the data processing method, the data processing device 60 of the point cloud media includes the following units:
  • the processing unit 601 is configured to generate point cloud object indication information of the ith sample group of point cloud media, the point cloud media includes N sample groups, and the ith sample group is any one of the N sample groups; Including point cloud objects, the point cloud object indication information of the ith sample group is used to indicate the attributes of the point cloud objects contained in the ith sample group, N and i are both positive integers and i ⁇ [1, N]; transmission unit 602 is configured to send the point cloud object indication information of the ith sample group to the first electronic device, so that the first electronic device parses the point cloud media according to the point cloud object indication information of the ith sample group.
  • the point cloud media includes multiple media frames; the processing unit 601 is further configured to: perform object recognition on each media frame of the point cloud media; when it is identified that at least one media frame of the point cloud media contains a point
  • the processing unit 601 is further configured to: perform object recognition on each media frame of the point cloud media; when it is identified that at least one media frame of the point cloud media contains a point
  • at least one media frame containing the point cloud object is encapsulated into the ith sample group, and the media frames in the point cloud media that do not contain the point cloud object are encapsulated into N sample groups, except for the ith sample group.
  • sample groups other than the ith sample group; wherein, all media frames in the i-th sample group constitute a set that can be independently encoded and decoded.
  • the processing unit 601 is further configured to: identify the number of point cloud objects in the ith sample group, and analyze the point cloud objects in the ith sample group according to the number of point cloud objects in the ith sample group
  • the object quantity field in the instruction information is configured and processed; the priority of the ith sample group is determined according to the number of point cloud objects in the ith sample group and the priority corresponding to the point cloud objects in the ith sample group, and according to the ith sample group.
  • the priority of the i sample group configures the object priority field in the point cloud object indication information of the i th sample group; wherein, the number of objects field is used to indicate the number of point cloud objects contained in the i th sample group; the object priority The level field is used to indicate the priority of the i-th sample group.
  • the number of point cloud objects in the ith sample group is M, and each point cloud object in the M point cloud objects corresponds to a priority, wherein M is a positive integer;
  • the processing unit 601 It is also configured as follows: when the value of M is 1, the priority corresponding to the point cloud object in the ith sample group is determined as the priority of the ith sample group; when the value of M is greater than 1, the ith sample is determined. The highest priority among the priorities corresponding to the M point cloud objects in the group is determined, and the highest priority is determined as the priority of the i-th sample group.
  • the processing unit 601 is further configured to: when it is detected that the discarding condition is satisfied, discard at least some samples in the point cloud media according to the priority of each sample group included in the point cloud media from low to high group; repackage the point cloud media from which at least part of the sample group has been discarded, and send it to the first electronic device; wherein, the discarding conditions include at least one of the following: network congestion, the storage space of the first electronic device is less than a storage space threshold, The processing capability of the first electronic device is less than the processing capability threshold.
  • the ith sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the ith sample group contains M object type fields and/or M object description fields;
  • the processing unit 601 is further configured to perform at least one of the following processes: identifying the type of the mth point cloud object in the ith sample group, and indicating information to the point cloud object of the ith sample group according to the type of the mth point cloud object.
  • the mth object type field in the configuration processing is performed; the description information of the mth point cloud object is obtained, and the mth object description field in the point cloud object indication information of the ith sample group is performed according to the description information of the mth point cloud object.
  • the mth point cloud object is any one of the M point cloud objects
  • the mth object type field is used to indicate the type of the mth point cloud object
  • the mth object description field is used to indicate the mth point cloud object. description information; m is a positive integer and m ⁇ [1, M].
  • the processing unit 601 is further configured to: obtain the application scene to which the point cloud object in the i-th sample group belongs, and perform an analysis on the i-th sample group according to the application scene to which the point cloud object in the i-th sample group belongs
  • the object scene field in the point cloud object indication information is configured and processed; wherein, the object scene field is used to indicate the application scene to which the point cloud object contained in the i-th sample group belongs.
  • the processing unit 601 is further configured to: send a description signaling file to the first electronic device, where the description signaling file includes at least one package file description information of the point cloud media; when receiving the first electronic device When sending the acquisition request carrying the description information of the target package file, send the target package file to the first electronic device, so that the first electronic device obtains the point cloud object indication information of the ith sample group from the target package file, and according to the first electronic device.
  • the point cloud object indication information of the i sample group independently decodes the i th sample group to obtain at least one media frame in the i th sample group.
  • each unit in the data processing apparatus 60 for point cloud media shown in FIG. 6 may be respectively or all merged into one or several other units to form, or some unit(s) among them. It can also be divided into multiple units with smaller functions, which can realize the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the above-mentioned units are divided based on logical functions.
  • the function of one unit may also be implemented by multiple units, or the functions of multiple units may be implemented by one unit.
  • the data processing apparatus 60 for point cloud media may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented by cooperation of multiple units.
  • a general-purpose computing device including a general-purpose computer such as a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), etc., and a general-purpose computer may be implemented
  • a computer program (including program code) capable of executing the steps involved in the corresponding method as shown in FIG. 4 is executed on the computer to construct a data processing device 60 for point cloud media as shown in FIG. 6 , and to realize the present application
  • the data processing method of the point cloud media of the embodiment can be recorded on, for example, a computer-readable storage medium, and loaded into the content production device 102 of the data processing system of the point cloud media shown in FIG. 1 through the computer-readable storage medium, and executed therein.
  • FIG. 7 shows a schematic structural diagram of a data processing device (or computer device) for point cloud media provided by an exemplary embodiment of the present application.
  • the data processing device 70 for point cloud media includes at least a processor 701 and a computer readable storage medium 702.
  • the processor 701 and the computer-readable storage medium 702 may be connected through a bus or other means.
  • the computer-readable storage medium 702 may be stored in a memory, and the computer-readable storage medium 702 is used for storing a computer program including computer instructions, and the processor 701 is used for executing the computer instructions stored in the computer-readable storage medium 702 .
  • the processor 701 is the computing core and the control core of the data processing device 70 of point cloud media, which is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to realize the corresponding method process or corresponding function.
  • the processor 701 may be a central processing unit (Central Processing Unit, CPU).
  • Embodiments of the present application further provide a computer-readable storage medium (Memory), where the computer-readable storage medium is a memory device in the data processing device 70 of the point cloud medium, used for storing programs and data.
  • the computer-readable storage medium 702 here may include both a built-in storage medium in the data processing device 70 for point cloud media, and of course an extended storage medium supported by the data processing device 70 for point cloud media.
  • the computer-readable storage medium provides storage space in which the operating system of the data processing device 70 of the point cloud media is stored.
  • one or more computer instructions suitable for being loaded and executed by the processor 701 are also stored in the storage space, and these computer instructions may be one or more computer programs (including program codes).
  • the computer-readable storage medium 702 here can be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, at least one storage medium located far away from the The computer-readable storage medium of the aforementioned processor 701 .
  • Non-Volatile Memory Non-Volatile Memory
  • the data processing device 70 for point cloud media may be the content consumption device 101 in the data processing system for point cloud media shown in FIG. 1 ;
  • the computer-readable storage medium 702 stores the first computer Instructions;
  • the processor 701 loads and executes the first computer instructions stored in the computer-readable storage medium 702 to implement corresponding steps in the method embodiment shown in FIG. 3 .
  • the data processing device 70 for point cloud media may be the content production device 102 in the data processing system for point cloud media shown in FIG. 1 ; the computer-readable storage medium 702 stores a second computer Instructions; the second computer instructions stored in the computer-readable storage medium 702 are loaded and executed by the processor 701 to implement corresponding steps in the method embodiment shown in FIG. 4 .
  • Embodiments of the present application provide a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing methods for point cloud media provided in the above-mentioned various optional manners.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本申请提供了一种点云媒体的数据处理方法、装置、设备及存储介质，该点云媒体的数据处理方法包括：获取点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个；第i样本组中包括点云对象，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；按照第i样本组的点云对象指示信息解析点云媒体。

Description

一种点云媒体的数据处理方法、装置、设备及存储介质
相关申请的交叉引用
本申请基于申请号为202011030289.3、申请日为2020年09月25日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及计算机技术领域，尤其涉及点云媒体技术领域，具体涉及一种点云媒体的数据处理方法、一种点云媒体的数据处理装置、一种点云媒体的数据处理设备以及一种计算机可读存储介质。
背景技术
随着科学技术的不断发展，目前已经能够以较低的成本、在较短的时间周期内获得大量高精度的点云数据，点云数据往往以点云媒体的形式在内容制作设备与内容消费设备之间进行传输。
点云媒体的传输过程具体如下：内容制作设备对点云媒体进行编码后，对编码后的点云媒体进行封装，得到点云媒体的封装文件，内容制作设备将点云媒体的封装文件传输给内容消费设备；内容消费设备对内容制作设备传输的点云媒体的封装文件进行解封装，然后再进行解码，最后内容消费设备呈现该媒体文件。由于点云媒体中包含的点云数据的数量较大，怎样提升点云媒体的解析处理效率，从而为点云媒体的消费带来较佳体验，是业界一直在持续解决的问题。
发明内容
本申请实施例提供一种点云媒体的数据处理方法，由第一电子设备执行，该点云媒体的数据处理方法包括：
获取点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个；第i样本组中包括点云对象，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；
按照第i样本组的点云对象指示信息解析点云媒体。
本申请实施例提供一种点云媒体的数据处理方法，由第二电子设备执行，该点云媒体的数据处理方法包括：
生成点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个；第i样本组中包括点云对象，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；
向第一电子设备发送第i样本组的点云对象指示信息，以使第一电子设备按照第i样本组的点云对象指示信息解析点云媒体。
本申请实施例提供一种点云媒体的数据处理装置，该点云媒体的数据处理装置包括：
获取单元，配置为获取点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个；第i样本组中包括点云对象，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，N、i均为正 整数且i∈[1，N]；
处理单元，配置为按照第i样本组的点云对象指示信息解析点云媒体。
本申请实施例提供一种点云媒体的数据处理装置，该点云媒体的数据处理装置包括：
处理单元，配置为生成点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个；第i样本组中包括点云对象，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；
传输单元，配置为向第一电子设备发送第i样本组的点云对象指示信息，以使第一电子设备按照第i样本组的点云对象指示信息解析点云媒体。
本申请实施例提供一种点云媒体的数据处理设备，该点云媒体的数据处理设备包括：
处理器，适于实现计算机指令；以及，
计算机可读存储介质，计算机可读存储介质存储有计算机指令，计算机指令适于由处理器加载并执行上述的点云媒体的数据处理方法。
本申请实施例提供一种计算机可读存储介质，该计算机可读存储介质存储有计算机指令，该计算机指令适于由处理器加载并执行上述的点云媒体的数据处理方法。
本申请实施例提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述的点云媒体的数据处理方法。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例提供的一种点云媒体的数据处理系统的架构示意图；
图2a示出了本申请实施例提供的一种点云媒体的数据处理架构的架构示意图；
图2b示出了本申请实施例提供的一种样本的结构示意图；
图2c示出了本申请实施例提供的一种包含多个文件轨道的容器的结构示意图；
图2d示出了本申请实施例提供的一种样本的结构示意图；
图3示出了本申请实施例提供的一种点云媒体的数据处理方法的流程示意图；
图4示出了本申请实施例提供的一种点云媒体的数据处理方法的流程示意图；
图5示出了本申请实施例提供的一种点云媒体的数据处理装置的结构示意图；
图6示出了本申请实施例提供的一种点云媒体的数据处理装置的结构示意图；
图7示出了本申请实施例提供的一种点云媒体的数据处理设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供一种点云媒体的数据处理方案。所谓点云(Point Cloud)是指空间中一组无规则分布的、表达三维物体或三维场景的空间结构及表面属性的离散点集。 点云数据(Point Cloud Data)是点云的具体记录形式，点云中每个点的点云数据可以包括几何信息(即三维位置信息)和属性信息，其中，点云中每个点的几何信息是指该点的笛卡尔三维坐标数据，点云中每个点的属性信息可以包括但不限于以下至少一种：色彩信息、材质信息、激光反射强度信息。这里，点云中的每个点可以具有相同数量的属性信息，例如，点云中的每个点都具有色彩信息和激光反射强度这两种属性信息；或者，点云中的每个点都具有色彩信息、材质信息和激光反射强度信息这三种属性信息。
在本申请实施例中，点云数据的获取途径可以包括但不限于以下至少一种：①计算机设备生成，计算机设备可以根据虚拟三维物体或虚拟三维场景生成点云数据；②三维(3-Dimension，3D)激光扫描获取，通过3D激光扫描可以获取静态现实世界三维物体或三维场景的点云数据，每秒可以获取百万级点云数据；③3D摄影测量获取，通过3D摄影设备(即一组摄像机，或具有多个镜头和传感器的摄像机设备)对现实世界的视觉场景进行采集以获取现实世界的视觉场景的点云数据，通过3D摄影可以获得动态现实世界三维物体或三维场景的点云数据；④通过医学设备获取生物组织器官的点云数据，在医学领域可以通过磁共振成像(Magnetic Resonance Imaging，MRI)、电子计算机断层扫描(Computed Tomography，CT)、电磁定位信息等医学设备获取生物组织器官的点云数据。
所谓点云媒体是指由点云数据形成的点云媒体文件，点云媒体中包括多个媒体帧，点云媒体中的每个媒体帧由点云数据组成。点云媒体可以灵活方便地表达三维物体或三维场景的空间结构及表面属性，因此被广泛应用。点云媒体的主要应用场景可以归为两大类别：第一类是机器感知点云，例如自主导航系统(Autonomous Navigation System，ANS)、实时巡检系统、地理信息系统(Geography Information System，GIS)、视觉分拣机器人、抢险救灾机器人等；第二类是人眼感知点云，例如数字文化遗产、自由视点广播、计算机辅助设计(Computer Aided Design，CAD)、三维沉浸通信、三维沉浸交互、生物组织器官三维重建等。
另外，本申请实施例提供的点云媒体的数据处理方案可以对点云媒体中包含的点云对象进行指示。点云对象是指存在于点云媒体的媒体帧中的对象，也就是从点云媒体的媒体帧中识别到的、某些应用场景下的特殊对象，点云对象的类型可以包括以下任一种：场景异常情况(又称场景异常对象，例如实时巡检场景中检查到的异常的高压电线节点、异常的路灯等)、场景指示对象(例如高精地图场景中识别到的交通信号灯、摄像头等)、目标对象(例如抢险救灾场景中识别到的等待救援的生物体，如人、狗等)。本申请实施例提供的点云媒体的数据处理方案在点云媒体的制作过程中，对点云媒体中包含点云对象的第i样本组进行指示，生成第i样本组的点云对象指示信息，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，属性包括以下至少一种：点云对象的数量、点云对象的类型、点云对象的描述信息、点云对象所属的应用场景以及点云对象的优先级。如此，在点云媒体的消费过程中，可以依据点云媒体的第i样本组的点云对象指示信息来指示点云媒体中的各种点云对象及点云对象的属性，这使得点云技术标准能够支持更丰富的应用场景；同时，依据点云对象指示信息所指示的属性，能够灵活确定点云媒体的传输策略，有效提升点云媒体在某些网络条件下的传输效率；还可有效提高第一电子设备对点云媒体的解析处理效率，从而为点云媒体的消费带来较佳体验。
基于上述描述，请参见图1，图1示出了本申请实施例提供的一种点云媒体的数据处理系统的架构示意图，以第一电子设备为内容消费设备，第二电子设备为内容制作设备为例，该点云媒体的数据处理系统10包括内容消费设备101和内容制作设备102。其中，内容制作设备102是指点云媒体的提供者(例如点云媒体的内容制作者)所使用的 计算机设备，该计算机设备可以是终端、服务器、可移动平台等具备点云媒体编码、封装能力的设备。内容消费设备101是指点云媒体的使用者(例如用户)所使用的计算机设备，该计算机设备可以是终端、可移动平台等具备点云媒体解封装、解码能力的设备。上述的终端可以是个人计算机(Personal Computer，PC)、智能移动设备(例如智能手机)、平板电脑、笔记本电脑、台式计算机、智能电视、智能手表、虚拟现实(Virtual Reality，VR)设备(例如VR头盔、VR眼镜)等，可移动平台可以是无人机(Unmanned Aerial Vehicle，UAV)、机器人等，对此不做限定。内容制作设备102和内容消费设备101可以通过有线通信或者无线通信的方式进行直接或间接地连接，本申请实施例在此不做限制。
图2a示出了本申请实施例提供的一种点云媒体的数据处理架构的架构示意图，下面将结合图1所示的点云媒体的数据处理系统以及图2a所示的点云媒体的数据处理架构，对本申请实施例提供的点云媒体的数据处理方案进行介绍，点云媒体的数据处理过程包括内容制作设备侧的数据处理过程以及内容消费设备侧的数据处理过程，具体处理过程如下：
一、内容制作设备侧的数据处理过程：
(1)点云数据的获取过程。
在一种实现方式中，从点云数据的获取方式看，点云数据的获取方式可以分为通过捕获设备采集真实世界的视觉场景以及通过计算机设备生成两种方式。在一种实现方式中，捕获设备可以是设置于内容制作设备中的硬件组件，例如捕获设备是终端的摄像头、传感器等。捕获设备也可以是与内容制作设备相连接的硬件装置，例如与服务器相连接的摄像头等。捕获设备用于为内容制作设备提供点云数据的获取服务，捕获设备可以包括但不限于以下任一种：摄像设备、传感设备、扫描设备；其中，摄像设备可以包括普通摄像头、立体摄像头、光场摄像头等；传感设备可以包括激光设备、雷达设备等；扫描设备可以包括3D激光扫描设备等。捕获设备的数量可以为多个，这些捕获设备被部署在现实空间中的一些特定位置以同时捕获该空间内不同角度的点云数据，捕获到的点云数据在时间上和空间上均保持同步。在一种实现方式中，计算机设备可以根据虚拟三维物体及虚拟三维场景生成点云数据。由于点云数据的获取方式不同，通过不同方式获取到的点云数据对应的压缩编码方式也可能有所区别。
(2)点云数据的编码及封装过程。
在一种实现方式中，内容制作设备可以采用基于几何的点云压缩(Geometry-Based Point Cloud Compression，GPCC)编码方式或者基于传统视频编码的点云压缩(Video-Based Point Cloud Compression，VPCC)编码方式对获取到的点云数据进行编码处理，得到点云数据的编码比特流，例如对应GPCC编码方式的GPCC比特流或者对应VPCC编码方式的VPCC比特流。这里，可以以媒体帧为单位，即对媒体帧进行编码处理，得到编码比特流。
在一种实现方式中，以GPCC编码方式为例，内容制作设备采用文件轨道对编码处理得到的GPCC比特流进行封装。所谓文件轨道是指GPCC比特流的封装容器，GPCC比特流可以封装在单个文件轨道中，也可以封装到多个文件轨道中，GPCC比特流封装在单个文件轨道中和封装在多个文件轨道中的示例情况如下：
①GPCC比特流封装在单个文件轨道中。当GPCC比特流在单个文件轨道中传输时，要求GPCC比特流根据单个文件轨道的传输规则进行声明并表示。例如GPCC比特流可以通过国际标准化组织基本媒体文件格式(International Organization for Standardization Base Media File Format，ISOBMFF)封装至单个文件轨道中。其中，封装在单个文件轨道中的每个样本(Sample)都包含一个或多个GPCC组件，所谓样本是指一个或多个点 云的封装结构集合，是点云媒体封装过程中的封装单位，点云媒体包含多个样本，一个样本通常表示点云媒体的一个媒体帧，每个样本由一个或多个类型-长度-值字节流格式(Type-Length-Value ByteStream Format，TLV)封装结构组成。图2b示出了本申请实施例提供的一种样本的结构示意图，如图2b所示，在进行单个文件轨道传输时，该文件轨道中的样本由GPCC参数集TLV、几何比特流TLV和属性比特流TLV组成，该样本被封装到单个文件轨道中。
②GPCC比特流封装在多个文件轨道中。当GPCC几何比特流和GPCC属性比特流在不同的文件轨道中进行传输时，文件轨道中的每个样本都包含至少一个TLV封装结构，该TLV封装结构中携带单个GPCC组件，并且TLV封装结构中不同时包含GPCC几何比特流和GPCC属性比特流。图2c示出了本申请实施例提供的一种包含多个文件轨道的封装容器的结构示意图，如图2c所示，在文件轨道1中传输的封装包1包含GPCC几何比特流，且不包含GPCC属性比特流；在文件轨道2中传输的封装包2包含GPCC属性比特流，且不包含GPCC几何比特流。由于内容消费设备在解码时首先对GPCC几何比特流进行解码，而GPCC属性比特流的解码取决于对GPCC几何比特流进行解码得到的几何信息，因此将不同的GPCC分量比特流封装在不同的文件轨道中，使得内容消费设备可以在GPCC属性比特流之前访问承载GPCC几何比特流的文件轨道。图2d示出了本申请实施例提供的一种样本的结构示意图，在进行多个文件轨道的传输时，GPCC几何比特流和GPCC属性比特流在不同的文件轨道中进行传输，图2d示出的文件轨道中的样本由GPCC参数集TLV、几何比特流TLV组成，且该样本中不包含属性比特流TLV，该样本被封装在多个文件轨道中的任一个文件轨道。
值得说明的是，上述的两种封装方式同样适用于VPCC比特流。
在一种实现方式中，获取到的点云数据经内容制作设备编码、封装后形成点云媒体的封装文件(例如若干个样本组)，该点云媒体的封装文件可以是整个媒体文件，也可以是媒体文件中的媒体片段。内容制作设备可以按照点云媒体的文件格式要求采用媒体呈现描述信息(即描述信令文件)(Media Presentation Description，MPD)记录该点云媒体的封装文件的元数据，此处的元数据是与点云媒体的呈现有关的信息的总称，该元数据可以包括对媒体内容的描述信息、对视窗的描述信息以及与媒体内容呈现相关的信令信息等。内容制作设备将MPD发送至内容消费设备，以使内容消费设备根据该MPD中的相关描述信息请求获取点云媒体的封装文件。例如，内容制作设备通过传输机制将点云媒体的封装文件和MPD发送至内容消费设备，其中，传输机制如动态自适应流媒体传输(Dynamic Adaptive Streaming over HTTP，DASH)、智能媒体传输(Smart Media Transport，SMT)等。
二、内容消费设备侧的数据处理过程：
(1)点云数据的解封装及解码过程。
在一种实现方式中，内容消费设备可以通过内容制作设备发送的MPD获取点云媒体的封装文件。内容消费设备端的文件解封装的过程与内容制作设备端的文件封装过程是相逆的，内容消费设备按照点云媒体的文件格式要求对点云媒体的封装文件进行解封装，得到编码比特流(即GPCC比特流或VPCC比特流)。内容消费设备端的解码过程与内容制作设备端的编码过程是相逆的，例如内容消费设备对编码比特流进行解码，以还原出点云数据。
(2)点云数据的渲染过程。
在一种实现方式中，内容消费设备根据MPD中与渲染、视窗相关的元数据(例如当前用户的观看(视窗)方向)对编码比特流解码得到的点云数据(如包含点云数据的媒体帧)进行渲染，渲染完成即实现了对点云数据对应的视觉场景的呈现。
本申请实施例中，对于内容制作设备端，首先通过捕获设备对真实世界的视觉场景进行采样，得到与真实世界的视觉场景对应的点云数据；然后通过GPCC编码方式或VPCC编码方式对获取的点云数据进行编码处理，得到GPCC比特流或VPCC比特流(这两种比特流中均包括几何比特流和属性比特流)；接着对GPCC比特流或者VPCC比特流进行封装得到点云媒体的封装文件(包括媒体文件或媒体片段)；内容制作设备还可以将元数据封装到媒体文件或媒体片段中，并通过传输机制(例如动态自适应流媒体传输机制)将点云媒体的封装文件发送至内容消费设备。对于内容消费设备端，首先接收内容制作设备发送的点云媒体的封装文件；然后对点云媒体的封装文件进行解封装处理，得到编码比特流(如GPCC比特流或者VPCC比特流)以及元数据；接着解析编码比特流中的元数据(即对编码比特流进行解码处理，得到点云数据)；最后基于当前用户的观看(视窗)方向，对解码得到的点云数据进行渲染，并显示在内容消费设备中，例如显示于内容消费设备所提供的人机交互界面中。需要说明的是，当前用户的观看(视窗)方向由头部跟踪和/或视觉跟踪功能确定。除了通过渲染器渲染当前用户的观看(视窗)方向的点云数据外，还可以通过音频解码器来对当前用户的观看(视窗)方向的音频进行解码优化。通过内容制作设备对采集到的点云数据进行编码、封装，实现了点云数据的有效存储；内容制作设备将封装得到的点云媒体的封装文件发送至内容消费设备，实现了点云数据的传输、发布和共享；内容消费设备对点云媒体的封装文件进行解封装、解码消费，使得真实世界的视觉场景在内容消费设备中得以呈现。
可以理解的是，本申请实施例描述的点云媒体的数据处理系统是为了更加清楚地说明本申请实施例的技术方案，并不构成对于本申请实施例提供的技术方案的限定，本领域普通技术人员可知，随着系统架构的演变和新业务场景的出现，本申请实施例提供的技术方案对于类似的技术问题同样适用。
由上述点云媒体的数据处理过程可知，内容制作设备需要对点云媒体进行编码、封装成点云媒体的封装文件后才能发送至内容消费设备，相应地，内容消费设备需要对点云媒体的封装文件进行解封装、解码后才能渲染呈现该点云媒体。在此基础上，本申请实施例提供的点云媒体的数据处理系统支持数据盒(Box)，例如ISOBMFF数据盒，数据盒是指包括元数据的数据块或包括元数据的对象，即数据盒中包含了点云媒体的元数据；点云媒体可以关联多个数据盒，例如，点云媒体包括N个样本组，点云媒体关联N个数据盒，第i样本组对应第i数据盒。本申请实施例对点云媒体的数据处理系统支持的数据盒进行扩展，将包含点云对象的第i样本组的点云对象指示信息封装在数据盒中；第i样本组的点云对象指示信息即数据盒中的点云对象指示组入口(PointCloudObjectIndicationGroupEntry)类，第i样本组的点云对象指示信息中包含以下至少一种字段：对象场景字段(object_sceiario)、对象优先级字段(object_priority)、对象数量字段(object_count)、对象类型字段(object_type)以及对象描述字段(object_description)；对象场景字段用于指示第i样本组内包含的点云对象所属的应用场景，对象优先级字段用于指示第i样本组的优先级，对象数量字段用于指示第i样本组内包含的点云对象的数量，对象类型字段用于指示第i样本组内包含的点云对象的类型，对象描述字段用于指示第i样本组内包含的点云对象的描述信息。数据盒中关于点云对象指示组入口类的语法可参见表1：
表1
Figure PCTCN2021115689-appb-000001
Figure PCTCN2021115689-appb-000002
上述表1所示的语法的语义如下①-⑤：
①对象场景字段object_scenario指示第i样本组中包含的点云对象所属的应用场景。不同应用场景下，对象场景字段的取值不同，对象场景字段的取值与应用场景之间的示例对应关系如表2所示，当对象场景字段的取值为0时，指示第i样本组中包含的点云对象所属的应用场景为高精地图场景；当对象场景字段的取值为1时，指示第i样本组中包含的点云对象所属的应用场景为实时巡检场景；当对象场景字段的取值为2时，指示第i样本组中包含的点云对象所属的应用场景为抢险救灾场景；需要说明的是，对象场景字段还存在其他扩展取值，也就是说，当点云对象存在于除上述三种应用场景(即高精地图场景、实时巡检场景、抢险救灾场景)外的其他应用场景时，只需对对象场景字段的取值进行扩展，即可对其他应用场景进行指示。
表2
对象场景字段的取值 含义
0 高精地图场景
1 实时巡检场景
2 抢险救灾场景
其他 保留
②对象优先级字段object_priority指示第i样本组的优先级。对象优先级字段的取值越小，则第i样本组的优先级越高，第i样本组在传输过程中被丢弃的可能性越小，第i样本组在解析时的解析顺序越靠前；对象优先级字段的取值越大，则第i样本组的优先级越低，第i样本组在传输过程中被丢弃的可能性越大，第i样本组在解析时的解析顺序越靠后。设第i样本组中包含M个点云对象，M为正整数，当M取值为1时，第i样本组内包含一个点云对象，且第i样本组内的点云对象对应一个优先级，则第i样本组的优先级等于第i样本组中包含的点云对象的优先级；当M取值大于1时，第i样本组内包含M个点云对象，M个点云对象中的每个点云对象分别对应一个优先级，则第i样本组的优先级等于M个优先级(即M个点云对象分别对应的优先级)中的最高优先级。
③对象数量字段object_count指示第i样本组内包含的点云对象的数量。设第i样本组中包含M个点云对象，则对象数量字段的取值为M，例如第i样本组内包含1个点云对象，则对象数量字段的取值为1；第i样本组内包含10个点云对象，则对象数量字段的取值为10。
④对象类型字段object_type指示第i样本组内包含的点云对象的类型。设第i样本组内包含M个点云对象，则第i样本组的点云对象指示信息中包含M个对象类型字段，M个对象类型字段用于分别指示M个点云对象的类型，例如第m点云对象是M个点云对象中的任一个，第m对象类型字段用于指示第m点云对象的类型。不同类型的点云对象对应的对象类型字段的取值不同，对象类型字段的取值与点云对象的类型之间的示 例对应关系如表3所示，当对象类型字段的取值为0时，指示点云对象的类型为场景异常情况(又称场景异常对象)；当对象类型字段的取值为1时，指示点云对象的类型为场景指示对象；当对象类型字段的取值为2时，指示点云对象的类型为目标对象；需要说明的是，对象类型字段还存在其他扩展取值，也就是说，当点云对象的类型为除上述三种类型(即场景异常情况、场景指示对象、目标对象)外的其他类型时，只需对对象类型字段的取值进行扩展，即可对其他类型进行指示。
表3
对象场景字段的取值 含义
0 场景异常情况
1 场景指示对象
2 目标对象
其他 保留
⑤对象描述字段object_description指示第i样本组内包含的点云对象的描述信息。设第i样本组内包含M个点云对象，则第i样本组的点云对象指示信息中包含M个对象描述字段，M个对象描述字段用于分别指示M个点云对象的描述信息，例如第m点云对象是M个点云对象中的任一个，第m对象描述字段的取值是以空字符结尾的8位元(Unicode Transformation Format-8，UTF-8)字符串，用于指示第m点云对象的描述信息。
结合表2可知，内容制作设备对点云媒体中包含点云对象的第i样本组进行指示后，生成第i样本组的点云对象指示信息，第i样本组的点云对象指示信息用于指示第i样本组包含的点云对象的属性(例如优先级、所属的应用场景、类型等等)；内容消费设备可以依据点云媒体的第i样本组的点云对象指示信息来指示点云媒体中的各种点云对象及点云对象的属性，从而解析点云媒体，这使得点云技术标准能够支持更丰富的应用场景；并且依据样本组的点云对象指示信息所指示的属性，能够灵活确定点云媒体的传输策略，有效提升点云媒体在某些网络条件下的传输效率，并且还可有效提高内容消费设备对点云媒体的解析处理效率；另外，在内容制作设备将点云媒体传输至内容消费设备的过程中，若检测到传输网络拥塞，可以根据点云媒体包含的各个样本组的对象指示信息中的对象优先级字段所指示的优先级，按照各个样本组的优先级由低至高的顺序丢弃点云媒体中的相应样本组，并对点云媒体进行重新封装后发送至内容消费设备，从而节省传输带宽，进一步提升点云媒体的传输效率和传输成功率；此外，点云对象指示信息中的对象场景字段和对象类型字段还存在其他扩展取值，进一步丰富了点云技术标准支持的应用场景以及点云对象的类型，从而为点云媒体的消费带来较佳体验。
基于上述描述，请参见图3，图3示出了本申请实施例提供的一种点云媒体的数据处理方法的流程示意图，该方法可以由第一电子设备(如图1所示实施例中的内容消费设备101)执行，该点云媒体的数据处理方法包括以下步骤S301至步骤S302：
步骤S301，获取点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个，第i样本组中包括点云对象，N、i均为正整数且i∈[1，N]。
步骤S302，按照第i样本组的点云对象指示信息解析点云媒体。
在一种实现方式中，第i样本组的点云对象指示信息(PointCloudObjectIndicationGroupEntry)用于指示第i样本组中包含的点云对象的属性，其中，属性可以包括以下至少一种：点云对象的数量、点云对象的类型、点云对象的描述信息、点云对象所属的应用场景以及点云对象的优先级。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象优先级字段 (object_priority)，对象优先级字段用于指示第i样本组的优先级。第i样本组的优先级可以根据第i样本组包含的点云对象的优先级进行确定，例如，设第i样本组中包含M个点云对象，M为正整数，当M取值为1时，第i样本组内包含一个点云对象，且第i样本组内的点云对象对应一个优先级，第i样本组的优先级等于第i样本组中包含的点云对象的优先级；当M取值大于1时，第i样本组内包含M个点云对象，M个点云对象中的每个点云对象分别对应一个优先级，第i样本组的优先级等于M个优先级中的最高优先级。
在一种实现方式中，第i样本组的点云对象指示信息中的对象优先级字段的取值越小，则第i样本组的优先级越高；第i样本组的点云对象指示信息中的对象优先级字段的取值越大，则第i样本组的优先级越低。若存在第一电子设备的存储空间有限(如存储空间小于存储空间阈值)和/或第一电子设备的处理能力有限(如处理能力小于处理能力阈值)等情况，第一电子设备可以优先解析点云媒体中优先级更高的样本组，即第i样本组的优先级越高，则第i样本组在解析时的解析顺序越靠前；第i样本组的优先级越低，则第i样本组在解析时的解析顺序越靠后。在此实现方式下，在点云媒体的解析过程中，可以优先对点云媒体中优先级更高的样本组进行解析，再对点云媒体中优先级更低的样本组进行解析。例如，第j样本组是N个样本组中除第i样本组外的任一个，j为正整数且j∈[1，N]，第j样本组中包括点云对象，并且第j样本组的点云对象指示信息中包含的对象优先级字段的取值大于第i样本组的点云对象指示信息中包含的对象优先级字段的取值，则第i样本组的优先级高于第j样本组的优先级，在点云媒体的解析过程中，优先按照第i样本组的点云对象指示信息解析第i样本组，再按照第j样本组的点云对象指示信息解析第j样本组。
在一种实现方式中，点云媒体中包含点云对象的样本组的优先级高于点云媒体中不包含点云对象的样本组的优先级。在此实现方式下，在点云媒体的解析过程中，优先对点云媒体中包含点云对象的样本组进行解析，再对点云媒体中不包含点云对象的样本组进行解析。例如，第j样本组是N个样本组中除第i样本组外的任一个，j为正整数且j∈[1，N]，并且第j样本组中不包含点云对象，第i样本组中包含点云对象，则第i样本组的优先级高于第j样本组的优先级，在点云媒体的解析过程中，优先按照第i样本组的点云对象指示信息解析第i样本组，再解析第j样本组。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象数量字段(object_count)，对象数量字段用于指示第i样本组内包含的点云对象的数量；设第i样本组中包含M个点云对象，则对象数量字段的取值为M。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象场景字段(object_sceiario)，对象场景字段用于指示第i样本组中包含的点云对象所属的应用场景，不同应用场景下，对象场景字段的取值不同。在此实现方式下，在点云媒体的解析过程中，读取第i样本组的点云对象指示信息中的对象场景字段，并根据对象场景字段的取值确定第i样本组内的点云对象所属的应用场景。其中，应用场景可以包括以下至少一种：地图场景(如高精地图场景)、实时巡检场景及抢险救灾场景。例如，根据第i样本组的点云对象指示信息中的对象场景字段的取值0，确定第i样本组内的点云对象所属的应用场景为高精地图场景。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象类型字段(object_type)，对象类型字段用于指示第i样本组中包含的点云对象的类型。例如，第i样本组内包含M个点云对象，M为正整数，第i样本组的点云对象指示信息中包含M个对象类型字段，M个对象类型字段用于分别指示M个点云对象的类型，不同类型的点云对象对应的对象类型字段的取值不同。设第m点云对象是M个点云对象中的任一 个，则第m对象类型字段用于指示第m点云对象的类型，m为正整数且m∈[1，M]。在此实现方式下，在点云媒体的解析过程中，读取第i样本组的点云对象指示信息中的第m对象类型字段，并根据第m对象类型字段的取值确定第i样本组内的第m点云对象的类型；其中，类型可以包括以下任一种：场景异常情况、场景指示对象及目标对象。例如，根据第i样本组的点云对象指示信息中的第m对象类型字段的取值0，确定第i样本组内的第m点云对象的类型为场景异常情况。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象描述字段(object_description)，对象描述字段用于指示第i样本组中包含的点云对象的描述信息。例如，第i样本组内包含M个点云对象，M为正整数，第i样本组的点云对象指示信息中包含M个对象描述字段，M个对象描述字段用于分别指示M个点云对象的描述信息。设第m点云对象是M个点云对象中的任一个，则第m对象描述字段的取值可以是以空字符结尾的8位元字符串，用于指示第m点云对象的描述信息，m为正整数且m∈[1，M]。在此实现方式下，在点云媒体的解析过程中，读取第i样本组的点云对象指示信息中的第m对象描述字段，并根据第m对象描述字段的取值确定第i样本组内的第m点云对象的描述信息，并响应描述信息；其中，描述信息可以包括以下至少一种：告警信息、突出显示信息及求救信息。例如，根据第m对象描述字段的取值alarm(报警)，确定第i样本组内的第m点云对象的描述信息为告警信息，并响应该告警信息，如触发本地告警系统；又如，根据第m对象描述字段的取值traffic light(交通信号灯)，确定第i样本组内的第m点云对象的描述信息为突出显示信息，并响应该突出显示信息，如在点云媒体中突出显示该交通信号灯；还如，根据第m对象描述字段的取值SOS(求救信号)，确定第i样本组内的第m点云对象的描述信息为求救信息，并响应该求救信息，如自动拨通救援电话。
在一种实现方式中，点云媒体包括多个媒体帧，多个媒体帧被封装至N个样本组中，每个样本组中包括至少一个媒体帧(这里的包括至少一个媒体帧是指封装有至少一个媒体帧)，第i样本组内的点云对象存在于第i样本组内的媒体帧中，第i样本组内的所有媒体帧构成一个能够被独立编解码的集合。在该情况下，第一电子设备获取第二电子设备发送的描述信令文件(MPD)，描述信令文件中包括点云媒体的至少一个封装文件描述信息；当描述信令文件中的目标封装文件描述信息被选择时，则向第二电子设备发送携带有目标封装文件描述信息的获取请求，以使第二电子设备根据获取请求返回目标封装文件。目标封装文件描述信息可以包括对目标封装文件中包含的媒体内容的描述信息、对视窗的描述信息以及与目标封装文件中包含的媒体内容呈现相关的信令信息等，目标封装文件中包括第i样本组以及第i样本组的点云对象指示信息。第一电子设备从接收到的目标封装文件中获取第i样本组的点云对象指示信息，并按照第i样本组的点云对象指示信息对第i样本组进行独立解码，得到第i样本组内的至少一个媒体帧。
在一种实现方式中，可以通过这样的方式来实现上述的按照第i样本组的点云对象指示信息对第i样本组进行独立解码，得到第i样本组内的至少一个媒体帧：按照第i样本组的点云对象指示信息对第i样本组进行解封装处理，得到编码比特流；对编码比特流进行解码处理，得到第i样本组内的至少一个媒体帧。这里，第i样本组中的样本是对编码比特流进行封装后得到的样本，其中，编码比特流可以是GPCC比特流或者VPCC比特流。因此，可以按照第i样本组的点云对象指示信息对第i样本组进行解封装处理，得到编码比特流，然后再对编码比特流进行解码处理，得到第i样本组内的至少一个媒体帧，如此，可以实现对点云数据的还原。
在一种实现方式中，编码比特流包括几何比特流及属性比特流；几何比特流及属性比特流用于共同封装至第i样本组内的同一样本中，或者用于分别封装至第i样本组内 的不同样本中；可以通过这样的方式来实现上述的对编码比特流进行解码处理，得到第i样本组内的至少一个媒体帧：对几何比特流进行解码处理，得到第i样本组内的至少一个媒体帧的几何信息；根据第i样本组内的至少一个媒体帧的几何信息对属性比特流进行解码处理，得到第i样本组内的至少一个媒体帧的属性信息。这里，在解码处理的过程中，可以首先对几何比特流进行解码处理，得到第i样本组内的至少一个媒体帧的几何信息，即至少一个媒体帧包括的点云数据中的点的几何信息。然后，根据第i样本组内的至少一个媒体帧的几何信息对属性比特流进行解码处理，得到第i样本组内的至少一个媒体帧的属性信息，即至少一个媒体帧包括的点云数据中的点的属性信息，如此，完成对编码比特流的解码处理，得到第i样本组内的至少一个媒体帧的完整信息。值得说明的是，编码比特流可以封装至单个文件轨道中，即几何比特流及属性比特流用于共同封装至同一样本中；编码比特流也可以封装至多个文件轨道中，即几何比特流及属性比特流用于分别封装至不同样本中。
在一种实现方式中，步骤302之后，还包括：执行以下任意一种处理：从描述信令文件中获取当前用户的观看方向，并根据当前用户的观看方向对第i样本组内的至少一个媒体帧进行渲染处理；对当前用户进行跟踪处理得到当前用户的观看方向，并根据当前用户的观看方向对第i样本组内的至少一个媒体帧进行渲染处理；其中，跟踪处理包括头部跟踪处理以及视觉跟踪处理中的至少一种。
在解码出第i样本组内的至少一个媒体帧后，可以根据当前用户的观看(视窗)方向对该至少一个媒体帧进行渲染，即对该至少一个媒体帧包括的点云数据进行渲染，实现对点云数据对应的视觉场景的呈现，如此，可以保证渲染的准确性，符合当前用户的观看需求。其中，当前用户的观看方向的获取方式包括但不限于以下两种：1)从描述信令文件中获取当前用户的观看方向；2)对当前用户进行跟踪处理得到当前用户的观看方向，其中，跟踪处理包括头部跟踪处理以及视觉跟踪处理中的至少一种。
本申请实施例中，在点云媒体的消费过程中，可以依据点云媒体的第i样本组的点云对象指示信息来指示点云媒体中的各种点云对象及点云对象的属性，这使得点云技术标准能够支持更丰富的应用场景；并且依据样本组的点云对象指示信息所指示的属性，能够灵活确定点云媒体的传输策略，有效提升点云媒体在某些网络条件下的传输效率，并且还可有效提高第一电子设备对点云媒体的解析处理效率，从而为点云媒体的消费带来较佳体验。
请参见图4，图4示出了本申请实施例提供的一种点云媒体的数据处理方法的流程示意图，该方法可以由第二电子设备(如图1所示实施例中的内容制作设备102)执行，该点云媒体的数据处理方法包括以下步骤S401至步骤S402：
步骤S401，生成点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个，第i样本组中包括点云对象，N、i均为正整数且i∈[1，N]。
在一种实现方式中，点云媒体包括多个媒体帧，多个媒体帧被封装至N个样本组中。例如，第二电子设备采用对象识别算法(例如目标识别算法、图像识别算法、图像处理算法等)对点云媒体的各个媒体帧进行对象识别；当识别出点云媒体的至少一个媒体帧中包含点云对象时，将识别出包括点云对象的至少一个媒体帧封装至第i样本组中，第i样本组内的所有媒体帧构成一个能够被独立编解码的集合；同时，第二电子设备将点云媒体中的识别出未包含点云对象的其他媒体帧分别封装至N个样本组中除第i样本组之外的其他样本组中。在一种实现方式中，第二电子设备也可以将点云媒体中包含点云对象的媒体帧分别封装至P个样本组中，将点云媒体中未包含点云对象的媒体帧分别封装至N个样本组中除P个样本组外的N-P个样本组中，P为大于1的正整数，且P≤N。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象数量字段，对象数量字段用于指示第i样本组内包含的点云对象的数量；第二电子设备识别第i样本组内的点云对象的数量，并根据第i样本组内的点云对象的数量配置第i样本组的点云对象指示信息中的对象数量字段。例如，第二电子设备识别到第i样本组内包括M个点云对象，则将第i样本组的点云对象指示信息中的对象数量字段的取值配置为M。
在一种实现方式中，第i样本组内包含M个点云对象，M个点云对象中每个点云对象分别对应一个优先级，则M个点云对象总共对应M个优先级，M为正整数；第i样本组的点云对象指示信息中包含对象优先级字段，对象优先级字段用于指示第i样本组的优先级。在该情况下，第二电子设备确定M个优先级中的最高优先级，并根据最高优先级配置第i样本组的点云对象指示信息中的对象优先级字段，其中，对象优先级字段的取值越小，则第i样本组的优先级越高，第i样本组在传输过程中被丢弃的可能性越小；对象优先级字段的取值越大，则第i样本组的优先级越低，第i样本组在传输过程中被丢弃的可能性越大。例如，第i样本组内包含3个点云对象，分别是第一点云对象、第二点云对象和第三点云对象，第一点云对象的优先级高于第二点云对象的优先级，第二点云对象的优先级高于第三点云对象的优先级；点云对象对应的优先级越高，点云对象的优先级数值越小，第一点云对象对应优先级数值0，第二点云对象对应优先级数值1，第三点云对象对应优先级数值2；3个点云对象对应的3个优先级中的最高优先级为第一点云对象的优先级，则第i样本组的优先级为第一点云对象的优先级，第二电子设备根据第一点云对象对应的优先级数值将第i样本组的点云对象指示信息中的对象优先级字段的取值配置为0。
在一种实现方式中，第i样本组内包含M个点云对象，M为正整数；第i样本组的点云对象指示信息中包含M个对象类型字段，M个对象类型字段用于分别指示M个点云对象的类型，不同类型的点云对象对应的对象类型字段的取值不同。设第m点云对象是M个点云对象中的任一个，则第m对象类型字段用于指示第m点云对象的类型，m为正整数且m∈[1，M]。在该情况下，第二电子设备识别第i样本组内的第m点云对象的类型，并根据第m点云对象的类型配置第i样本组的点云对象指示信息中的第m对象类型字段。例如，第二电子设备识别到第i样本组内的第m点云对象的类型为场景异常情况，则根据第m点云对象的类型将第i样本组的点云对象指示信息中的第m对象类型字段的取值配置为0。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象场景字段，对象场景字段用于指示第i样本组内包含的点云对象所属的应用场景；第二电子设备获取第i样本组内的点云对象的所属的应用场景，并根据第i样本组内的点云对象的所属的应用场景配置第i样本组的点云对象指示信息中的对象场景字段。例如，第二电子设备获取到第i样本组内的点云对象的所属的应用场景为高精地图场景，则根据第i样本组内的点云对象的所属的应用场景将第i样本组的点云对象指示信息中的对象场景字段的取值配置为0。
在一种实现方式中，第i样本组内包含M个点云对象，M为正整数；第i样本组的点云对象指示信息中包含M个对象描述字段，M个对象类型字段用于分别指示M个点云对象的描述信息。设第m点云对象是M个点云对象中的任一个，则第m对象描述字段用于指示第m点云对象的描述信息，m为正整数且m∈[1，M]。在该情况下，第二电子设备获取第i样本组内的第m点云对象的描述信息，并根据第m点云对象的描述信息配置第i样本组的点云对象指示信息中的第m对象描述字段。例如，第二电子设备获取到的第i样本组内的第m点云对象的描述信息为告警信息，则根据获取到的第m点云对象的描述信息将第i样本组的点云对象指示信息中的第m对象描述字段的取值配 置为alarm。
步骤S402，向第一电子设备发送第i样本组的点云对象指示信息，以使第一电子设备按照第i样本组的点云对象指示信息解析点云媒体。
在一种实现方式中，第二电子设备生成描述信令文件，描述信令文件中包括点云媒体的至少一个封装文件描述信息；第二电子设备向第一电子设备发送描述信令文件，并接收第一电子设备发送的获取请求，获取请求中携带描述信令文件中被选择的目标封装文件描述信息；第二电子设备根据获取请求向第一电子设备发送目标封装文件，目标封装文件中包括第i样本组以及第i样本组的点云对象指示信息，以使第一电子设备按照第i样本组的点云对象指示信息对第i样本组进行解析。
在一种实现方式中，第二电子设备将点云媒体传输至第一电子设备的过程中，当第二电子设备检测到满足丢弃条件时，第二电子设备根据点云媒体包含的各个样本组的对象指示信息中的对象优先级字段所指示的优先级，按照各个样本组的优先级由低至高的顺序丢弃点云媒体中的至少部分样本组，并对已丢弃至少部分样本组的点云媒体进行重新封装后发送至第一电子设备，其中，丢弃条件包括以下至少之一：网络拥塞、第一电子设备的存储空间有限(如小于存储空间阈值)、第一电子设备的处理能力有限(如小于处理能力阈值)。在一种实现方式中，第二电子设备将点云媒体传输(发送)至第一电子设备的传输网络中包括多个中间节点，当第一中间节点(传输网络中的任一个中间节点)检测到网络拥塞时，第一中间节点按照各个样本组的优先级由低至高的顺序丢弃点云媒体中的至少部分样本组，并对已丢弃至少部分样本组的点云媒体进行重新封装后发送至第二中间节点(传输网络中除第一中间节点之外的任一个中间节点)。其中，第二电子设备将点云媒体传输至第一电子设备的传输网络可以为内容分发网络(Content Delivery Network，CDN)。
在一种实现方式中，点云媒体包括多个媒体帧；第二电子设备对多个媒体帧中包括点云对象的至少一个媒体帧进行编码处理，得到编码比特流；对编码比特流进行封装处理，得到第i样本组。这里，对于多个媒体帧中包括点云对象的至少一个媒体帧，第二电子设备可以对该至少一个媒体帧进行编码处理，得到编码比特流，如GPCC比特流或者VPCC比特流。然后，对编码比特流进行封装处理，得到第i样本组，其中，封装处理可以是指封装至单个文件轨道或多个文件轨道。例如，在编码比特流包括几何比特流及属性比特流的情况下，封装至单个文件轨道可以是指将几何比特流及属性比特流共同封装至同一样本中，该同一样本用于作为第i样本组中的样本；封装至多个文件轨道可以是指将几何比特流及属性比特流分别封装至不同样本中，该不同样本用于作为第i样本组中的样本。
本申请实施例中，在第二电子设备将点云媒体传输至第一电子设备的过程中，当第二电子设备或传输网络中的中间节点检测到传输网络拥塞时，可以根据点云媒体包含的各个样本组的对象指示信息中的对象优先级字段所指示的优先级，按照各个样本组的优先级由低至高的顺序丢弃点云媒体中的至少部分样本组，并对已丢弃至少部分样本组的点云媒体进行重新封装后发送至第一电子设备，从而节省传输带宽，进一步提升点云媒体的传输效率和传输成功率。
下面结合图3实施例和图4实施例所描述的内容，针对具体的应用场景，对本申请实施例提供的点云媒体的数据处理方案举例描述。
例如，第二电子设备为无人机，应用场景为实时巡检场景，无人机在检修高压电线节点时拍摄得到点云媒体，点云媒体中包含多个媒体帧。无人机对点云媒体包含的各个媒体帧进行对象识别，识别到点云媒体包含的所有媒体帧中均包含1个点云对象，该点云对象为异常的高压电线节点，该点云对象的类型为场景异常情况，该点云对象对应的 优先级数值为0。无人机在点云媒体的制作过程中，将点云媒体包含的所有媒体帧封装至一个样本组(即目标样本组)中，目标样本组内的所有媒体帧构成一个能够被独立编解码的集合。无人机还生成目标样本组的点云对象指示信息，具体地，无人机根据目标样本组内的点云对象的所属的应用场景，将目标样本组的点云对象指示信息中的对象场景字段的取值配置为1；根据目标样本组内的点云对象对应的优先级数值，将目标样本组的点云对象指示信息中的对象优先级字段的取值配置为0；根据识别到的目标样本组内的点云对象的数量，将目标样本组的点云对象指示信息中的对象数量字段的取值配置为1；根据目标样本组内的点云对象的类型，将目标样本组的点云对象指示信息中的对象类型字段的取值配置为0；根据获取到的目标样本组内的点云对象的描述信息，将目标样本组的点云对象指示信息中的对象描述字段的取值配置为alarm。目标样本组的点云对象指示信息中包含的各个字段与各个字段的取值之间的对应关系如表4所示。
表4
字段 取值
对象场景字段object_sceiario 1
对象优先级字段object_priority 0
对象数量字段object_count 1
对象类型字段object_type 0
对象描述字段object_description alarm
无人机将目标样本组的点云对象指示信息封装至点云媒体的封装文件中，并将点云媒体的封装文件传输至终端(例如维修人员使用的终端)。终端从点云媒体的封装文件中获取目标样本组以及目标样本组的点云对象指示信息，并按照目标样本组的点云对象指示信息对目标样本组进行解析，得到目标样本组包含的所有媒体帧。终端根据解析得到的各个媒体帧以及目标样本组的点云对象指示信息中包含的对象描述字段的取值alarm，确定目标样本组的点云对象的描述信息为告警信息。终端响应该告警信息，并触发终端的系统告警，提示维修人员前往对应地点对异常的高压电线节点进行检修。
又如，第二电子设备为无人机，应用场景为高精地图场景，无人机在采集高精地图素材时拍摄得到点云媒体，点云媒体中包含60个媒体帧。无人机对点云媒体包含的各个媒体帧进行对象识别，识别到点云媒体包含的30个媒体帧中均包含2个点云对象，另外30个媒体帧中均不包含点云对象；2个点云对象分别为第一点云对象和第二点云对象；第一点云对象为交通信号灯，第一点云对象的类型为场景指示对象，第一点云对象对应的优先级数值为0；第二点云对象为车辆，第二点云对象的类型为保留，第二点云对象对应的优先级数值为2，第一点云对象的优先级高于第二点云对象的优先级。无人机在点云媒体的制作过程中，将点云媒体中包含点云对象的30个媒体帧封装至第一样本组中，将点云媒体中不包含点云对象的另外30个媒体帧封装至第二样本组中，第一样本组内的所有媒体帧构成一个能够被独立编解码的集合，第二样本组内的所有媒体帧同样构成一个能够被独立编解码的集合。无人机还生成第一样本组的点云对象指示信息，具体地，无人机根据第一样本组内的点云对象的所属的应用场景，将第一样本组的点云对象指示信息中的对象场景字段的取值配置为0；根据第一样本组内的第一点云对象对应的优先级数值，将第一样本组的点云对象指示信息中的对象优先级字段的取值配置为0；根据识别到的第一样本组内的点云对象的数量，将第一样本组的点云对象指示信息中的对象数量字段的取值配置为2；根据第一样本组内的第一点云对象的类型，将第一样本组的点云对象指示信息中的第一对象类型字段的取值配置为1，根据第一样本组内的第二点云对象的类型，将第一样本组的点云对象指示信息中的第二对象类型字段的取值配置为其他；根据获取到的第一样本组内的第一点云对象的描述信息，将第一样本组 的第一点云对象指示信息中的对象描述字段的取值配置为traffic light；根据获取到的第一样本组内的第二点云对象的描述信息，将第一样本组的第二点云对象指示信息中的对象描述字段的取值配置为其他。第一样本组的点云对象指示信息中包含的各个字段与各个字段的取值之间的对应关系如表5所示。
表5
字段 取值
对象场景字段object_sceiario 0
对象优先级字段object_priority 0
对象数量字段object_count 2
第一对象类型字段object_type 1
第一对象描述字段object_description traffic light
第二对象类型字段object_type 其他
第二对象描述字段object_description 其他
无人机将第一样本组的点云对象指示信息封装至点云媒体的封装文件中，并将点云媒体的封装文件(包含第一样本组、第二样本组以及第一样本组的点云对象指示信息)传输至终端。无人机将点云媒体的封装文件传输至终端的传输网络中包括一个中间节点(即目标中间节点)。若目标中间节点检测到传输网络拥塞，目标中间节点按照第一样本组与第二样本组的优先级由低至高的顺序丢弃第二样本组，并对已丢弃第二样本组的点云媒体进行重新封装后发送至终端。终端从点云媒体的封装文件中获取第一样本组以及第一样本组的点云对象指示信息，并按照第一样本组的点云对象指示信息对第一样本组进行解析，得到第一样本组包含的30个媒体帧。终端根据解析得到的30个媒体帧、以及第一样本组的点云对象指示信息中包含的第一对象描述字段的取值traffic light，确定第一样本组的第一点云对象的描述信息为突出显示信息。终端响应该突出显示信息，并在点云媒体中突出显示交通信号灯。
请参见图5，图5示出了本申请实施例提供的一种点云媒体的数据处理装置的结构示意图，该点云媒体的数据处理装置50可以用于执行图3所示的点云媒体的数据处理方法中的相应步骤，该点云媒体的数据处理装置50包括如下单元：
获取单元501，配置为获取点云媒体的第i样本组的点云对象指示信息，点云媒体包括N个样本组，第i样本组为N个样本组中的任一个；第i样本组中包括点云对象，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；处理单元502，配置为按照第i样本组的点云对象指示信息解析点云媒体。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象优先级字段以及对象数量字段中的至少之一；其中，对象优先级字段用于指示第i样本组的优先级；对象数量字段用于指示第i样本组内包含的点云对象的数量。
在一种实现方式中，第j样本组是N个样本组中除第i样本组外的任一个，j为正整数且j∈[1，N]；第i样本组的优先级高于第j样本组的优先级；处理单元502，还配置为：按照第i样本组的点云对象指示信息解析第i样本组；在解析第i样本组之后，解析第j样本组。
在一种实现方式中，处理单元502，还配置为：当第j样本组中不包括点云对象时，确定第i样本组的优先级高于第j样本组的优先级；当第j样本组中包括点云对象、且第i样本组的点云对象指示信息中包含的对象优先级字段指示的优先级高于第j样本组的点云对象指示信息中包含的对象优先级字段指示的优先级时，确定第i样本组的优先级高于第j样本组的优先级。
在一种实现方式中，第i样本组的点云对象指示信息中包含对象场景字段，对象场景字段用于指示第i样本组中包含的点云对象所属的应用场景，且不同应用场景对应对象场景字段的不同取值；处理单元502，还配置为：读取第i样本组的点云对象指示信息中的对象场景字段，并根据对象场景字段的取值确定第i样本组内的点云对象所属的应用场景。
在一种实现方式中，第i样本组内包含M个点云对象，M为正整数；第i样本组的点云对象指示信息中包含M个对象类型字段，M个对象类型字段用于分别指示M个点云对象的类型；不同类型的点云对象对应的对象类型字段的取值不同；处理单元502，还配置为：读取第i样本组的点云对象指示信息中的第m对象类型字段，并根据第m对象类型字段的取值确定第i样本组内的第m点云对象的类型；其中，第m点云对象是M个点云对象中的任一个，第m对象类型字段用于指示第m点云对象的类型；m为正整数且m∈[1，M]。
在一种实现方式中，第i样本组内包含M个点云对象，M为正整数；第i样本组的点云对象指示信息中包含M个对象描述字段，M个对象描述字段用于分别指示M个点云对象的描述信息；处理单元502，还配置为：读取第i样本组的点云对象指示信息中的第m对象描述字段，并根据第m对象描述字段的取值确定第i样本组内的第m点云对象的描述信息，并响应描述信息；其中，第m点云对象是M个点云对象中的任一个，第m对象描述字段用于指示第m点云对象的描述信息；m为正整数且m∈[1，M]；。在一种实现方式中，点云媒体包括多个媒体帧，多个媒体帧被封装至N个样本组中，每个样本组中包括至少一个媒体帧；第i样本组内的点云对象存在于第i样本组内的媒体帧中；第i样本组内的所有媒体帧构成一个能够被独立编解码的集合；获取单元501，还配置为：获取第二电子设备发送的描述信令文件，描述信令文件中包括点云媒体的至少一个封装文件描述信息；当描述信令文件中的目标封装文件描述信息被选择时，向第二电子设备发送携带有目标封装文件描述信息的获取请求，并接收第二电子设备根据获取请求发送的目标封装文件；从目标封装文件中获取第i样本组的点云对象指示信息；
处理单元502，还配置为：按照第i样本组的点云对象指示信息对第i样本组进行独立解码，得到第i样本组内的至少一个媒体帧。
根据本申请的一个实施例，图5所示的点云媒体的数据处理装置50中的各个单元可以分别或全部合并为一个或若干个另外的单元来构成，或者其中的某个(些)单元还可以再拆分为功能上更小的多个单元来构成，这可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述单元是基于逻辑功能划分的，在实际应用中，一个单元的功能也可以由多个单元来实现，或者多个单元的功能由一个单元实现。在本申请的其它实施例中，该点云媒体的数据处理装置50也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。根据本申请的另一个实施例，可以通过在包括例如中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的通用计算机的通用计算设备上运行能够执行如图3中所示的相应方法所涉及的各步骤的计算机程序(包括程序代码)，来构造如图5中所示的点云媒体的数据处理装置50，以及来实现本申请实施例的点云媒体的数据处理方法。计算机程序可以记载于例如计算机可读存储介质上，并通过计算机可读存储介质装载于图1所示点云媒体的数据处理系统的内容消费设备101中，并在其中运行。
请参见图6，图6示出了本申请实施例提供的一种点云媒体的数据处理装置的结构示意图，该点云媒体的数据处理装置60可以用于执行图4所示的点云媒体的数据处理方法中的相应步骤，该点云媒体的数据处理装置60包括如下单元：
处理单元601，配置为生成点云媒体的第i样本组的点云对象指示信息，点云媒体 包括N个样本组，第i样本组为N个样本组中的任一个；第i样本组中包括点云对象，第i样本组的点云对象指示信息用于指示第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；传输单元602，配置为向第一电子设备发送第i样本组的点云对象指示信息，以使第一电子设备按照第i样本组的点云对象指示信息解析点云媒体。
在一种实现方式中，点云媒体包括多个媒体帧；处理单元601，还配置为：对点云媒体的各个媒体帧进行对象识别；当识别出点云媒体的至少一个媒体帧中包含点云对象时，则将包含点云对象的至少一个媒体帧封装至第i样本组中，将点云媒体中的未包含点云对象的媒体帧分别封装至N个样本组中除第i样本组之外的其他样本组中；其中，第i样本组内的所有媒体帧构成一个能够被独立编解码的集合。
在一种实现方式中，处理单元601，还配置为：识别第i样本组内的点云对象的数量，并根据第i样本组内的点云对象的数量对第i样本组的点云对象指示信息中的对象数量字段进行配置处理；根据第i样本组内的点云对象的数量、以及第i样本组内的点云对象对应的优先级确定第i样本组的优先级，并根据第i样本组的优先级对第i样本组的点云对象指示信息中的对象优先级字段进行配置处理；其中，对象数量字段用于指示第i样本组内包含的点云对象的数量；对象优先级字段用于指示第i样本组的优先级。
在一种实现方式中，第i样本组内的点云对象的数量为M，M个点云对象中的每个点云对象分别对应一个优先级，其中，M为正整数；处理单元601，还配置为：当M的取值为1时，将第i样本组内的点云对象对应的优先级确定为第i样本组的优先级；当M的取值大于1时，确定第i样本组内的M个点云对象分别对应的优先级中的最高优先级，并将最高优先级确定为第i样本组的优先级。
在一种实现方式中，处理单元601，还配置为：当检测到满足丢弃条件时，根据点云媒体包含的各个样本组的优先级由低至高的顺序，丢弃点云媒体中的至少部分样本组；对已丢弃至少部分样本组的点云媒体进行重新封装，并发送至第一电子设备；其中，丢弃条件包括以下至少之一：网络拥塞、第一电子设备的存储空间小于存储空间阈值、第一电子设备的处理能力小于处理能力阈值。
在一种实现方式中，第i样本组内包含M个点云对象，M为正整数；第i样本组的点云对象指示信息中包含M个对象类型字段和/或M个对象描述字段；处理单元601，还配置为：执行以下至少一种处理：识别第i样本组内的第m点云对象的类型，并根据第m点云对象的类型对第i样本组的点云对象指示信息中的第m对象类型字段进行配置处理；获取第m点云对象的描述信息，并根据第m点云对象的描述信息对第i样本组的点云对象指示信息中的第m对象描述字段进行配置处理；其中，第m点云对象是M个点云对象中的任一个，第m对象类型字段用于指示第m点云对象的类型，第m对象描述字段用于指示第m点云对象的描述信息；m为正整数且m∈[1，M]。
在一种实现方式中，处理单元601，还配置为：获取第i样本组内的点云对象所属的应用场景，并根据第i样本组内的点云对象所属的应用场景对第i样本组的点云对象指示信息中的对象场景字段进行配置处理；其中，对象场景字段用于指示第i样本组内包含的点云对象所属的应用场景。
在一种实现方式中，处理单元601，还配置为：向第一电子设备发送描述信令文件，描述信令文件中包括点云媒体的至少一个封装文件描述信息；当接收到第一电子设备发送的携带有目标封装文件描述信息的获取请求时，向第一电子设备发送目标封装文件，以使第一电子设备从目标封装文件中获取第i样本组的点云对象指示信息，并按照第i样本组的点云对象指示信息对第i样本组进行独立解码，得到第i样本组内的至少一个媒体帧。
根据本申请的一个实施例，图6所示的点云媒体的数据处理装置60中的各个单元 可以分别或全部合并为一个或若干个另外的单元来构成，或者其中的某个(些)单元还可以再拆分为功能上更小的多个单元来构成，这可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述单元是基于逻辑功能划分的，在实际应用中，一个单元的功能也可以由多个单元来实现，或者多个单元的功能由一个单元实现。在本申请的其它实施例中，该点云媒体的数据处理装置60也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。根据本申请的另一个实施例，可以通过在包括例如中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的通用计算机的通用计算设备上运行能够执行如图4中所示的相应方法所涉及的各步骤的计算机程序(包括程序代码)，来构造如图6中所示的点云媒体的数据处理装置60，以及来实现本申请实施例的点云媒体的数据处理方法。计算机程序可以记载于例如计算机可读存储介质上，并通过计算机可读存储介质装载于图1所示点云媒体的数据处理系统的内容制作设备102中，并在其中运行。
请参见图7，图7示出了本申请示例性实施例提供的一种点云媒体的数据处理设备(或称计算机设备)的结构示意图，该点云媒体的数据处理设备70至少包括处理器701以及计算机可读存储介质702。其中，处理器701以及计算机可读存储介质702可通过总线或者其它方式连接。计算机可读存储介质702可以存储在存储器中，计算机可读存储介质702用于存储计算机程序，计算机程序包括计算机指令，处理器701用于执行计算机可读存储介质702存储的计算机指令。处理器701是点云媒体的数据处理设备70的计算核心以及控制核心，其适于实现一条或多条计算机指令，具体适于加载并执行一条或多条计算机指令从而实现相应方法流程或相应功能，其中，处理器701可以是中央处理器(Central Processing Unit，CPU)。
本申请实施例还提供了一种计算机可读存储介质(Memory)，计算机可读存储介质是点云媒体的数据处理设备70中的记忆设备，用于存放程序和数据。可以理解的是，此处的计算机可读存储介质702既可以包括点云媒体的数据处理设备70中的内置存储介质，当然也可以包括点云媒体的数据处理设备70所支持的扩展存储介质。计算机可读存储介质提供存储空间，该存储空间存储了点云媒体的数据处理设备70的操作系统。并且，在该存储空间中还存放了适于被处理器701加载并执行的一条或多条的计算机指令，这些计算机指令可以是一个或多个的计算机程序(包括程序代码)。需要说明的是，此处的计算机可读存储介质702可以是高速RAM存储器，也可以是非不稳定的存储器(Non-Volatile Memory)，例如至少一个磁盘存储器；可选的还可以是至少一个位于远离前述处理器701的计算机可读存储介质。
在一种实现方式中，该点云媒体的数据处理设备70可以是图1所示的点云媒体的数据处理系统中的内容消费设备101；该计算机可读存储介质702中存储有第一计算机指令；由处理器701加载并执行计算机可读存储介质702中存放的第一计算机指令，以实现图3所示方法实施例中的相应步骤。
在一种实现方式中，该点云媒体的数据处理设备70可以是图1所示的点云媒体的数据处理系统中的内容制作设备102；该计算机可读存储介质702中存储有第二计算机指令；由处理器701加载并执行计算机可读存储介质702中存放的第二计算机指令，以实现图4所示方法实施例中的相应步骤。
本申请实施例提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述各种可选方式中提供的点云媒体的数据处理方法。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何 熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。

Claims (25)

  1. 一种点云媒体的数据处理方法，由第一电子设备执行，所述方法包括：
    获取点云媒体的第i样本组的点云对象指示信息，所述点云媒体包括N个样本组，所述第i样本组为所述N个样本组中的任一个；所述第i样本组中包括点云对象，所述第i样本组的点云对象指示信息用于指示所述第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；
    按照所述第i样本组的点云对象指示信息解析所述点云媒体。
  2. 如权利要求1所述的方法，其中，所述第i样本组的点云对象指示信息中包含对象优先级字段以及对象数量字段中的至少之一；
    其中，所述对象优先级字段用于指示所述第i样本组的优先级；所述对象数量字段用于指示所述第i样本组内包含的点云对象的数量。
  3. 如权利要求1所述的方法，其中，第j样本组是所述N个样本组中除所述第i样本组外的任一个，j为正整数且j∈[1，N]；所述第i样本组的优先级高于所述第j样本组的优先级；
    所述按照所述第i样本组的点云对象指示信息解析所述点云媒体，包括：
    按照所述第i样本组的点云对象指示信息解析所述第i样本组；
    在解析所述第i样本组之后，解析所述第j样本组。
  4. 如权利要求3所述的方法，其中，所述方法还包括：
    当所述第j样本组中不包括点云对象时，确定所述第i样本组的优先级高于所述第j样本组的优先级；
    当所述第j样本组中包括点云对象、且所述第i样本组的点云对象指示信息中包含的对象优先级字段指示的优先级高于所述第j样本组的点云对象指示信息中包含的对象优先级字段指示的优先级时，确定所述第i样本组的优先级高于所述第j样本组的优先级。
  5. 如权利要求1所述的方法，其中，所述第i样本组的点云对象指示信息中包含对象场景字段，所述对象场景字段用于指示所述第i样本组中包含的点云对象所属的应用场景，且不同应用场景对应所述对象场景字段的不同取值；
    所述按照所述第i样本组的点云对象指示信息解析所述点云媒体，包括：
    读取所述第i样本组的点云对象指示信息中的对象场景字段，并根据所述对象场景字段的取值确定所述第i样本组内的点云对象所属的应用场景。
  6. 如权利要求1所述的方法，其中，所述第i样本组内包含M个点云对象，M为正整数；所述第i样本组的点云对象指示信息中包含M个对象类型字段，所述M个对象类型字段用于分别指示所述M个点云对象的类型；不同类型的点云对象对应的对象类型字段的取值不同；
    所述按照所述第i样本组的点云对象指示信息解析所述点云媒体，包括：
    读取所述第i样本组的点云对象指示信息中的第m对象类型字段，并根据所述第m对象类型字段的取值确定所述第i样本组内的第m点云对象的类型；
    其中，所述第m点云对象是所述M个点云对象中的任一个，所述第m对象类型字段用于指示所述第m点云对象的类型；m为正整数且m∈[1，M]。
  7. 如权利要求1所述的方法，其中，所述第i样本组内包含M个点云对象，M为正整数；所述第i样本组的点云对象指示信息中包含M个对象描述字段，所述M个对象描述字段用于分别指示所述M个点云对象的描述信息；
    所述按照所述第i样本组的点云对象指示信息解析所述点云媒体，包括：
    读取所述第i样本组的点云对象指示信息中的第m对象描述字段，并根据所述第m对象描述字段的取值确定所述第i样本组内的第m点云对象的描述信息；
    其中，所述第m点云对象是所述M个点云对象中的任一个，所述第m对象描述字段用于指示所述第m点云对象的描述信息；m为正整数且m∈[1，M]；
    所述方法还包括：
    响应所述描述信息。
  8. 如权利要求1-7任一项所述的方法，其中，所述点云媒体包括多个媒体帧，所述多个媒体帧被封装至所述N个样本组中，每个样本组中包括至少一个媒体帧；所述第i样本组内的点云对象存在于所述第i样本组内的媒体帧中；所述第i样本组内的所有媒体帧构成一个能够被独立编解码的集合；
    所述获取点云媒体的第i样本组的点云对象指示信息，包括：
    获取第二电子设备发送的描述信令文件，所述描述信令文件中包括所述点云媒体的至少一个封装文件描述信息；
    当所述描述信令文件中的目标封装文件描述信息被选择时，向所述第二电子设备发送携带有所述目标封装文件描述信息的获取请求，并接收所述第二电子设备根据所述获取请求发送的目标封装文件；
    从所述目标封装文件中获取所述第i样本组的点云对象指示信息；
    所述按照所述第i样本组的点云对象指示信息解析所述点云媒体，包括：
    按照所述第i样本组的点云对象指示信息对所述第i样本组进行独立解码，得到所述第i样本组内的至少一个媒体帧。
  9. 如权利要求8所述的方法，其中，所述按照所述第i样本组的点云对象指示信息对所述第i样本组进行独立解码，得到所述第i样本组内的至少一个媒体帧，包括：
    按照所述第i样本组的点云对象指示信息对所述第i样本组进行解封装处理，得到编码比特流；
    对所述编码比特流进行解码处理，得到所述第i样本组内的至少一个媒体帧。
  10. 如权利要求9所述的方法，其中，所述编码比特流包括几何比特流及属性比特流；所述几何比特流及所述属性比特流用于共同封装至所述第i样本组内的同一样本中，或者用于分别封装至所述第i样本组内的不同样本中；
    所述对所述编码比特流进行解码处理，得到所述第i样本组内的至少一个媒体帧，包括：
    对所述几何比特流进行解码处理，得到所述第i样本组内的至少一个媒体帧的几何信息；
    根据所述第i样本组内的至少一个媒体帧的几何信息对所述属性比特流进行解码处理，得到所述第i样本组内的至少一个媒体帧的属性信息。
  11. 如权利要求8所述的方法，其中，所述按照所述第i样本组的点云对象指示信息对所述第i样本组进行独立解码，得到所述第i样本组内的至少一个媒体帧之后，所述方法还包括：
    执行以下任意一种处理：
    从所述描述信令文件中获取当前用户的观看方向，并根据所述当前用户的观看方向对所述第i样本组内的至少一个媒体帧进行渲染处理；
    对所述当前用户进行跟踪处理得到所述当前用户的观看方向，并根据所述当前用户的观看方向对所述第i样本组内的至少一个媒体帧进行渲染处理；其中，所述跟踪处理包括头部跟踪处理以及视觉跟踪处理中的至少一种。
  12. 一种点云媒体的数据处理方法，由第二电子设备执行，所述方法包括：
    生成点云媒体的第i样本组的点云对象指示信息，所述点云媒体包括N个样本组，所述第i样本组为所述N个样本组中的任一个；所述第i样本组中包括点云对象，所述第i样本组的点云对象指示信息用于指示所述第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；
    向第一电子设备发送所述第i样本组的点云对象指示信息，以使所述第一电子设备按照所述第i样本组的点云对象指示信息解析所述点云媒体。
  13. 如权利要求12所述的方法，其中，所述点云媒体包括多个媒体帧；所述方法还包括：
    对所述点云媒体的各个媒体帧进行对象识别；
    当识别出所述点云媒体的至少一个媒体帧中包含点云对象时，则将包含点云对象的所述至少一个媒体帧封装至所述第i样本组中，将所述点云媒体中的未包含点云对象的媒体帧分别封装至所述N个样本组中除所述第i样本组之外的其他样本组中；
    其中，所述第i样本组内的所有媒体帧构成一个能够被独立编解码的集合。
  14. 如权利要求12所述的方法，其中，所述生成点云媒体的第i样本组的点云对象指示信息，包括：
    识别所述第i样本组内的点云对象的数量，并根据所述第i样本组内的点云对象的数量对所述第i样本组的点云对象指示信息中的对象数量字段进行配置处理；
    根据所述第i样本组内的点云对象的数量、以及所述第i样本组内的点云对象对应的优先级确定所述第i样本组的优先级，并根据所述第i样本组的优先级对所述第i样本组的点云对象指示信息中的对象优先级字段进行配置处理；
    其中，所述对象数量字段用于指示所述第i样本组内包含的点云对象的数量；所述对象优先级字段用于指示所述第i样本组的优先级。
  15. 如权利要求14所述的方法，其中，所述第i样本组内的点云对象的数量为M，M个点云对象中的每个点云对象分别对应一个优先级，其中，M为正整数；
    所述根据所述第i样本组内的点云对象的数量、以及所述第i样本组内的点云对象对应的优先级确定所述第i样本组的优先级，包括：
    当M的取值为1时，将所述第i样本组内的点云对象对应的优先级确定为所述第i样本组的优先级；
    当M的取值大于1时，确定所述第i样本组内的M个点云对象分别对应的优先级中的最高优先级，并将所述最高优先级确定为所述第i样本组的优先级。
  16. 如权利要求12所述的方法，其中，所述方法还包括：
    当检测到满足丢弃条件时，根据所述点云媒体包含的各个样本组的优先级由低至高的顺序，丢弃所述点云媒体中的至少部分样本组；
    对已丢弃所述至少部分样本组的所述点云媒体进行重新封装，并发送至所述第一电子设备；
    其中，所述丢弃条件包括以下至少之一：网络拥塞、所述第一电子设备的存储空间小于存储空间阈值、所述第一电子设备的处理能力小于处理能力阈值。
  17. 如权利要求12所述的方法，其中，所述第i样本组内包含M个点云对象，M为正整数；所述第i样本组的点云对象指示信息中包含M个对象类型字段和/或M个对象描述字段；
    所述生成点云媒体的第i样本组的点云对象指示信息，包括：
    执行以下至少一种处理：
    识别所述第i样本组内的第m点云对象的类型，并根据所述第m点云对象的类型对所述第i样本组的点云对象指示信息中的第m对象类型字段进行配置处理；
    获取所述第m点云对象的描述信息，并根据所述第m点云对象的描述信息对所述第i样本组的点云对象指示信息中的第m对象描述字段进行配置处理；
    其中，所述第m点云对象是所述M个点云对象中的任一个，所述第m对象类型字段用于指示所述第m点云对象的类型，所述第m对象描述字段用于指示所述第m点云对象的描述信息；m为正整数且m∈[1，M]。
  18. 如权利要求12所述的方法，其中，所述生成点云媒体的第i样本组的点云对象指示信息，包括：
    获取所述第i样本组内的点云对象所属的应用场景，并根据所述第i样本组内的点云对象所属的应用场景对所述第i样本组的点云对象指示信息中的对象场景字段进行配置处理；
    其中，所述对象场景字段用于指示所述第i样本组内包含的点云对象所属的应用场景。
  19. 如权利要求12所述的方法，其中，所述点云媒体包括多个媒体帧，所述多个媒体帧被封装至所述N个样本组中，每个样本组中包括至少一个媒体帧；所述第i样本组内的点云对象存在于所述第i样本组内的媒体帧中；所述第i样本组内的所有媒体帧构成一个能够被独立编解码的集合；
    所述向第一电子设备发送所述第i样本组的点云对象指示信息，以使所述第一电子设备按照所述第i样本组的点云对象指示信息解析所述点云媒体，包括：
    向所述第一电子设备发送描述信令文件，所述描述信令文件中包括所述点云媒体的至少一个封装文件描述信息；
    当接收到所述第一电子设备发送的携带有目标封装文件描述信息的获取请求时，向所述第一电子设备发送目标封装文件，以使
    所述第一电子设备从所述目标封装文件中获取所述第i样本组的点云对象指示信息，并按照所述第i样本组的点云对象指示信息对所述第i样本组进行独立解码，得到所述第i样本组内的至少一个媒体帧。
  20. 如权利要求12-19任一项所述的方法，其中，所述点云媒体包括多个媒体帧；所述方法还包括：
    对所述多个媒体帧中包括点云对象的至少一个媒体帧进行编码处理，得到编码比特流；
    对所述编码比特流进行封装处理，得到所述第i样本组。
  21. 如权利要求20所述的方法，其中，所述编码比特流包括几何比特流及属性比特流，所述几何比特流用于表示所述至少一个媒体帧的几何信息，所述属性比特流用于表示所述至少一个媒体帧的属性信息；
    所述对所述编码比特流进行封装处理，得到所述第i样本组，包括：
    执行以下任意一种处理：
    将所述几何比特流及所述属性比特流共同封装至同一样本中，并根据所述同一样本构建所述第i样本组；
    将所述几何比特流及所述属性比特流分别封装至不同样本中，并根据所述不同样本构建所述第i样本组。
  22. 一种点云媒体的数据处理装置，所述点云媒体的数据处理装置包括：
    获取单元，配置为获取点云媒体的第i样本组的点云对象指示信息，所述点云媒体包括N个样本组，所述第i样本组为所述N个样本组中的任一个；所述第i样本组中包括点云对象，所述第i样本组的点云对象指示信息用于指示所述第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；
    处理单元，配置为按照所述第i样本组的点云对象指示信息解析所述点云媒体。
  23. 一种点云媒体的数据处理装置，所述点云媒体的数据处理装置包括：
    处理单元，配置为生成点云媒体的第i样本组的点云对象指示信息，所述点云媒体包括N个样本组，所述第i样本组为所述N个样本组中的任一个；所述第i样本组中包括点云对象，所述第i样本组的点云对象指示信息用于指示所述第i样本组中包含的点云对象的属性，N、i均为正整数且i∈[1，N]；
    传输单元，配置为向第一电子设备发送所述第i样本组的点云对象指示信息，以使所述第一电子设备按照所述第i样本组的点云对象指示信息解析所述点云媒体。
  24. 一种点云媒体的数据处理设备，所述点云媒体的数据处理设备包括：
    处理器，适于实现计算机指令；以及，
    计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令适于由所述处理器加载并执行如权利要求1至11任一项所述的点云媒体的数据处理方法，或者如权利要求12至21任一项所述的点云媒体的数据处理方法。
  25. 一种计算机可读存储介质，所述计算机可读存储介质包括计算机指令，所述计算机指令适于由处理器加载并执行如权利要求1至11任一项所述的点云媒体的数据处理方法，或者如权利要求12至21任一项所述的点云媒体的数据处理方法。
PCT/CN2021/115689 2020-09-25 2021-08-31 一种点云媒体的数据处理方法、装置、设备及存储介质 WO2022062860A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011030289.3 2020-09-25
CN202011030289.3A CN114257816A (zh) 2020-09-25 2020-09-25 一种点云媒体的数据处理方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2022062860A1 true WO2022062860A1 (zh) 2022-03-31

Family

ID=80790773

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/115689 WO2022062860A1 (zh) 2020-09-25 2021-08-31 一种点云媒体的数据处理方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN114257816A (zh)
WO (1) WO2022062860A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115396647A (zh) * 2022-08-22 2022-11-25 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质
WO2024037137A1 (zh) * 2022-08-18 2024-02-22 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备、介质和产品

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115379189B (zh) * 2022-08-19 2023-12-12 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法及相关设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110996098A (zh) * 2018-10-02 2020-04-10 腾讯美国有限责任公司 处理点云数据的方法和装置
CN111133476A (zh) * 2017-09-18 2020-05-08 苹果公司 点云压缩
CN111641834A (zh) * 2019-03-01 2020-09-08 腾讯美国有限责任公司 点云编码的方法和装置、计算机设备和存储介质
US20200302632A1 (en) * 2019-03-21 2020-09-24 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus, and point cloud data reception method
US20200304834A1 (en) * 2019-03-19 2020-09-24 Mediatek Singapore Pte. Ltd. Methods and apparatus for track derivation for immersive media data tracks

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111133476A (zh) * 2017-09-18 2020-05-08 苹果公司 点云压缩
CN110996098A (zh) * 2018-10-02 2020-04-10 腾讯美国有限责任公司 处理点云数据的方法和装置
CN111641834A (zh) * 2019-03-01 2020-09-08 腾讯美国有限责任公司 点云编码的方法和装置、计算机设备和存储介质
US20200304834A1 (en) * 2019-03-19 2020-09-24 Mediatek Singapore Pte. Ltd. Methods and apparatus for track derivation for immersive media data tracks
US20200302632A1 (en) * 2019-03-21 2020-09-24 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus, and point cloud data reception method

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024037137A1 (zh) * 2022-08-18 2024-02-22 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备、介质和产品
CN115396647A (zh) * 2022-08-22 2022-11-25 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质
CN115396647B (zh) * 2022-08-22 2024-04-26 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN114257816A (zh) 2022-03-29

Similar Documents

Publication Publication Date Title
WO2022062860A1 (zh) 一种点云媒体的数据处理方法、装置、设备及存储介质
WO2022037386A1 (zh) 点云媒体的数据处理方法、装置、设备及存储介质
US9147291B2 (en) Method and apparatus of processing data to support augmented reality
CN114374709B (zh) 基于边云协同的5g视频与物联网配网监测系统及方法
US20230421810A1 (en) Encapsulation and decapsulation methods and apparatuses for point cloud media file, and storage medium
CN115379189B (zh) 一种点云媒体的数据处理方法及相关设备
WO2024041238A1 (zh) 一种点云媒体的数据处理方法及相关设备
WO2024041239A1 (zh) 一种沉浸媒体的数据处理方法、装置、设备、存储介质及程序产品
WO2023226504A1 (zh) 一种媒体数据处理方法、装置、设备以及可读存储介质
WO2022193875A1 (zh) 多视角视频的处理方法、装置、设备及存储介质
US20230061573A1 (en) Point Cloud Encoding and Decoding Method and Apparatus, Computer-Readable Medium, and Electronic Device
US20230034937A1 (en) Media file encapsulating method, media file decapsulating method, and related devices
US20230025664A1 (en) Data processing method and apparatus for immersive media, and computer-readable storage medium
CN114554243B (zh) 点云媒体的数据处理方法、装置、设备及存储介质
WO2022174745A1 (zh) 非时序点云媒体的数据处理方法、装置以及设备
CN114598692B (zh) 点云文件传输方法、应用方法、装置、设备及存储介质
CN117176962B (zh) 一种视频编解码方法、装置及相关设备
CN115834857B (zh) 点云数据处理方法、装置、设备及存储介质
WO2023024839A1 (zh) 媒体文件封装与解封装方法、装置、设备及存储介质
WO2023169003A1 (zh) 点云媒体的解码方法、点云媒体的编码方法及装置
CN115379188B (zh) 点云数据处理方法、装置、设备及存储介质
CN116939290A (zh) 媒体数据处理方法、装置、设备及存储介质
CN117082262A (zh) 点云文件封装与解封装方法、装置、设备及存储介质
CN117880562A (zh) 一种数据处理方法、装置、设备及存储介质
CN117978992A (zh) 点云媒体的数据处理方法、装置、设备、存储介质及产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21871232

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 14/08/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21871232

Country of ref document: EP

Kind code of ref document: A1