WO2022222641A1 - 点云编解码方法、装置、计算机可读介质以及电子设备 - Google Patents

点云编解码方法、装置、计算机可读介质以及电子设备 Download PDF

Info

Publication number
WO2022222641A1
WO2022222641A1 PCT/CN2022/080266 CN2022080266W WO2022222641A1 WO 2022222641 A1 WO2022222641 A1 WO 2022222641A1 CN 2022080266 W CN2022080266 W CN 2022080266W WO 2022222641 A1 WO2022222641 A1 WO 2022222641A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
frame rate
file
track
media
Prior art date
Application number
PCT/CN2022/080266
Other languages
English (en)
French (fr)
Inventor
胡颖
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to KR1020237031030A priority Critical patent/KR20230144620A/ko
Priority to JP2023552564A priority patent/JP2024508865A/ja
Publication of WO2022222641A1 publication Critical patent/WO2022222641A1/zh
Priority to US17/982,927 priority patent/US20230061573A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/762—Media network packet handling at the source 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/23605—Creation or processing of packetized elementary streams [PES]

Definitions

  • the present application belongs to the field of computer and communication technologies, and in particular relates to a point cloud encoding and decoding method, a point cloud encoding and decoding device, a computer-readable medium, and an electronic device.
  • a point cloud is a set of discrete points that are randomly distributed in space and express the spatial structure and surface properties of a three-dimensional object or scene. After the large-scale point cloud data is acquired through the point cloud acquisition device, the point cloud data can be encoded and packaged for transmission and presentation to the user.
  • Embodiments of the present application provide a point cloud encoding and decoding method, a point cloud encoding and decoding apparatus, a computer-readable medium, and an electronic device.
  • a point cloud decoding method comprising: receiving a point cloud file transmitted by a data source, the point cloud file including one or more point cloud media having the same point cloud content track, the point cloud file includes some point cloud media tracks with different frame rates; parses the file encapsulation information of the one or more point cloud media tracks, and obtains the frame rate indication information carried in the file encapsulation information.
  • the frame rate indication information is used to indicate the frame rate of the one or more point cloud media tracks; according to the frame rate indication information carried in the file encapsulation information, select and decode the point cloud file with the specified frame rate.
  • the point cloud media track is used to indicate the frame rate of the one or more point cloud media tracks.
  • a point cloud decoding apparatus the apparatus includes: a receiving module configured to receive a point cloud file transmitted by a data source, where the point cloud file includes one or more points having the same point The point cloud media track of the cloud content, the point cloud file includes some point cloud media tracks with different frame rates; the parsing module is configured to parse the file encapsulation information of the one or more point cloud media tracks, and obtain the The frame rate indication information carried in the file encapsulation information, where the frame rate indication information is used to indicate the frame rate of the one or more point cloud media tracks; the decoding module is configured to, according to the frame rate carried in the file encapsulation information rate indication information, and select and decode the point cloud media track with the specified frame rate from the point cloud file.
  • the receiving module includes: a signaling receiving unit configured to receive streaming media signaling sent by a data source for transmitting point cloud data; a signaling parsing unit , which is configured to parse the streaming media signaling to obtain a time-domain hierarchical group identifier carried in the streaming media signaling for identifying a track group, where the track group includes one or more points with the same point cloud content Cloud media tracks, the track group includes some point cloud media tracks with different frame rates; a request sending unit is configured to send a first data transmission request to the data source according to the time-domain hierarchical group identifier; file reception a unit configured to receive a point cloud file corresponding to the first data transmission request transmitted by the data source.
  • the request sending unit includes: a bandwidth acquisition subunit, configured to acquire network bandwidth for data transmission with the data source; a track selection subunit, configured to According to the time-domain hierarchical group identifier, one or more target point cloud media tracks with a target frame rate that match the network bandwidth are selected from the track group; the request sending subunit is configured to send a request to the track group.
  • the data source sends a first data transmission request requesting transmission of the one or more target point cloud media tracks.
  • the parsing module includes: an information parsing unit, configured to parse the file encapsulation information of the one or more point cloud media tracks, to determine the relationship with the one or more point cloud media tracks. a frame rate indication field corresponding to the frame rates of multiple point cloud media tracks; the information determination unit is configured to determine the frame rate indication of the one or more point cloud media tracks according to the value of the frame rate indication field information.
  • the point cloud decoding apparatus further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a track selection module, configured to According to the frame rate indication information carried in the file encapsulation information, other point cloud media tracks with the same frame rate as the point cloud media track to be displayed are selected from the point cloud file; the first replacement module is configured To replace the point cloud media track to be displayed with the other point cloud media track, so as to decode and display the other point cloud media track.
  • the point cloud decoding apparatus further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a track selection module, configured to According to the frame rate indication information carried in the file encapsulation information, select one or more other point cloud media tracks with the same frame rate as the point cloud media track to be displayed from the point cloud file; first merge The module is configured to decode the one or more other point cloud media tracks, and combine and display the one or more other point cloud media tracks and the point cloud media track to be displayed.
  • the point cloud decoding apparatus further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a file acquisition module, configured to Sending a second data transmission request to the data source to receive a supplemental point cloud file transmitted by the data source, the supplemental point cloud file comprising the same point cloud content and the same point cloud media track as the point cloud media track to be displayed other point cloud media tracks of the frame rate; and a second replacement module, configured to replace the point cloud media tracks to be displayed with the other point cloud media tracks, so as to decode and display the other point cloud media tracks.
  • the point cloud decoding apparatus further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a file acquisition module, configured to Sending a third data transmission request to the data source to receive a supplemental point cloud file transmitted by the data source, the supplemental point cloud file comprising the same point cloud content and the same point cloud media track as the point cloud media track to be displayed frame rate of one or more other point cloud media tracks; a second merging module configured to decode the one or more other point cloud media tracks and combine the one or more other point cloud media tracks with the The point cloud media tracks to be displayed are combined and displayed.
  • a point cloud encoding method comprising: encoding point cloud data to be transmitted according to different encoding standards to obtain multiple point cloud code streams having the same point cloud content,
  • the multiple point cloud code streams include some point cloud code streams with different frame rates; the multiple point cloud code streams are encapsulated into multiple point cloud media tracks, and the multiple point cloud media tracks are filled with Frame rate indication information corresponding to the multiple point cloud code streams, where the frame rate indication information is used to indicate frame rates of the multiple point cloud media tracks.
  • a point cloud encoding device the device includes: an encoding module configured to encode point cloud data to be transmitted according to different encoding standards, to obtain multi-point cloud data with the same point cloud content.
  • point cloud code streams the multiple point cloud code streams include some point cloud code streams with different frame rates;
  • the encapsulation module is configured to encapsulate the multiple point cloud code streams into multiple point cloud media tracks, and filling the multiple point cloud media tracks with frame rate indication information corresponding to the multiple point cloud code streams, where the frame rate indication information is used to indicate the frame rates of the multiple point cloud media tracks.
  • the point cloud encoding apparatus further includes: a signaling generation module, configured to generate streaming media signaling for transmitting point cloud data; a signaling filling module, configured by is configured to fill the streaming media signaling with a time-domain hierarchical group identifier for identifying a track group, where the track group includes one or more point cloud media tracks with the same point cloud content, and the track group includes part of point cloud media tracks with different frame rates; the signaling sending module is configured to send the streaming media signaling to the data receiver.
  • a signaling generation module configured to generate streaming media signaling for transmitting point cloud data
  • a signaling filling module configured by is configured to fill the streaming media signaling with a time-domain hierarchical group identifier for identifying a track group, where the track group includes one or more point cloud media tracks with the same point cloud content, and the track group includes part of point cloud media tracks with different frame rates
  • the signaling sending module is configured to send the streaming media signaling to the data receiver.
  • the point cloud encoding apparatus further includes: a request receiving module configured to receive the data transmission generated based on the streaming media signaling and sent by the data receiver request; a file transmission module configured to transmit a point cloud file to the data receiver according to the data transmission request, where the point cloud file includes one or more point cloud media tracks with the same point cloud content, the The point cloud file includes some point cloud media tracks with different frame rates.
  • the encapsulation module includes: an information determination unit configured to determine, from the file encapsulation information of the plurality of point cloud media tracks, the frame rate indication information corresponding to the frame rate indication information. A corresponding frame rate indication field; an information filling unit configured to fill the frame rate indication field in the file encapsulation information with frame rate indication information corresponding to the plurality of point cloud code streams.
  • a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processor, implements the method in the above technical solution.
  • an electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the The executable instructions are described to execute the method as in the above technical solution.
  • a computer program product or computer program where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method as in the above technical solutions.
  • FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.
  • Figure 2 shows the placement of the point cloud encoding device and the point cloud decoding device in a streaming environment.
  • FIG. 3 shows a flowchart of steps of a point cloud decoding method in an embodiment of the present application.
  • FIG. 4 shows a schematic diagram of an alternative group packaged in multiple tracks in one embodiment of the present application.
  • FIG. 5 shows a flowchart of steps for receiving a point cloud file from a data source in an embodiment of the present application.
  • FIG. 6 shows a flowchart of steps of a point cloud encoding method in an embodiment of the present application.
  • FIG. 7 shows a flowchart of steps for encoding and decoding point cloud data in an application scenario according to an embodiment of the present application.
  • FIG. 8 schematically shows a structural block diagram of a point cloud decoding apparatus provided by an embodiment of the present application.
  • FIG. 9 schematically shows a structural block diagram of a point cloud encoding apparatus provided by an embodiment of the present application.
  • FIG. 10 schematically shows a structural block diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
  • a point cloud is a set of discrete points that are randomly distributed in space and express the spatial structure and surface properties of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may also have color, material or other information depending on the application scenario. Typically, each point in a point cloud has the same number of additional properties.
  • point cloud media can be further divided into Video-based Point Cloud Compression (VPCC), which is compressed based on traditional video encoding methods, and Point Cloud Compression (Geometry-based Point Cloud Compression), which is compressed based on geometric features. GPCC).
  • VPCC Video-based Point Cloud Compression
  • Geometry-based Point Cloud Compression Point Cloud Compression
  • the three-dimensional position information is usually called the geometric component of the point cloud file (Geometry Component), and the attribute information is called the attribute component (Attribute Component) of the point cloud file.
  • a point cloud file may have only one geometric component, but can have one or more attribute components.
  • Point cloud can express the spatial structure and surface properties of 3D objects or scenes flexibly and conveniently, so it is widely used, and its main application scenarios can be classified into two categories.
  • Machine perception point cloud such as autonomous navigation system, real-time inspection system, geographic information system, visual sorting robot, rescue and disaster relief robot.
  • Human eyes perceive point clouds such as point cloud application scenarios such as digital cultural heritage, free viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.
  • the acquisition of point cloud mainly includes the following methods: computer generation, 3D laser scanning, 3D photogrammetry, etc.
  • Computers can generate point clouds of virtual 3D objects and scenes.
  • 3D scanning can obtain point clouds of static real-world 3D objects or scenes, and millions of point clouds can be obtained per second.
  • 3D cameras can obtain point clouds of dynamic real-world three-dimensional objects or scenes, and can obtain tens of millions of point clouds per second.
  • point clouds of biological tissues and organs can be obtained from MRI, CT, and electromagnetic positioning information.
  • the encoded data stream needs to be encapsulated and transmitted to the user.
  • the point cloud file needs to be decapsulated first, then decoded, and finally the decoded data stream is presented.
  • FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.
  • the system architecture 100 includes a plurality of end devices that can communicate with each other through, for example, a network 150 .
  • the system architecture 100 may include a first end device 110 and a second end device 120 interconnected by a network 150 .
  • the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.
  • the first terminal device 110 may encode point cloud data (eg, a point cloud code stream collected by the first terminal device 110 ) for transmission to the second terminal device 120 through the network 150 , and the encoded point cloud data is
  • the second terminal device 120 may receive the encoded point cloud data from the network 150, decode the encoded point cloud data, and display the decoded point cloud data.
  • the system architecture 100 may include a third end device 130 and a fourth end device 140 that perform bidirectional transmission of encoded point cloud data, such as may occur during a video conference.
  • each of the third terminal device 130 and the fourth terminal device 140 may encode point cloud data (eg, a point cloud code stream collected by the terminal device) for transmission to the third terminal device through the network 150 Another terminal device among the terminal device 130 and the fourth terminal device 140 .
  • Each of the third terminal device 130 and the fourth terminal device 140 may also receive encoded point cloud data transmitted by the other of the third terminal device 130 and the fourth terminal device 140, and may The encoded point cloud data is decoded to recover the point cloud data, and the point cloud data can be displayed on an accessible display device based on the recovered point cloud data.
  • the first terminal device 110 , the second terminal device 120 , the third terminal device 130 and the fourth terminal device 140 may be servers, personal computers and smart phones, but the principles disclosed in this application may not be limited thereto . Embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and/or dedicated videoconferencing equipment.
  • the network 150 represents any number of networks that communicate encoded point cloud data between the first end device 110, the second end device 120, the third end device 130, and the fourth end device 140, including, for example, wired and/or wireless communication networks .
  • Network 150 may exchange data in circuit-switched and/or packet-switched channels.
  • the network may include a telecommunications network, a local area network, a wide area network, and/or the Internet. For the purposes of this application, unless explained below, the architecture and topology of network 150 may be immaterial to the operations disclosed herein.
  • FIG. 2 shows the placement of the point cloud encoding device and the point cloud decoding device in a streaming environment.
  • the subject matter disclosed herein is equally applicable to other point cloud enabled applications including, for example, videoconferencing, digital television, storing compressed point cloud data on digital media including CDs, DVDs, memory sticks, and the like.
  • the streaming system may include an acquisition subsystem 213 , which may include a point cloud data source 201 such as a digital camera, and the point cloud data source 201 may, for example, create uncompressed point cloud data 202 .
  • the point cloud data 202 includes samples captured by a digital camera.
  • point cloud data 202 is depicted as a thick line to emphasize high data volume point cloud data that can be processed by electronic device 220
  • the electronic device 220 includes a video encoding device 203 coupled to the video source 201 .
  • Video encoding device 203 may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in greater detail below.
  • encoded point cloud data 204 (or encoded point cloud codestream 204) is depicted as a thin line to emphasize the lower amount of encoded point cloud data 204 (or encoded point cloud data 204).
  • point cloud code stream 204 which may be stored on the streaming server 205 for future use.
  • One or more streaming client subsystems such as client subsystem 206 and client subsystem 208 in FIG. 2 , may access streaming server 205 to retrieve encoded point cloud data 207 that is a copy of point cloud data 204 and point cloud data 209.
  • Client subsystem 206 may include, for example, point cloud decoding device 210 in electronic device 230 .
  • the point cloud decoding device 210 decodes the incoming copy 207 of the encoded point cloud data and produces output point cloud data 211 that can be presented on a display 212 (eg, a display screen) or another presentation device.
  • the encoded point cloud data 204, point cloud data 207, and point cloud data 209 may be encoded according to certain point cloud encoding/compression standards. Examples of these standards may include standards developed by MPEG for GPCC.
  • electronic device 220 and the electronic device 230 may include other components not shown in the figures.
  • electronic device 220 may include a point cloud decoding device
  • electronic device 230 may also include a point cloud encoding device.
  • FIG. 3 shows a flow chart of the steps of a point cloud decoding method in an embodiment of the present application.
  • the method can be applied to links such as a server, a client, and an intermediate node of a point cloud media system.
  • a point cloud decoding method is installed.
  • the point cloud decoding method performed by the client device of the apparatus is taken as an example.
  • the point cloud decoding method may mainly include the following steps S310 to S330.
  • Step S310 Receive the point cloud file transmitted by the data source, the point cloud file includes one or more point cloud media tracks with the same point cloud content, and the point cloud file includes some point cloud media tracks with different frame rates.
  • the point cloud file may individually encapsulate a point cloud media track corresponding to a specified frame rate, or may encapsulate multiple point cloud media tracks with the same or different frame rates.
  • multiple point cloud media tracks may form a track group, and the content of the track group may include point cloud media tracks (including geometric components and attribute components) in single-track packaging mode or may include geometric component tracks in multi-track packaging mode ( The attribute component track is indexed by the geometry component track).
  • the point cloud media tracks of the same frame rate can be substituted for each other during decoding and display, and the point cloud media tracks of the same frame rate can be combined for consumption to achieve a better point cloud presentation effect.
  • some point cloud media tracks in the point cloud file have the same frame rate, while other point cloud media tracks may have different frame rates.
  • three point cloud media tracks, track1, track2, and track3 are encapsulated in the point cloud file.
  • the frame rate of track1 is 60fps, while the frame rate of track2 and track3 are both 30fps.
  • Point cloud media tracks with the same point cloud content and different point cloud qualities can be divided into the same alternative group.
  • Cloud quality may include various quality parameters corresponding to different standards, such as bit rate, frame rate, and resolution.
  • the tracks of the point cloud content of different qualities belong to the same replaceable group.
  • the geometric component tracks of point cloud contents of different qualities belong to the same replaceable group, and the attribute component track can be associated with the geometric component track.
  • FIG. 4 shows a schematic diagram of an alternative group packaged in multiple tracks in one embodiment of the present application.
  • the first point cloud data 410 and the second point cloud data 420 having the same point cloud content are included in the alternative group 400 .
  • the first point cloud data 410 is lossless compressed point cloud data with relatively high point cloud quality (Lossless coded GPCC)
  • the second point cloud data 420 is lossy compressed point cloud data with relatively low point cloud quality (Lossy coded GPCC) GPCC).
  • the first point cloud data 410 includes a first geometrical component track 411 and a first attribute component track 412 associated with the first geometrical component track 411
  • the second point cloud data 420 includes a second geometrical component track 421 and a second geometrical component track 421 associated with the second geometrical component track 411
  • the second attribute component track 422 of the geometry component track 421 is included in the first point cloud data 410 .
  • Step S320 Parse the file encapsulation information of one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information, where the frame rate indication information is used to indicate the frame rate of one or more point cloud media tracks.
  • the method for parsing file encapsulation information to obtain quality indication information may include: parsing the file encapsulation information of one or more point cloud media tracks to determine the relationship with one or more point cloud media tracks.
  • the frame rate indication field corresponding to the frame rate of the cloud media track; the frame rate indication information of one or more point cloud media tracks is determined according to the value of the frame rate indication field.
  • the file encapsulation information is an ISOBMFF (ISO Base Media File Format) data box generated when a point cloud code stream is encapsulated into a point cloud media track.
  • ISOBMFF ISO Base Media File Format
  • the file encapsulation information may specifically be expressed as an expanded track group data box TrackGroupTypeBox, and its syntax is as follows.
  • the frame_rate is frame rate indication information used to indicate the frame rate corresponding to the point cloud file, and its value is an unsigned integer with a length of 8 bytes.
  • point cloud media tracks with the same content and different frame rates can be associated with each other.
  • the point cloud orbits that belong to the same orbit group meet the following conditions.
  • the track is a point cloud track (including geometry and attribute components) in the single-track packaging mode or a geometric component track in the multi-track packaging mode (the attribute component track is obtained from the geometric component track index).
  • Step S330 According to the frame rate indication information carried in the file encapsulation information, select and decode the point cloud media track with the specified frame rate from the point cloud file.
  • the file encapsulation information corresponding to each point cloud media track carries frame rate indication information of the point cloud media track, and the frame rate indication information identifies the frame rate of the point cloud media track in an explicit way.
  • the data receiver can decode the point cloud media track with the specified frame rate according to the device performance and user requirements.
  • the device performance of the data receiver can be collected, and the device performance and the quality indication information (frame rate indication information) carried in the file encapsulation information can be matched and detected to determine the data receiver's device. performance-matched frame rate, and then select and decode point cloud media tracks with the specified frame rate from the point cloud file.
  • the quality indication information frame rate indication information
  • Device capabilities may include at least one of hardware capabilities, software capabilities, and network capabilities.
  • the hardware performance may include, for example, the device model, processor model, memory capacity, display size, etc. of the electronic device.
  • the software performance may include, for example, the program version of the point cloud decoder installed by the data receiver.
  • the network performance may include, for example, network bandwidth, network transfer status, etc.
  • the frame rate selection rule configured by the data receiver can be obtained, and the frame rate selection rule and the frame rate indication information carried in the file encapsulation information can be matched and detected, and the frame rate configured with the data receiver can be determined. Pick the frame rate that matches the rule, and then pick and decode the point cloud media track with the specified frame rate from the point cloud file.
  • the frame rate selection rule may be a selection rule configured according to user requirements for selecting point cloud data with a specified frame rate, such as selecting point cloud data with a frame rate greater than (or less than) a specified value according to user instructions.
  • FIG. 5 shows a flowchart of steps for receiving a point cloud file from a data source in an embodiment of the present application.
  • receiving the point cloud file transmitted by the data source in step S310 may include the following steps S510 to S540.
  • Step S510 Receive streaming media signaling sent by the data source for transmitting point cloud data.
  • the streaming media signaling for transmitting point cloud data may be dynamic adaptive streaming over HTTP (DASH) signaling based on HTTP
  • DASH is an adaptive bit rate streaming technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP web servers.
  • Step S520 Parse the streaming media signaling to obtain a time-domain hierarchical group identifier carried in the streaming media signaling for identifying the track group, where the track group includes one or more point cloud media tracks with the same point cloud content, and the track group Includes some point cloud media tracks with different frame rates.
  • the frame rate indication information of the point cloud media track can be stored by using an existing field, for example, the frameRate field in the DASH signaling can be used to indicate the frame rates of various different point cloud media tracks.
  • a group ID can be used to identify them in the DASH signaling, for example, the GPCC time-domain hierarchical group Identifier (GPCCTemporalScaleGroupId).
  • the set ID element is a child element of the AdaptationSet element.
  • the GPCCTemporalScaleGroupId element may appear at the adaptation set level, but not at any other level.
  • Table 1 shows the semantics and attributes of the GPCC time-domain hierarchical group ID in an embodiment of the present application.
  • Step S530 Send a data transmission request to the data source according to the time-domain hierarchical group identifier.
  • one or more of the frame rate indication information can be selected as the target frame rate, and a data transmission request corresponding to the target frame rate is further sent to the data source.
  • the data transmission request is, for example, the first data transmission request.
  • the method for sending a data transmission request may include: obtaining a network bandwidth for data transmission with a data source; One or more target point cloud media tracks at a rate; send a data transfer request to the data source for requesting transfer of the target point cloud media tracks.
  • two or more target point cloud media tracks may be selected.
  • a target point cloud media track can be selected.
  • Step S540 Receive the point cloud file corresponding to the data transmission request transmitted by the data source.
  • the data source may transmit a corresponding one target point cloud media track to the data receiver based on the request.
  • the data source may transmit corresponding multiple data transmissions with the same frame rate or different frame rates to the data receiver based on the request. target point cloud media track.
  • each point cloud media file can be replaced or merged to improve the display effect of the point cloud media files.
  • point cloud media track to be displayed fails to select, fails to decode, or the quality of the point cloud obtained after decoding is poor
  • other point cloud media tracks with the same frame rate can be used to perform track replacement .
  • two point cloud media files with the same point cloud content at 30fps can be replaced with each other.
  • multiple point cloud media tracks with the same frame rate can be track merged to improve the display frame of point cloud data.
  • Rate For example, two point cloud media tracks with the same point cloud content with a frame rate of 30fps can be merged to form a point cloud media track with a frame rate of 60fps, so that a better point cloud can be obtained by increasing the frame rate.
  • Media file display effect when the network environment of the data recipient is optimized and the network bandwidth is high, multiple point cloud media tracks with the same frame rate can be track merged to improve the display frame of point cloud data.
  • the point cloud media track replacement method obtain the frame rate of the point cloud media track to be displayed; according to the frame rate indication information carried in the file encapsulation information, select from the point cloud file the same frame rate as the point cloud media track to be displayed.
  • Other point cloud media tracks replace the point cloud media tracks to be displayed with other point cloud media tracks to decode and display other point cloud media tracks.
  • Method for merging point cloud media tracks obtain the frame rate of the point cloud media track to be displayed; according to the frame rate indication information carried in the file encapsulation information, select from the point cloud file the same frame rate as the point cloud media track to be displayed.
  • One or more other point cloud media tracks decode one or more other point cloud media tracks, and combine and display the one or more other point cloud media tracks with the point cloud media track to be displayed.
  • the point cloud media track replacement method can be implemented: obtain the frame rate of the point cloud media track to be displayed; send a data transmission request to the data source (for example, the second data transmission request) to receive the supplementary point cloud file transmitted by the data source, the supplementary point cloud file includes other point cloud media tracks with the same point cloud content and the same frame rate as the point cloud media track to be displayed; The point cloud media tracks are replaced with other point cloud media tracks to decode and display the other point cloud media tracks.
  • the point cloud media track merging method can be implemented: obtain the frame rate of the point cloud media track to be displayed; send a data transmission request to the data source (for example, the third data transmission request) to receive a supplemental point cloud file transmitted by the data source, the supplemental point cloud file including one or more other point cloud media tracks having the same point cloud content and the same frame rate as the point cloud media track to be displayed; Decode one or more other point cloud media tracks, and combine the one or more other point cloud media tracks with the point cloud media track to be displayed.
  • FIG. 6 shows a flowchart of steps of a point cloud encoding method in an embodiment of the present application.
  • the point cloud encoding method can be applied to links such as a server, a client, and an intermediate node of a point cloud media system.
  • the point cloud encoding method performed by the server device of the point cloud encoding apparatus is taken as an example.
  • the point cloud encoding method may mainly include the following steps S610 to S620.
  • Step S610 Encode the point cloud data to be transmitted according to different encoding standards to obtain multiple point cloud code streams with the same point cloud content, and the multiple point cloud code streams include some point cloud code streams with different frame rates.
  • the point cloud data of a certain point cloud content can be encoded according to a variety of different encoding standards to obtain multiple frames with the same frame rate or different frame rates.
  • the coding standard may include parameter values of one or more quality parameters, and multiple different coding standards may be formed by combining different parameter values of various quality parameters. For example, when the quality parameter includes code rate and frame rate, and the code rate includes two different code rate values A1 and A2, and the frame rate also includes two different frame rate values B1 and B2, it can be determined that the corresponding Four encoding standards for quality parameter values: A1B1, A1B2, A2B1, and A2B2.
  • Step S620 Encapsulate multiple point cloud code streams into multiple point cloud media tracks, and fill the multiple point cloud media tracks with frame rate indication information corresponding to the multiple point cloud code streams, and the frame rate indication information is used for Indicates the frame rate of multiple point cloud media tracks.
  • Each point cloud media track has corresponding file encapsulation information
  • the file encapsulation information can be an ISOBMFF data box generated when the point cloud code stream is encapsulated into a point cloud media track, for example, it can be extended track group data Box TrackGroupTypeBox.
  • the frame rate indication field corresponding to the frame rate indication information is determined in the file encapsulation information of the point cloud media track
  • the frame rate indication corresponding to the point cloud code stream can be indicated by referring to the frame rate of the point cloud code stream.
  • the information is filled into the frame rate indication field in the file encapsulation information.
  • streaming media signaling for transmitting point cloud data can be generated according to data transmission requirements; the streaming media signaling can be the above DASH signaling in an embodiment.
  • the streaming media signaling can be the above DASH signaling in an embodiment.
  • the track group includes one or more point cloud media tracks with the same point cloud content, and the track group includes some point cloud media tracks with different frame rates. .
  • streaming media signaling is sent to the data receiver for point cloud data transmission between the data source and the data receiver.
  • the data source may receive a data transmission request sent by the data receiver and generated based on the streaming media signaling;
  • the receiver transmits the point cloud file, the point cloud file includes one or more point cloud media tracks with the same point cloud content, and the point cloud file includes some point cloud media tracks with different frame rates.
  • FIG. 7 shows a flowchart of steps for encoding and decoding point cloud data in an application scenario according to an embodiment of the present application.
  • the server is used as the data source for producing point cloud data
  • the method for transmitting and encoding and decoding point cloud data between the server and the client where the user resides may include the following steps.
  • Step S701 Encode the point cloud content A on the server to obtain three point cloud code streams S1, S2 and S3 corresponding to two different frame rates.
  • the frame rate of point cloud code stream S1 is 60fps
  • the frame rate of point cloud code stream S2 is 30fps
  • the frame rate of point cloud code stream S3 is also 30fps.
  • Step S703 use the frameRate field in the DASH signaling to indicate the frame rate of each point cloud media track, use the time domain hierarchical group identifier GPCCTemporalScaleGroupId in the DASH signaling to indicate a track group composed of multiple point cloud media tracks, and send the DASH signaling To the client C1 and C2 where the user is located.
  • Step S704 The clients C1 and C2 request the point cloud file according to the network bandwidth and the information in the DASH signaling.
  • the point cloud file requested by C1 includes the point cloud media track Track1; the point cloud file requested by C2 includes the point cloud media track Track2.
  • Step S705 The server transmits the point cloud files to the clients C1 and C2 respectively.
  • the point cloud file transmitted by the server to the client C1 includes the point cloud media track Track1; the point cloud file transmitted to the client C2 includes the point cloud media track Track2.
  • Step S706 The client receives the point cloud file, decodes and displays the corresponding point cloud media track through the frameRate field information in Track1 and Track2.
  • the client C2 can further request the point cloud media track Track3, and display and consume the point cloud media track Track2 and the point cloud media track Track3 together, so as to achieve the point cloud display effect with a frame rate of 60fps.
  • FIG. 8 shows a structural block diagram of a point cloud decoding apparatus in an embodiment of the present application.
  • the point cloud decoding apparatus 800 may mainly include: a receiving module 810 configured to receive a point cloud file transmitted by a data source, where the point cloud file includes one or more point clouds with the same point cloud content media track, the point cloud file includes some point cloud media tracks with different frame rates; the parsing module 820 is configured to parse the file encapsulation information of the one or more point cloud media tracks, and obtain the file encapsulation information in the The frame rate indication information carried, where the frame rate indication information is used to indicate the frame rate of the one or more point cloud media tracks; the decoding module 830 is configured to, according to the frame rate indication information carried in the file encapsulation information , select and decode the point cloud media track with the specified frame rate from the point cloud file.
  • the receiving module 810 includes: a signaling receiving unit configured to receive streaming media signaling sent by a data source for transmitting point cloud data; signaling A parsing unit, configured to parse the streaming media signaling to obtain a time-domain hierarchical group identifier carried in the streaming media signaling and used to identify a track group, where the track group includes one or more points with the same point cloud content The point cloud media track, the track group includes some point cloud media tracks with different frame rates; the request sending unit is configured to send a first data transmission request to the data source according to the time domain hierarchical group identifier; A file receiving unit, configured to receive a point cloud file transmitted by the data source and corresponding to the first data transmission request.
  • the request sending unit includes: a bandwidth acquisition subunit, configured to acquire network bandwidth for data transmission with the data source; a track selection subunit, configured In order to select one or more target point cloud media tracks with a target frame rate matching the network bandwidth from the track group according to the time-domain hierarchical group identifier; the request sending subunit is configured to send The data source sends a first data transmission request for requesting transmission of the one or more target point cloud media tracks.
  • the parsing module 820 includes: an information parsing unit, configured to parse the file encapsulation information of the one or more point cloud media tracks to determine the a frame rate indication field corresponding to the frame rate of one or more point cloud media tracks; the information determination unit is configured to determine the frame of the one or more point cloud media tracks according to the value of the frame rate indication field rate indication information.
  • the point cloud decoding apparatus 800 further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a track selection module, configured by is configured to select other point cloud media tracks with the same frame rate as the point cloud media track to be displayed from the point cloud file according to the frame rate indication information carried in the file encapsulation information; the first replacement module, is configured to replace the point cloud media track to be presented with the other point cloud media track to decode and present the other point cloud media track.
  • a frame rate acquisition module configured to acquire the frame rate of the point cloud media track to be displayed
  • a track selection module configured by is configured to select other point cloud media tracks with the same frame rate as the point cloud media track to be displayed from the point cloud file according to the frame rate indication information carried in the file encapsulation information
  • the first replacement module is configured to replace the point cloud media track to be presented with the other point cloud media track to decode and present the other point cloud media track.
  • the point cloud decoding apparatus 800 further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a track selection module, configured by is configured to select one or more other point cloud media tracks with the same frame rate as the point cloud media track to be displayed from the point cloud file according to the frame rate indication information carried in the file encapsulation information; A merging module configured to decode the one or more other point cloud media tracks, and combine and display the one or more other point cloud media tracks and the to-be-displayed point cloud media track.
  • the point cloud decoding apparatus 800 further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a file acquisition module, configured by is configured to send a second data transmission request to the data source to receive a supplemental point cloud file transmitted by the data source, the supplemental point cloud file comprising the same point cloud content as the point cloud media track to be displayed and other point cloud media tracks of the same frame rate; the second replacement module is configured to replace the point cloud media track to be displayed with the other point cloud media tracks to decode and display the other point cloud media tracks .
  • the point cloud decoding apparatus 800 further includes: a frame rate acquisition module, configured to acquire the frame rate of the point cloud media track to be displayed; a file acquisition module, configured by is configured to send a third data transmission request to the data source to receive a supplemental point cloud file transmitted by the data source, the supplemental point cloud file comprising the same point cloud content as the point cloud media track to be displayed and one or more other point cloud media tracks of the same frame rate; the second merging module is configured to decode the one or more other point cloud media tracks and combine the one or more other point cloud media tracks with The point cloud media tracks to be displayed are combined and displayed.
  • FIG. 9 shows a structural block diagram of a point cloud encoding apparatus in an embodiment of the present application.
  • the point cloud encoding apparatus 900 may mainly include: an encoding module 910 configured to encode the point cloud data to be transmitted according to different encoding standards to obtain multiple point cloud code streams with the same point cloud content , the multiple point cloud code streams include some point cloud code streams with different frame rates; the encapsulation module 920 is configured to encapsulate the multiple point cloud code streams into multiple point cloud media tracks, and report to the Multiple point cloud media tracks are filled with frame rate indication information corresponding to the multiple point cloud code streams, where the frame rate indication information is used to indicate the frame rates of the multiple point cloud media tracks.
  • the point cloud encoding apparatus 900 further includes: a signaling generation module configured to generate streaming media signaling for transmitting point cloud data; a signaling filling module is configured to fill the streaming media signaling with a time-domain hierarchical group identifier for identifying a track group, where the track group includes one or more point cloud media tracks with the same point cloud content, and the track group includes one or more point cloud media tracks. It includes some point cloud media tracks with different frame rates; the signaling sending module is configured to send the streaming media signaling to the data receiver.
  • the point cloud encoding apparatus 900 further includes: a request receiving module configured to receive the data sent by the data receiver and generated based on the streaming media signaling. a data transmission request; a file transmission module configured to transmit a point cloud file to the data receiver according to the data transmission request, where the point cloud file includes one or more point cloud media tracks with the same point cloud content, The point cloud file includes some point cloud media tracks with different frame rates.
  • the encapsulation module 920 includes: an information determination unit configured to determine the frame rate indication in the file encapsulation information of the multiple point cloud media tracks a frame rate indication field corresponding to the information; the information filling unit is configured to fill the frame rate indication field in the file encapsulation information with the frame rate indication information corresponding to the plurality of point cloud code streams.
  • FIG. 10 schematically shows a structural block diagram of a computer system for implementing an electronic device according to an embodiment of the present application.
  • the computer system 1000 includes a central processing unit 1001 (Central Processing Unit, CPU), which can be loaded into a random device according to a program stored in a read-only memory 1002 (Read-Only Memory, ROM) or from a storage part 1008 Various appropriate actions and processes are performed by accessing the program in the memory 1003 (Random Access Memory, RAM). In the random access memory 1003, various programs and data necessary for system operation are also stored.
  • the central processing unit 1001 , the read-only memory 1002 and the random access memory 1003 are connected to each other through a bus 1004 .
  • An input/output interface 1005 (Input/Output interface, ie, I/O interface) is also connected to the bus 1004 .
  • the following components are connected to the input/output interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc. ; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a local area network card, a modem, and the like.
  • the communication section 1009 performs communication processing via a network such as the Internet.
  • a driver 1010 is also connected to the input/output interface 1005 as required.
  • a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 1010 as needed so that a computer program read therefrom is installed into the storage section 1008 as needed.
  • embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via the communication portion 1009, and/or installed from the removable medium 1011.
  • the central processing unit 1001 various functions defined in the system of the present application are executed.
  • the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above.
  • Computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Erasable Programmable Read Only Memory (EPROM), flash memory, optical fiber, portable Compact Disc Read-Only Memory (CD-ROM), optical storage device, magnetic storage device, or any suitable of the above The combination.
  • a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein.
  • Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium, other than a computer-readable storage medium, that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请属于计算机及通信技术领域，具体涉及一种点云编解码方法、点云编解码装置、计算机可读介质以及电子设备。本申请实施例中的点云解码方法包括：接收由数据源传输的点云文件，点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，点云文件中包括部分帧率不同的点云媒体轨道；解析一个或者多个点云媒体轨道的文件封装信息，得到文件封装信息中携带的帧率指示信息，帧率指示信息用于指示一个或者多个点云媒体轨道的帧率；根据文件封装信息中携带的帧率指示信息，从点云文件中选取并解码具有指定帧率的点云媒体轨道。

Description

点云编解码方法、装置、计算机可读介质以及电子设备
本申请要求2021年04月22日提交的申请号为202110437255.4、发明名称为“点云编解码方法及相关设备”的中国专利申请的优先权。
技术领域
本申请属于计算机及通信技术领域，具体涉及一种点云编解码方法、点云编解码装置、计算机可读介质以及电子设备。
背景技术
点云是空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集。在通过点云采集设备获取到大规模的点云数据后，可以对点云数据进行编码封装以向用户传输和呈现。
发明内容
本申请实施例提供了一种点云编解码方法、点云编解码装置、计算机可读介质以及电子设备。
根据本申请实施例的一个方面，提供一种点云解码方法，该方法包括：接收由数据源传输的点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道；解析所述一个或者多个点云媒体轨道的文件封装信息，得到所述文件封装信息中携带的帧率指示信息，所述帧率指示信息用于指示所述一个或者多个点云媒体轨道的帧率；根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道。
根据本申请实施例的一个方面，提供一种点云解码装置，该装置包括：接收模块，被配置为接收由数据源传输的点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道；解析模块，被配置为解析所述一个或者多个点云媒体轨道的文件封装信息，得到所述文件封装信息中携带的帧率指示信息，所述帧率指示信息用于指示所述一个或者多个点云媒体轨道的帧率；解码模块，被配置为根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道。
在本申请的一些实施例中，基于以上技术方案，所述接收模块包括：信令接收单元， 被配置为接收由数据源发送的用于传输点云数据的流媒体信令；信令解析单元，被配置为解析所述流媒体信令，得到所述流媒体信令中携带的用于标识轨道组的时域层级组标识，所述轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，所述轨道组中包括部分帧率不同的点云媒体轨道；请求发送单元，被配置为根据所述时域层级组标识，向所述数据源发送第一数据传输请求；文件接收单元，被配置为接收由所述数据源传输的与所述第一数据传输请求相对应的点云文件。
在本申请的一些实施例中，基于以上技术方案，所述请求发送单元包括：带宽获取子单元，被配置为获取与所述数据源进行数据传输的网络带宽；轨道选取子单元，被配置为根据所述时域层级组标识，从所述轨道组中选取与所述网络带宽相匹配的具有目标帧率的一个或者多个目标点云媒体轨道；请求发送子单元，被配置为向所述数据源发送用于请求传输所述一个或者多个目标点云媒体轨道的第一数据传输请求。
在本申请的一些实施例中，基于以上技术方案，所述解析模块包括：信息解析单元，被配置为解析所述一个或者多个点云媒体轨道的文件封装信息，以确定与所述一个或者多个点云媒体轨道的帧率相对应的帧率指示字段；信息确定单元，被配置为根据所述帧率指示字段的取值，确定所述一个或者多个点云媒体轨道的帧率指示信息。
在本申请的一些实施例中，基于以上技术方案，所述点云解码装置还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；轨道选取模块，被配置为根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取与所述待展示的点云媒体轨道具有相同帧率的其他点云媒体轨道；第一替换模块，被配置为将所述待展示的点云媒体轨道替换为所述其他点云媒体轨道，以解码并展示所述其他点云媒体轨道。
在本申请的一些实施例中，基于以上技术方案，所述点云解码装置还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；轨道选取模块，被配置为根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取与所述待展示的点云媒体轨道具有相同帧率的一个或者多个其他点云媒体轨道；第一合并模块，被配置为解码所述一个或者多个其他点云媒体轨道，并将所述一个或者多个其他点云媒体轨道与所述待展示的点云媒体轨道进行合并展示。
在本申请的一些实施例中，基于以上技术方案，所述点云解码装置还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；文件获取模块，被配置为向所述数据源发送第二数据传输请求，以接收由所述数据源传输的补充点云文件，所述补充点云文件包括与所述待展示的点云媒体轨道具有相同点云内容和相同帧率的其他点云媒体轨道；第二替换模块，被配置为将所述待展示的点云媒体轨道替换为所述其他点云媒体 轨道，以解码并展示所述其他点云媒体轨道。
在本申请的一些实施例中，基于以上技术方案，所述点云解码装置还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；文件获取模块，被配置为向所述数据源发送第三数据传输请求，以接收由所述数据源传输的补充点云文件，所述补充点云文件包括与所述待展示的点云媒体轨道具有相同点云内容和相同帧率的一个或者多个其他点云媒体轨道；第二合并模块，被配置为解码所述一个或者多个其他点云媒体轨道，并将所述一个或者多个其他点云媒体轨道与所述待展示的点云媒体轨道进行合并展示。
根据本申请实施例的一个方面，提供一种点云编码方法，该方法包括：按照不同的编码标准对待传输的点云数据进行编码处理，得到具有相同点云内容的多个点云码流，所述多个点云码流中包括部分帧率不同的点云码流；将所述多个点云码流封装为多个点云媒体轨道，并向所述多个点云媒体轨道中填充与所述多个点云码流相对应的帧率指示信息，所述帧率指示信息用于指示所述多个点云媒体轨道的帧率。
根据本申请实施例的一个方面，提供一种点云编码装置，该装置包括：编码模块，被配置为按照不同的编码标准对待传输的点云数据进行编码处理，得到具有相同点云内容的多个点云码流，所述多个点云码流中包括部分帧率不同的点云码流；封装模块，被配置为将所述多个点云码流封装为多个点云媒体轨道，并向所述多个点云媒体轨道中填充与所述多个点云码流相对应的帧率指示信息，所述帧率指示信息用于指示所述多个点云媒体轨道的帧率。
在本申请的一些实施例中，基于以上技术方案，所述点云编码装置还包括：信令生成模块，被配置为生成用于传输点云数据的流媒体信令；信令填充模块，被配置为向所述流媒体信令中填充用于标识轨道组的时域层级组标识，所述轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，所述轨道组中包括部分帧率不同的点云媒体轨道；信令发送模块，被配置为向数据接收方发送所述流媒体信令。
在本申请的一些实施例中，基于以上技术方案，所述点云编码装置还包括：请求接收模块，被配置为接收由所述数据接收方发送的基于所述流媒体信令生成的数据传输请求；文件传输模块，被配置为根据所述数据传输请求，向所述数据接收方传输点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道。
在本申请的一些实施例中，基于以上技术方案，所述封装模块包括：信息确定单元，被配置为在所述多个点云媒体轨道的文件封装信息中确定与所述帧率指示信息相对应的帧率指示字段；信息填充单元，被配置为将与所述多个点云码流相对应的帧率指示信息 填充至所述文件封装信息中的帧率指示字段。
根据本申请实施例的一个方面，提供一种计算机可读介质，其上存储有计算机程序，该计算机程序被处理器执行时实现如以上技术方案中的方法。
根据本申请实施例的一个方面，提供一种电子设备，该电子设备包括：处理器；以及存储器，用于存储所述处理器的可执行指令；其中，所述处理器被配置为经由执行所述可执行指令来执行如以上技术方案中的方法。
根据本申请实施例的一个方面，提供一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行如以上技术方案中的方法。
在本申请实施例提供的技术方案中，通过对具有相同帧率以及具有不同帧率的点云媒体资源进行关联，构造了点云媒体在时间上的渐进关系。基于这种时间上的渐进关系，用户在消费点云媒体时，可以根据自身需求请求对应的点云媒体文件，从而节省传输网络带宽，提高点云数据编解码的灵活性。
附图说明
图1示出了可以应用本申请实施例的技术方案的示例性系统架构的示意图。
图2示出点云编码装置和点云解码装置在流式传输环境中的放置方式。
图3示出了本申请一个实施例中的点云解码方法的步骤流程图。
图4示出了本申请一个实施例中以多轨道封装的可替换组示意图。
图5示出了本申请一个实施例中从数据源接收点云文件的步骤流程图。
图6示出了本申请一个实施例中的点云编码方法的步骤流程图。
图7示出了本申请实施例在一个应用场景中进行点云数据编解码的步骤流程图。
图8示意性地示出了本申请实施例提供的点云解码装置的结构框图。
图9示意性地示出了本申请实施例提供的点云编码装置的结构框图。
图10示意性示出了适于用来实现本申请实施例的电子设备的计算机系统结构框图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本申请将更 加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。
此外，所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施例中。在下面的描述中，提供许多具体细节从而给出对本申请的实施例的充分理解。然而，本领域技术人员将意识到，可以实践本申请的技术方案而没有特定细节中的一个或更多，或者可以采用其它的方法、组元、装置、步骤等。在其它情况下，不详细示出或描述公知方法、装置、实现或者操作以避免模糊本申请的各方面。
附图中所示的方框图仅仅是功能实体，不一定必须与物理上独立的实体相对应。即，可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
附图中所示的流程图仅是示例性说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解，而有的操作/步骤可以合并或部分合并，因此实际执行的顺序有可能根据实际情况改变。
点云是空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集。点云中的每个点至少具有三维位置信息，根据应用场景的不同，还可能具有色彩、材质或其他信息。通常，点云中的每个点都具有相同数量的附加属性。点云媒体从编码方式上又可以分为基于传统视频编码方式进行压缩的点云媒体(Video-based Point Cloud Compression，VPCC)以及基于几何特征进行压缩的点云媒体(Geometry-based Point Cloud Compression，GPCC)。在点云媒体的文件封装中，三维位置信息通常称为点云文件的几何分量(Geometry Component)，属性信息称为点云文件的属性分量(Attribute Component)。一个点云文件可能仅有一个几何分量，但可以存在一个或多个属性分量。
点云可以灵活方便地表达三维物体或场景的空间结构及表面属性，因而应用广泛，其主要应用场景可以归为两大类别。1)机器感知点云，例如自主导航系统、实时巡检系统、地理信息系统、视觉分拣机器人、抢险救灾机器人。2)人眼感知点云，例如数字文化遗产、自由视点广播、三维沉浸通信、三维沉浸交互等点云应用场景。
点云的获取主要有以下途径：计算机生成、3D激光扫描、3D摄影测量等。计算机可以生成虚拟三维物体及场景的点云。3D扫描可以获得静态现实世界三维物体或场景的点云，每秒可以获取百万级点云。3D摄像可以获得动态现实世界三维物体或场景的点云，每秒可以获取千万级点云。此外，在医学领域，由MRI、CT、电磁定位信息，可以获得生物组织器官的点云。这些技术降低了点云数据获取成本和时间周期，提高了数据的精度。点云数据获取方式的变革，使大量点云数据的获取成为可能。伴随着大规模的点云数据不断积累，点云数据的高效存储、传输、发布、共享和标准化，成为点云应用的关键。
在对点云媒体进行编码后，需要对编码后的数据流进行封装并传输给用户。相对应的，在点云媒体播放器端，需要先对点云文件进行解封装，然后再进行解码，最后将解码后的数据流呈现。
图1示出了可以应用本申请实施例的技术方案的示例性系统架构的示意图。
如图1所示，系统架构100包括多个终端装置，所述终端装置可通过例如网络150彼此通信。举例来说，系统架构100可以包括通过网络150互连的第一终端装置110和第二终端装置120。在图1的实施例中，第一终端装置110和第二终端装置120执行单向数据传输。
举例来说，第一终端装置110可以对点云数据(例如由第一终端装置110采集的点云码流)进行编码以通过网络150传输到第二终端装置120，已编码的点云数据以一个或多个已编码点云码流的形式传输，第二终端装置120可从网络150接收已编码点云数据，对已编码点云数据进行解码并显示已解码的点云数据。
在本申请的一个实施例中，系统架构100可以包括执行已编码点云数据的双向传输的第三终端装置130和第四终端装置140，所述双向传输比如可以发生在视频会议期间。对于双向数据传输，第三终端装置130和第四终端装置140中的每个终端装置可对点云数据(例如由终端装置采集的点云码流)进行编码，以通过网络150传输到第三终端装置130和第四终端装置140中的另一终端装置。第三终端装置130和第四终端装置140中的每个终端装置还可接收由第三终端装置130和第四终端装置140中的另一终端装置传输的已编码点云数据，且可对已编码点云数据进行解码以恢复点云数据，并可根据恢复的点云数据在可访问的显示装置上显示点云数据。
在图1的实施例中，第一终端装置110、第二终端装置120、第三终端装置130和第四终端装置140可为服务器、个人计算机和智能电话，但本申请公开的原理可不限于此。本申请公开的实施例适用于膝上型计算机、平板电脑、媒体播放器和/或专用视频会议设备。网络150表示在第一终端装置110、第二终端装置120、第三终端装置130和第四终端装置140之间传送已编码点云数据的任何数目的网络，包括例如有线和/或无线通信网络。网络150可在电路交换和/或分组交换信道中交换数据。该网络可包括电信网络、局域网、广域网和/或互联网。出于本申请的目的，除非在下文中有所解释，否则网络150的架构和拓扑对于本申请公开的操作来说可能是无关紧要的。
在本申请的一个实施例中，图2示出点云编码装置和点云解码装置在流式传输环境中的放置方式。本申请所公开主题可同等地适用于其它支持点云的应用，包括例如视频会议、数字电视、在包括CD、DVD、存储棒等的数字介质上存储已压缩的点云数据等等。
流式传输系统可包括采集子系统213，采集子系统213可包括数码相机等点云数据源201，点云数据源201例如可以创建未压缩的点云数据202。在实施例中，点云数据202包括由数码相机拍摄的样本。相较于已编码的点云数据204(或已编码的点云码流204)，点云数据202被描绘为粗线以强调高数据量的点云数据，点云数据202可由电子装置220处理，电子装置220包括耦接到视频源201的视频编码装置203。视频编码装置203可包括硬件、软件或软硬件组合以实现或实施如下文更详细地描述的所公开主题的各方面。相较于点云数据202，已编码的点云数据204(或已编码的点云码流204)被描绘为细线以强调较低数据量的已编码的点云数据204(或已编码的点云码流204)，其可存储在流式传输服务器205上以供将来使用。一个或多个流式传输客户端子系统，例如图2中的客户端子系统206和客户端子系统208，可访问流式传输服务器205以检索已编码的作为点云数据204的副本的点云数据207和点云数据209。客户端子系统206可包括例如电子装置230中的点云解码装置210。点云解码装置210对已编码的点云数据的传入副本207进行解码，且产生可在显示器212(例如显示屏)或另一呈现装置上呈现的输出点云数据211。在一些流式传输系统中，可根据某些点云编码/压缩标准对已编码的点云数据204、点云数据207和点云数据209(例如点云码流)进行编码。这些标准的实施例可以包括由MPEG为GPCC开发的标准。
应注意，电子装置220和电子装置230可包括图中未示出的其它组件。举例来说，电子装置220可包括点云解码装置，且电子装置230还可包括点云编码装置。
下面结合具体实施方式对本申请提供的点云编解码方法、点云编解码装置、计算机可读介质以及电子设备等技术方案做出详细说明。
图3示出了本申请一个实施例中的点云解码方法的步骤流程图，该方法可以应用于点云媒体系统的服务器、客户端以及中间节点等环节，本申请实施例以安装有点云解码装置的客户端设备执行的点云解码方法作为示例。如图3所示，该点云解码方法主要可以包括如下的步骤S310至步骤S330。
步骤S310：接收由数据源传输的点云文件，点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，点云文件中包括部分帧率不同的点云媒体轨道。
在本申请的一个实施例中，点云文件可以单独封装一个对应于指定帧率的点云媒体轨道，或者也可以封装多个具有相同或者不同帧率的点云媒体轨道。其中，多个点云媒体轨道可以组成一个轨道组，轨道组的内容可以包括单轨封装模式下的点云媒体轨道(包含几何成分、属性成分)或者可以包括多轨封装模式下的几何成分轨道(属性成分轨道由几何成分轨道索引得到)。相同帧率的点云媒体轨道在解码和展示时，可以相互 进行替代，而且相同帧率的点云媒体轨道可以组合消费，以达到更好的点云呈现效果。
在本申请的一个实施例中，点云文件中的部分点云媒体轨道具有相同的帧率，而其他的点云媒体轨道可以具有不同的帧率。例如，点云文件中封装track1、track2和track3三个点云媒体轨道，其中track1的帧率为60fps，而track2和track3的帧率均为30fps。
在点云文件中可以封装对应于不同点云媒体轨道的可替换组(alternative group)，具有相同点云内容且具有不同点云质量的点云媒体轨道可以划分在同一个可替换组内，点云质量可以包括码率、帧率、分辨率等各种对应于不同标准的质量参数。
当GPCC点云数据以单轨道封装时，不同质量的点云内容的轨道同属一个可替换组。当GPCC点云数据以多轨道封装时，不同质量的点云内容的几何成分轨道同属一个可替换组，而属性成分轨道可以关联至几何成分轨道。
图4示出了本申请一个实施例中以多轨道封装的可替换组示意图。如图4所示，在可替换组400中包括具有相同点云内容的第一点云数据410和第二点云数据420。其中，第一点云数据410是点云质量相对较高的无损压缩点云数据(Lossless coded GPCC)，第二点云数据420是点云质量相对较低的有损压缩点云数据(Lossy coded GPCC)。
第一点云数据410中包括第一几何成分轨道411和关联至第一几何成分轨道411的第一属性成分轨道412，第二点云数据420中包括第二几何成分轨道421和关联至第二几何成分轨道421的第二属性成分轨道422。
步骤S320：解析一个或者多个点云媒体轨道的文件封装信息，得到文件封装信息中携带的帧率指示信息，帧率指示信息用于指示一个或者多个点云媒体轨道的帧率。
在本申请的一个实施例中，解析文件封装信息得到质量指示信息(帧率指示信息)的方法可以包括：解析一个或者多个点云媒体轨道的文件封装信息，以确定与一个或者多个点云媒体轨道的帧率相对应的帧率指示字段；根据帧率指示字段的取值，确定一个或者多个点云媒体轨道的帧率指示信息。
在本申请的一个实施例中，文件封装信息是在将点云码流封装为点云媒体轨道时生成的ISOBMFF(ISO基础媒体文件格式)数据盒，ISOBMFF的具体内容可参考国际标准ISO/IEC 14496-12。
在本申请的一个实施例中，文件封装信息具体可以表现为扩展得到的轨道组数据盒TrackGroupTypeBox，其语法如下。
Figure PCTCN2022080266-appb-000001
Figure PCTCN2022080266-appb-000002
其中，frame_rate为用于指示点云文件对应帧率的帧率指示信息，其取值是一个长度为8字节的无符号整数。
通过扩展轨道组数据盒，可以将同一内容、不同帧率的点云媒体轨道进行相互关联。同属于一个轨道组的点云轨道满足如下条件。
(1)该轨道为单轨封装模式下的点云轨道(包含几何、属性成分)或者多轨封装模式下的几何成分轨道(属性成分轨道由几何成分轨道索引得到)。
(2)同帧率的轨道在解码和呈现时可以相互替代。
(3)同帧率的轨道可以组合消费，以达到更高帧率的呈现效果。
步骤S330：根据文件封装信息中携带的帧率指示信息，从点云文件中选取并解码具有指定帧率的点云媒体轨道。
在每个点云媒体轨道对应的文件封装信息中，均携带有该点云媒体轨道的帧率指示信息，该帧率指示信息以显性标识的方式标识出点云媒体轨道的帧率。当点云文件由数据源传输至用户所在的数据接收方时，数据接收方可以根据设备性能以及用户需求解码具有指定的帧率的点云媒体轨道。
在本申请的一个实施例中，可以通过采集数据接收方的设备性能，并将设备性能与文件封装信息中携带的质量指示信息(帧率指示信息)进行匹配检测，确定与数据接收方的设备性能相匹配的帧率，进而从点云文件中选取并解码具有指定帧率的点云媒体轨道。
设备性能可以包括硬件性能、软件性能和网络性能中的至少一种。硬件性能例如可以包括电子设备的设备型号、处理器型号、存储器容量、显示器尺寸等等，软件性能例如可以包括数据接收方安装的点云解码器的程序版本，网络性能例如可以包括网络带宽、网络传输状态等等。
在本申请的一个实施例中，可以获取数据接收方配置的帧率选取规则，并将帧率选取规则与文件封装信息中携带的帧率指示信息进行匹配检测，确定与数据接收方配置帧率选取规则相匹配的帧率，进而从点云文件中选取并解码具有指定帧率的点云媒体轨道。
帧率选取规则可以是根据用户需求配置的用于选取具有指定帧率的点云数据的选取规则，例如根据用户指令选取帧率大于(或者小于)指定数值的点云数据。
图5示出了本申请一个实施例中从数据源接收点云文件的步骤流程图。如图5所示，在以上实施例的基础上，步骤S310中的接收由数据源传输的点云文件，可以包括如下的步骤S510至步骤S540。
步骤S510：接收由数据源发送的用于传输点云数据的流媒体信令。
在本申请的一个实施例中，用于传输点云数据的流媒体信令可以是基于HTTP的动态自适应流(dynamic adaptive streaming over HTTP，DASH)信令，DASH是一种自适应比特率流技术，该技术可以使高质量流媒体通过传统的HTTP网络服务器进行互联网传递。
步骤S520：解析流媒体信令，得到流媒体信令中携带的用于标识轨道组的时域层级组标识，轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，轨道组中包括部分帧率不同的点云媒体轨道。
在DASH信令中，可以利用已有的字段来存储点云媒体轨道的帧率指示信息，例如可以利用DASH信令中的frameRate字段来指示多种不同点云媒体轨道的帧率。
在本申请的一个实施例中，对于在文件封装中同属于一个gpts轨道组的所有帧率的点云媒体轨道，在DASH信令中可以通过一个组ID进行标识，例如，GPCC时域层级组标识(GPCCTemporalScaleGroupId)。该组ID元素为自适应集AdaptationSet元素的子元素。GPCCTemporalScaleGroupId元素可以出现在adaptation set层级，而不能出现在任何其他层级。表1示出了本申请一个实施例中GPCC时域层级组ID的语义及属性。
表1
Figure PCTCN2022080266-appb-000003
步骤S530：根据时域层级组标识，向所述数据源发送数据传输请求。
根据解析得到的流媒体信令中携带的时域层级组标识，可以选择其中一种或者多种帧率指示信息作为目标帧率，并进一步向数据源发送对应于目标帧率的数据传输请求。该数据传输请求例如为第一数据传输请求。
在本申请的一个实施例中，发送数据传输请求的方法可以包括：获取与数据源进行数据传输的网络带宽；根据时域层级组标识，从轨道组中选取与网络带宽相匹配的具有目标帧率的一个或者多个目标点云媒体轨道；向数据源发送用于请求传输目标点云媒体轨道的数据传输请求。
在本申请的一个实施例中，当网络带宽大于设定阈值时，可以选取两个或者两个以 上的目标点云媒体轨道。当网络带宽小于或等于设定阈值时，可以选取一个目标点云媒体轨道。
步骤S540：接收由数据源传输的与数据传输请求相对应的点云文件。
当根据时域层级组标识向数据源发送针对一个对应于目标点云媒体轨道的数据传输请求时，数据源可以基于该请求向数据接收方传输相应的一个目标点云媒体轨道。当根据时域层级组标识向数据源发送针对多个对应于目标点云媒体轨道的数据传输请求时，数据源可以基于该请求向数据接收方传输相应的具有相同帧率或者不同帧率的多个目标点云媒体轨道。
在本申请的一个实施例中，通过获取具有相同帧率的多个点云媒体文件，可以对各个点云媒体文件进行替换或者合并，以改善点云媒体文件的展示效果。
在本申请的一个实施例中，当待展示的点云媒体轨道出现选取失败、解码失败或者解码后得到点云质量差等问题时，可以利用具有相同帧率的其他点云媒体轨道进行轨道替换。例如，两个帧率均为30fps的具有相同点云内容的点云媒体文件可以相互进行替换。
在本申请的一个实施例中，当数据接收方的网络环境得到优化、网络带宽较高时，可以将多个具有相同帧率的点云媒体轨道进行轨道合并，以提高点云数据的展示帧率。例如，两个帧率均为30fps的具有相同点云内容的点云媒体轨道可以经过轨道合并后形成一个帧率为60fps的点云媒体轨道，从而以提高帧率的方式获得更好的点云媒体文件展示效果。
在本申请的一个实施例中，当从点云文件中选取并解码出待展示的点云媒体轨道后，如果点云文件中还包括有具有相同帧率的其他点云媒体轨道，则可以执行点云媒体轨道替换方法：获取待展示的点云媒体轨道的帧率；根据文件封装信息中携带的帧率指示信息，从点云文件中选取与待展示的点云媒体轨道具有相同帧率的其他点云媒体轨道；将待展示的点云媒体轨道替换为其他点云媒体轨道，以解码并展示其他点云媒体轨道。
在本申请的一个实施例中，当从点云文件中选取并解码出待展示的点云媒体轨道后，如果点云文件中还包括有具有相同帧率的其他点云媒体轨道，则可以执行点云媒体轨道合并方法：获取待展示的点云媒体轨道的帧率；根据文件封装信息中携带的帧率指示信息，从点云文件中选取与待展示的点云媒体轨道具有相同帧率的一个或者多个其他点云媒体轨道；解码一个或者多个其他点云媒体轨道，并将一个或者多个其他点云媒体轨道与待展示的点云媒体轨道进行合并展示。
在本申请的一个实施例中，当从点云文件中选取并解码出待展示的点云媒体轨道后，如果点云文件仅封装有一个点云媒体轨道，或者点云文件中的其他点云媒体轨道的帧率 均与待展示的点云媒体轨道不一致，则可以执行点云媒体轨道替换方法：获取待展示的点云媒体轨道的帧率；向数据源发送数据传输请求(例如，第二数据传输请求)，以接收由数据源传输的补充点云文件，补充点云文件包括与待展示的点云媒体轨道具有相同点云内容和相同帧率的其他点云媒体轨道；将待展示的点云媒体轨道替换为其他点云媒体轨道，以解码并展示其他点云媒体轨道。
在本申请的一个实施例中，当从点云文件中选取并解码出待展示的点云媒体轨道后，如果点云文件仅封装有一个点云媒体轨道，或者点云文件中的其他点云媒体轨道的帧率均与待展示的点云媒体轨道不一致，则可以执行点云媒体轨道合并方法：获取待展示的点云媒体轨道的帧率；向数据源发送数据传输请求(例如，第三数据传输请求)，以接收由数据源传输的补充点云文件，补充点云文件包括与待展示的点云媒体轨道具有相同点云内容和相同帧率的一个或者多个其他点云媒体轨道；解码一个或者多个其他点云媒体轨道，并将一个或者多个其他点云媒体轨道与待展示的点云媒体轨道进行合并展示。
图6示出了本申请一个实施例中的点云编码方法的步骤流程图，该点云编码方法可以应用于点云媒体系统的服务器、客户端以及中间节点等环节，本申请实施例以安装有点云编码装置的服务器设备执行的点云编码方法作为示例。如图6所示，该点云编码方法主要可以包括如下的步骤S610至步骤S620。
步骤S610：按照不同的编码标准对待传输的点云数据进行编码处理，得到具有相同点云内容的多个点云码流，多个点云码流中包括部分帧率不同的点云码流。
为了满足不同的数据接收方对于点云数据的帧率需求，针对某一点云内容的点云数据，可以按照多种不同的编码标准进行编码处理后得到多个具有相同帧率或者不同帧率的点云码流。编码标准可以包括一种或者多种质量参数的参数取值，将各种质量参数的不同参数取值进行组合可以形成多种不同的编码标准。例如当质量参数包括码率和帧率，而码率包括两种不同的码率取值A1、A2，帧率也包括两种不同的帧率取值B1、B2，由此可以确定对应于不同质量参数取值的四种编码标准：A1B1、A1B2、A2B1和A2B2。
步骤S620：将多个点云码流封装为多个点云媒体轨道，并向多个点云媒体轨道中填充与多个点云码流相对应的帧率指示信息，帧率指示信息用于指示多个点云媒体轨道的帧率。
每个点云媒体轨道均具有与之对应的文件封装信息，该文件封装信息可以是在将点云码流封装为点云媒体轨道时生成的ISOBMFF数据盒，例如可以是扩展得到的轨道组数据盒TrackGroupTypeBox。在点云媒体轨道的文件封装信息中确定与帧率指示信息相对应的帧率指示字段之后，可以通过引用点云码流的帧率的方式，将与点云码流相对应的帧 率指示信息填充至文件封装信息中的帧率指示字段。
在本申请的一个实施例中，在将点云码流封装为点云媒体轨道之后，可以根据数据传输需求，生成用于传输点云数据的流媒体信令；该流媒体信令可以是以上实施例中的DASH信令。向流媒体信令中填充用于标识轨道组的时域层级组标识，轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，轨道组中包括部分帧率不同的点云媒体轨道。然后向数据接收方发送流媒体信令，以便在数据源与数据接收方之间进行点云数据传输。
在本申请的一个实施例中，当数据接收方对流媒体信令做出响应后，数据源可以接收由数据接收方发送的基于流媒体信令生成的数据传输请求；根据数据传输请求，向数据接收方传输点云文件，点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，点云文件中包括部分帧率不同的点云媒体轨道。
图7示出了本申请实施例在一个应用场景中进行点云数据编解码的步骤流程图。如图7所示，服务器作为生产点云数据的数据源，在服务器与用户所在客户端之间进行点云数据传输和编解码的方法可以包括如下步骤。
步骤S701：对服务器上的点云内容A进行编码处理，得到对应于两种不同帧率的三个点云码流S1、S2和S3。例如，点云码流S1的帧率为60fps，点云码流S2的帧率为30fps，点云码流S3的帧率也是30fps。
步骤S702：将三个点云码流分别封装为三个相互关联的点云媒体轨道Track1、Track2和Track3，对应填充每个点云媒体轨道的帧率指示信息。例如，在轨道组数据盒中，将点云媒体轨道Track1的帧率指示字段填充为frame_rate＝60；将点云媒体轨道Track2的帧率指示字段填充为frame_rate＝30；将点云媒体轨道Track3的帧率指示字段填充为frame_rate＝30。
步骤S703：利用DASH信令中的frameRate字段指示各个点云媒体轨道的帧率，利用DASH信令中的时域层级组标识GPCCTemporalScaleGroupId指示多个点云媒体轨道组成的轨道组，将DASH信令发送给用户所在的客户端C1和C2。
步骤S704：客户端C1和C2根据网络带宽和DASH信令中的信息，请求点云文件。其中，C1请求的点云文件中包括点云媒体轨道Track1；C2请求的点云文件中包括点云媒体轨道Track2。
步骤S705：服务器分别向客户端C1和C2传输点云文件。服务器向客户端C1传输的点云文件中包括点云媒体轨道Track1；向客户端C2传输的点云文件中包括点云媒体轨道Track2。
步骤S706：客户端接收点云文件，通过Track1和Track2中的frameRate字段信息解 码并展示相应的点云媒体轨道。
当客户端C2的网络状态好转后，可以进一步请求点云媒体轨道Track3，将点云媒体轨道Track2与点云媒体轨道Track3一起进行呈现消费，从而达到帧率为60fps的点云展示效果。
在本申请实施例提供的技术方案中，通过对具有相同帧率以及具有不同帧率的点云媒体资源进行关联，构造了点云媒体在时间上的渐进关系。基于这种时间上的渐进关系，用户在消费点云媒体时，可以根据自身需求请求对应的点云媒体文件，从而节省传输带宽。
应当注意，尽管在附图中以特定顺序描述了本申请中方法的各个步骤，但是，这并非要求或者暗示必须按照该特定顺序来执行这些步骤，或是必须执行全部所示的步骤才能实现期望的结果。附加的或备选的，可以省略某些步骤，将多个步骤合并为一个步骤执行，以及/或者将一个步骤分解为多个步骤执行等。
以下介绍本申请的装置实施例，可以用于执行本申请上述实施例中的点云编解码方法。
图8示出了本申请一个实施例中的点云解码装置的结构框图。如图8所示，点云解码装置800主要可以包括：接收模块810，被配置为接收由数据源传输的点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道；解析模块820，被配置为解析所述一个或者多个点云媒体轨道的文件封装信息，得到所述文件封装信息中携带的帧率指示信息，所述帧率指示信息用于指示所述一个或者多个点云媒体轨道的帧率；解码模块830，被配置为根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道。
在本申请的一些实施例中，基于以上各实施例，所述接收模块810包括：信令接收单元，被配置为接收由数据源发送的用于传输点云数据的流媒体信令；信令解析单元，被配置为解析所述流媒体信令，得到所述流媒体信令中携带的用于标识轨道组的时域层级组标识，所述轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，所述轨道组中包括部分帧率不同的点云媒体轨道；请求发送单元，被配置为根据所述时域层级组标识，向所述数据源发送第一数据传输请求；文件接收单元，被配置为接收由所述数据源传输的与所述第一数据传输请求相对应的点云文件。
在本申请的一些实施例中，基于以上各实施例，所述请求发送单元包括：带宽获取子单元，被配置为获取与所述数据源进行数据传输的网络带宽；轨道选取子单元，被配置为根据所述时域层级组标识，从所述轨道组中选取与所述网络带宽相匹配的具有目标帧率的一个或者多个目标点云媒体轨道；请求发送子单元，被配置为向所述数据源发送 用于请求传输所述一个或者多个目标点云媒体轨道的第一数据传输请求。
在本申请的一些实施例中，基于以上各实施例，所述解析模块820包括：信息解析单元，被配置为解析所述一个或者多个点云媒体轨道的文件封装信息，以确定与所述一个或者多个点云媒体轨道的帧率相对应的帧率指示字段；信息确定单元，被配置为根据所述帧率指示字段的取值，确定所述一个或者多个点云媒体轨道的帧率指示信息。
在本申请的一些实施例中，基于以上各实施例，所述点云解码装置800还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；轨道选取模块，被配置为根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取与所述待展示的点云媒体轨道具有相同帧率的其他点云媒体轨道；第一替换模块，被配置为将所述待展示的点云媒体轨道替换为所述其他点云媒体轨道，以解码并展示所述其他点云媒体轨道。
在本申请的一些实施例中，基于以上各实施例，所述点云解码装置800还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；轨道选取模块，被配置为根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取与所述待展示的点云媒体轨道具有相同帧率的一个或者多个其他点云媒体轨道；第一合并模块，被配置为解码所述一个或者多个其他点云媒体轨道，并将所述一个或者多个其他点云媒体轨道与所述待展示的点云媒体轨道进行合并展示。
在本申请的一些实施例中，基于以上各实施例，所述点云解码装置800还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；文件获取模块，被配置为向所述数据源发送第二数据传输请求，以接收由所述数据源传输的补充点云文件，所述补充点云文件包括与所述待展示的点云媒体轨道具有相同点云内容和相同帧率的其他点云媒体轨道；第二替换模块，被配置为将所述待展示的点云媒体轨道替换为所述其他点云媒体轨道，以解码并展示所述其他点云媒体轨道。
在本申请的一些实施例中，基于以上各实施例，所述点云解码装置800还包括：帧率获取模块，被配置为获取待展示的点云媒体轨道的帧率；文件获取模块，被配置为向所述数据源发送第三数据传输请求，以接收由所述数据源传输的补充点云文件，所述补充点云文件包括与所述待展示的点云媒体轨道具有相同点云内容和相同帧率的一个或者多个其他点云媒体轨道；第二合并模块，被配置为解码所述一个或者多个其他点云媒体轨道，并将所述一个或者多个其他点云媒体轨道与所述待展示的点云媒体轨道进行合并展示。
图9示出了本申请一个实施例中的点云编码装置的结构框图。如图9所示，点云编码装置900主要可以包括：编码模块910，被配置为按照不同的编码标准对待传输的点云数据进行编码处理，得到具有相同点云内容的多个点云码流，所述多个点云码流中包括部 分帧率不同的点云码流；封装模块920，被配置为将所述多个点云码流封装为多个点云媒体轨道，并向所述多个点云媒体轨道中填充与所述多个点云码流相对应的帧率指示信息，所述帧率指示信息用于指示所述多个点云媒体轨道的帧率。
在本申请的一些实施例中，基于以上各实施例，所述点云编码装置900还包括：信令生成模块，被配置为生成用于传输点云数据的流媒体信令；信令填充模块，被配置为向所述流媒体信令中填充用于标识轨道组的时域层级组标识，所述轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，所述轨道组中包括部分帧率不同的点云媒体轨道；信令发送模块，被配置为向数据接收方发送所述流媒体信令。
在本申请的一些实施例中，基于以上各实施例，所述点云编码装置900还包括：请求接收模块，被配置为接收由所述数据接收方发送的基于所述流媒体信令生成的数据传输请求；文件传输模块，被配置为根据所述数据传输请求，向所述数据接收方传输点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道。
在本申请的一些实施例中，基于以上各实施例，所述封装模块920包括：信息确定单元，被配置为在所述多个点云媒体轨道的文件封装信息中确定与所述帧率指示信息相对应的帧率指示字段；信息填充单元，被配置为将与所述多个点云码流相对应的帧率指示信息填充至所述文件封装信息中的帧率指示字段。
本申请各实施例中提供的点云编解码装置的具体细节已经在对应的方法实施例中进行了详细的描述，此处不再赘述。
图10示意性地示出了用于实现本申请实施例的电子设备的计算机系统结构框图。
需要说明的是，图10示出的电子设备的计算机系统1000仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图10所示，计算机系统1000包括中央处理器1001(Central Processing Unit，CPU)，其可以根据存储在只读存储器1002(Read-Only Memory，ROM)中的程序或者从存储部分1008加载到随机访问存储器1003(Random Access Memory，RAM)中的程序而执行各种适当的动作和处理。在随机访问存储器1003中，还存储有系统操作所需的各种程序和数据。中央处理器1001、在只读存储器1002以及随机访问存储器1003通过总线1004彼此相连。输入/输出接口1005(Input/Output接口，即I/O接口)也连接至总线1004。
以下部件连接至输入/输出接口1005：包括键盘、鼠标等的输入部分1006；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal Display，LCD)等以及扬声器等的输出部分1007；包括硬盘等的存储部分1008；以及包括诸如局域网卡、调 制解调器等的网络接口卡的通信部分1009。通信部分1009经由诸如因特网的网络执行通信处理。驱动器1010也根据需要连接至输入/输出接口1005。可拆卸介质1011，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器1010上，以便于从其上读出的计算机程序根据需要被安装入存储部分1008。
特别地，根据本申请的实施例，各个方法流程图中所描述的过程可以被实现为计算机软件程序。例如，本申请的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分1009从网络上被下载和安装，和/或从可拆卸介质1011被安装。在该计算机程序被中央处理器1001执行时，执行本申请的系统中限定的各种功能。
需要说明的是，本申请实施例所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、有线等等，或者上述的任意合适的组合。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (15)

  1. 一种点云解码方法，其特征在于，包括：
    接收由数据源传输的点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道；
    解析所述一个或者多个点云媒体轨道的文件封装信息，得到所述文件封装信息中携带的帧率指示信息，所述帧率指示信息用于指示所述一个或者多个点云媒体轨道的帧率；
    根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道。
  2. 根据权利要求1所述的点云解码方法，其特征在于，接收由数据源传输的点云文件，包括：
    接收由数据源发送的用于传输点云数据的流媒体信令；
    解析所述流媒体信令，得到所述流媒体信令中携带的用于标识轨道组的时域层级组标识，所述轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，所述轨道组中包括部分帧率不同的点云媒体轨道；
    根据所述时域层级组标识，向所述数据源发送第一数据传输请求；
    接收由所述数据源传输的与所述第一数据传输请求相对应的点云文件。
  3. 根据权利要求2所述的点云解码方法，其特征在于，根据所述时域层级组标识，向所述数据源发送第一数据传输请求，包括：
    获取与所述数据源进行数据传输的网络带宽；
    根据所述时域层级组标识，从所述轨道组中选取与所述网络带宽相匹配的具有目标帧率的一个或者多个目标点云媒体轨道；
    向所述数据源发送用于请求传输所述一个或者多个目标点云媒体轨道的第一数据传输请求。
  4. 根据权利要求1所述的点云解码方法，其特征在于，解析所述一个或者多个点云媒体轨道的文件封装信息，得到所述文件封装信息中携带的帧率指示信息，包括：
    解析所述一个或者多个点云媒体轨道的文件封装信息，以确定与所述一个或者多个点云媒体轨道的帧率相对应的帧率指示字段；
    根据所述帧率指示字段的取值，确定所述一个或者多个点云媒体轨道的帧率指示信息。
  5. 根据权利要求1～4中任一项所述的点云解码方法，其特征在于，在根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道之后，所述方法还包括：
    获取待展示的点云媒体轨道的帧率；
    根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取与所述待展示的点云媒体轨道具有相同帧率的其他点云媒体轨道；
    将所述待展示的点云媒体轨道替换为所述其他点云媒体轨道，以解码并展示所述其他点云媒体轨道。
  6. 根据权利要求1～4中任一项所述的点云解码方法，其特征在于，在根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道之后，所述方法还包括：
    获取待展示的点云媒体轨道的帧率；
    根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取与所述待展示的点云媒体轨道具有相同帧率的一个或者多个其他点云媒体轨道；
    解码所述一个或者多个其他点云媒体轨道，并将所述一个或者多个其他点云媒体轨道与所述待展示的点云媒体轨道进行合并展示。
  7. 根据权利要求1～4中任一项所述的点云解码方法，其特征在于，在根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道之后，所述方法还包括：
    获取待展示的点云媒体轨道的帧率；
    向所述数据源发送第二数据传输请求，以接收由所述数据源传输的补充点云文件，所述补充点云文件包括与所述待展示的点云媒体轨道具有相同点云内容和相同帧率的其他点云媒体轨道；
    将所述待展示的点云媒体轨道替换为所述其他点云媒体轨道，以解码并展示所述其他点云媒体轨道。
  8. 根据权利要求1～4中任一项所述的点云解码方法，其特征在于，在根据所述文件封装信息中携带的帧率指示信息，从所述点云文件中选取并解码具有指定帧率的点云媒体轨道之后，所述方法还包括：
    获取待展示的点云媒体轨道的帧率；
    向所述数据源发送第三数据传输请求，以接收由所述数据源传输的补充点云文件，所述补充点云文件包括与所述待展示的点云媒体轨道具有相同点云内容和相同帧率的一个或者多个其他点云媒体轨道；
    解码所述一个或者多个其他点云媒体轨道，并将所述一个或者多个其他点云媒体轨道与所述待展示的点云媒体轨道进行合并展示。
  9. 一种点云编码方法，其特征在于，包括：
    按照不同的编码标准对待传输的点云数据进行编码处理，得到具有相同点云内容的多个点云码流，所述多个点云码流中包括部分帧率不同的点云码流；
    将所述多个点云码流封装为多个点云媒体轨道，并向所述多个点云媒体轨道中填充与所述多个点云码流相对应的帧率指示信息，所述帧率指示信息用于指示所述多个点云媒体轨道的帧率。
  10. 根据权利要求9所述的点云编码方法，其特征在于，在将所述多个点云码流封装为多个点云媒体轨道之后，所述方法还包括：
    生成用于传输点云数据的流媒体信令；
    向所述流媒体信令中填充用于标识轨道组的时域层级组标识，所述轨道组包括一个或者多个具有相同点云内容的点云媒体轨道，所述轨道组中包括部分帧率不同的点云媒体轨道；
    向数据接收方发送所述流媒体信令。
  11. 根据权利要求10所述的点云编码方法，其特征在于，在向数据接收方发送所述流媒体信令之后，所述方法还包括：
    接收由所述数据接收方发送的基于所述流媒体信令生成的数据传输请求；
    根据所述数据传输请求，向所述数据接收方传输点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道。
  12. 根据权利要求9～11中任一项所述的点云解码方法，其特征在于，向所述多个点云媒体轨道中填充与所述多个点云码流相对应的帧率指示信息，包括：
    在所述多个点云媒体轨道的文件封装信息中确定与所述帧率指示信息相对应的帧率指示字段；
    将与所述多个点云码流相对应的帧率指示信息填充至所述文件封装信息中的帧率指示字段。
  13. 一种点云解码装置，其特征在于，包括：
    接收模块，被配置为接收由数据源传输的点云文件，所述点云文件包括一个或者多个具有相同点云内容的点云媒体轨道，所述点云文件中包括部分帧率不同的点云媒体轨道；
    解析模块，被配置为解析所述一个或者多个点云媒体轨道的文件封装信息，得到所述文件封装信息中携带的帧率指示信息，所述帧率指示信息用于指示所述一个或者多个点云媒体轨道的帧率；
    解码模块，被配置为根据所述文件封装信息中携带的帧率指示信息从所述点云文件 中选取并解码具有指定帧率的点云媒体轨道。
  14. 一种点云编码装置，其特征在于，包括：
    编码模块，被配置为按照不同的编码标准对待传输的点云数据进行编码处理，得到具有相同点云内容的多个点云码流，所述多个点云码流中包括部分帧率不同的点云码流；
    封装模块，被配置为将所述多个点云码流封装为多个点云媒体轨道，并向所述多个点云媒体轨道中填充与所述多个点云码流相对应的帧率指示信息，所述帧率指示信息用于指示所述多个点云媒体轨道的帧率。
  15. 一种电子设备，其特征在于，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1至12中任意一项所述的方法。
PCT/CN2022/080266 2021-04-22 2022-03-11 点云编解码方法、装置、计算机可读介质以及电子设备 WO2022222641A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
KR1020237031030A KR20230144620A (ko) 2021-04-22 2022-03-11 포인트 클라우드 인코딩 및 디코딩 방법, 포인트 클라우드 인코딩 및 디코딩 장치, 컴퓨터로 판독 가능한 매체, 그리고 전자 디바이스
JP2023552564A JP2024508865A (ja) 2021-04-22 2022-03-11 点群符号化・復号方法、装置、及び電子機器
US17/982,927 US20230061573A1 (en) 2021-04-22 2022-11-08 Point Cloud Encoding and Decoding Method and Apparatus, Computer-Readable Medium, and Electronic Device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110437255.4 2021-04-22
CN202110437255.4A CN115243053B (zh) 2021-04-22 2021-04-22 点云编解码方法及相关设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/982,927 Continuation US20230061573A1 (en) 2021-04-22 2022-11-08 Point Cloud Encoding and Decoding Method and Apparatus, Computer-Readable Medium, and Electronic Device

Publications (1)

Publication Number Publication Date
WO2022222641A1 true WO2022222641A1 (zh) 2022-10-27

Family

ID=83666808

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/080266 WO2022222641A1 (zh) 2021-04-22 2022-03-11 点云编解码方法、装置、计算机可读介质以及电子设备

Country Status (6)

Country Link
US (1) US20230061573A1 (zh)
JP (1) JP2024508865A (zh)
KR (1) KR20230144620A (zh)
CN (1) CN115243053B (zh)
TW (1) TWI803274B (zh)
WO (1) WO2022222641A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190139266A1 (en) * 2017-11-09 2019-05-09 Samsung Electronics Co., Ltd. Point cloud compression using non-orthogonal projection
WO2020060813A1 (en) * 2018-09-18 2020-03-26 Vid Scale, Inc. Methods and apparatus for point cloud compression bitstream format
US20200132822A1 (en) * 2018-10-29 2020-04-30 Dji Technology, Inc. User interface for displaying point clouds generated by a lidar device on a uav
WO2020137642A1 (ja) * 2018-12-28 2020-07-02 ソニー株式会社 情報処理装置および情報処理方法
CN114079781A (zh) * 2020-08-18 2022-02-22 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法、装置、设备及存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110012279B (zh) * 2018-01-05 2020-11-17 上海交通大学 基于3d点云数据的分视角压缩和传输方法及系统
US10984541B2 (en) * 2018-04-12 2021-04-20 Samsung Electronics Co., Ltd. 3D point cloud compression systems for delivery and access of a subset of a compressed 3D point cloud
US11049266B2 (en) * 2018-07-31 2021-06-29 Intel Corporation Point cloud viewpoint and scalable compression/decompression
CN113170238B (zh) * 2018-09-12 2023-08-01 诺基亚技术有限公司 用于视频编码和解码的装置、方法和计算机程序
WO2020070379A1 (en) * 2018-10-03 2020-04-09 Nokia Technologies Oy Method and apparatus for storage and signaling of compressed point clouds
US11457231B2 (en) * 2019-03-15 2022-09-27 Mediatek Singapore Pte. Ltd. Methods and apparatus for signaling spatial relationships for point cloud multimedia data tracks
US11581022B2 (en) * 2019-05-29 2023-02-14 Nokia Technologies Oy Method and apparatus for storage and signaling of compressed point clouds
US11704867B2 (en) * 2019-09-27 2023-07-18 Intel Corporation Methods for timed metadata priority rank signaling for point clouds
WO2021022266A2 (en) * 2019-10-07 2021-02-04 Futurewei Technologies, Inc. Video-based point cloud compression (v-pcc) timing information
US11302063B2 (en) * 2020-07-21 2022-04-12 Facebook Technologies, Llc 3D conversations in an artificial reality environment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190139266A1 (en) * 2017-11-09 2019-05-09 Samsung Electronics Co., Ltd. Point cloud compression using non-orthogonal projection
WO2020060813A1 (en) * 2018-09-18 2020-03-26 Vid Scale, Inc. Methods and apparatus for point cloud compression bitstream format
US20200132822A1 (en) * 2018-10-29 2020-04-30 Dji Technology, Inc. User interface for displaying point clouds generated by a lidar device on a uav
WO2020137642A1 (ja) * 2018-12-28 2020-07-02 ソニー株式会社 情報処理装置および情報処理方法
CN114079781A (zh) * 2020-08-18 2022-02-22 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
JP2024508865A (ja) 2024-02-28
KR20230144620A (ko) 2023-10-16
CN115243053B (zh) 2024-04-16
US20230061573A1 (en) 2023-03-02
TWI803274B (zh) 2023-05-21
TW202243481A (zh) 2022-11-01
CN115243053A (zh) 2022-10-25

Similar Documents

Publication Publication Date Title
US20190158933A1 (en) Method, device, and computer program for improving streaming of virtual reality media content
CN112804256B (zh) 多媒体文件中轨道数据的处理方法、装置、介质及设备
CN114697668B (zh) 点云媒体的编解码方法及相关产品
WO2023029858A1 (zh) 点云媒体文件的封装与解封装方法、装置及存储介质
EP3637722A1 (en) Method and apparatus for processing media information
CN115396645A (zh) 一种沉浸媒体的数据处理方法、装置、设备及存储介质
WO2022206200A1 (zh) 点云编解码方法、装置、计算机可读介质及电子设备
WO2023226504A1 (zh) 一种媒体数据处理方法、装置、设备以及可读存储介质
CN115396647B (zh) 一种沉浸媒体的数据处理方法、装置、设备及存储介质
WO2022222641A1 (zh) 点云编解码方法、装置、计算机可读介质以及电子设备
CN115150368B (zh) 媒体文件的关联处理方法、装置、介质及电子设备
WO2023169003A1 (zh) 点云媒体的解码方法、点云媒体的编码方法及装置
WO2022134962A1 (zh) 点云视窗的呈现方法、装置、计算机可读介质及电子设备
US20240129537A1 (en) Method and apparatus for signaling cmaf switching sets in isobmff
KR102661694B1 (ko) 미디어 파일 캡슐화 방법, 미디어 파일 캡슐화 해제 방법 및 관련 디바이스
WO2023024843A1 (zh) 媒体文件封装与解封装方法、设备及存储介质
WO2023169004A1 (zh) 点云媒体的数据处理方法、装置、设备及介质
CN116455880A (zh) 流媒体传输方法及相关产品
CN116347118A (zh) 一种沉浸媒体的数据处理方法及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22790750

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023552564

Country of ref document: JP

ENP Entry into the national phase

Ref document number: 20237031030

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 1020237031030

Country of ref document: KR

NENP Non-entry into the national phase

Ref country code: DE