WO2023284487A1 - 容积媒体的数据处理方法、装置、设备以及存储介质 - Google Patents

容积媒体的数据处理方法、装置、设备以及存储介质 Download PDF

Info

Publication number
WO2023284487A1
WO2023284487A1 PCT/CN2022/099706 CN2022099706W WO2023284487A1 WO 2023284487 A1 WO2023284487 A1 WO 2023284487A1 CN 2022099706 W CN2022099706 W CN 2022099706W WO 2023284487 A1 WO2023284487 A1 WO 2023284487A1
Authority
WO
WIPO (PCT)
Prior art keywords
media
replacement
playback
group
attribute information
Prior art date
Application number
PCT/CN2022/099706
Other languages
English (en)
French (fr)
Inventor
胡颖
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2023284487A1 publication Critical patent/WO2023284487A1/zh
Priority to US18/224,165 priority Critical patent/US20230360678A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/363—Image reproducers using image projection screens
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip

Definitions

  • the present application relates to the field of computer technology, specifically to the field of point cloud technology, and in particular to a data processing method, device, device and storage medium for volumetric media.
  • a point cloud is a set of discrete points that are randomly distributed in space and express the spatial structure and surface properties of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may also have color, material or other information depending on the application scenario.
  • multi-view video and free viewpoint use real or virtual cameras to capture 3D scene information, and support the presentation of 3D 3D scenes in six degrees of freedom (6DoF) within a limited range of viewing positions and directions.
  • Volumetric media usually includes multiple media tracks, such as video tracks, audio tracks, text tracks, etc.; in practice, it is found that when there is a certain relationship between the media tracks of the volumetric media, the playback terminal only needs to read part of the volumetric media. The track is decoded to reconstruct and present the volumetric media. For example, when the volumetric media includes two video tracks with a replacement relationship, only one of the video tracks needs to be decoded by the playback terminal. However, since the playback terminal cannot recognize the relationship between these media tracks, the playback terminal needs to perform decoding operations on all media tracks of the volumetric media, resulting in waste of processing resources of the playback terminal.
  • a volumetric medium data processing method, device, device, and storage medium are provided.
  • an embodiment of the present application provides a volumetric media data processing method, executed by a service device, including:
  • the replacement relationship indication information is used to indicate media track pairs with a replacement relationship in each of the two playback groups , the above replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with the replacement relationship;
  • an embodiment of the present application provides a volumetric media data processing method, executed by a service device, including:
  • Each of the at least two play groups is marked to obtain the replacement relationship indication information and replacement attribute information between the above-mentioned every two play groups;
  • the replacement relationship indication information is used to indicate that each of the two play groups A media track pair with a replacement relationship, the above-mentioned replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with a replacement relationship;
  • an embodiment of the present application provides a volumetric media data processing method, executed by a playback terminal, including:
  • the replacement relationship indication information is used to indicate media track pairs with a replacement relationship in each of the two playback groups , the above replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with the replacement relationship;
  • the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information identify a target play group that matches the play terminal from the above-mentioned at least two play groups;
  • Decoding the above-mentioned target playback group is performed to obtain the above-mentioned volume media.
  • an embodiment of the present application provides a volumetric media data processing device, including:
  • the obtaining module is used to obtain the replacement relationship indication information and replacement attribute information between every two play groups in the at least two play groups of the volume media;
  • a media track pair with a relationship the above-mentioned replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with a replacement relationship;
  • a sending module configured to send the above replacement relationship indication information and the above replacement attribute information to the playback terminal, so as to instruct the playback terminal to select from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information.
  • a target playback group matching the playback terminal is identified, and decoding processing is performed on the target playback group to obtain the volume medium.
  • an embodiment of the present application provides a volumetric media data processing device, including:
  • An acquisition module configured to acquire at least two playback groups of volume media
  • a marking module configured to mark each of the at least two play groups to obtain the replacement relationship indication information and replacement attribute information between the above-mentioned two play groups;
  • the replacement relationship indication information is used to indicate the above-mentioned There are media track pairs with a replacement relationship in every two play groups, and the above-mentioned replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pairs with a replacement relationship;
  • a sending module configured to send the above replacement relationship indication information and the above replacement attribute information to the playback terminal, so as to instruct the playback terminal to select from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information.
  • a target playback group matching the playback terminal is identified, and decoding processing is performed on the target playback group to obtain the volume media.
  • an embodiment of the present application provides a volumetric media data processing device, including:
  • the obtaining module is used to obtain the replacement relationship indication information and replacement attribute information between every two play groups in the at least two play groups of the volume media;
  • a media track pair with a relationship the above-mentioned replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with a replacement relationship;
  • An identification module configured to identify, from the at least two playback groups, a target playback group that matches the playback terminal according to the replacement relationship indication information and the replacement attribute information;
  • the decoding module is configured to perform decoding processing on the above-mentioned target play group to obtain the above-mentioned volume media.
  • One aspect of the present application provides a computer device, including: one or more processors and memory;
  • the above-mentioned memory is used to store computer-readable instructions
  • the above-mentioned processor is used to call the above-mentioned computer-readable instructions to execute the steps in implementing various method embodiments of the present application.
  • One aspect of the present application provides a computer program product, including computer readable instructions, and when the computer readable instructions are executed by one or more processors, the steps in the various method embodiments of the present application are implemented.
  • an embodiment of the present application provides one or more computer-readable storage media.
  • the above-mentioned computer-readable storage medium stores computer-readable instructions, and the above-mentioned computer-readable instructions include program instructions.
  • the above-mentioned program instructions are executed by a processor, To implement the steps in the various method embodiments of the present application.
  • FIG. 1 shows a structural diagram of a volumetric media system provided by the present application
  • FIG. 2 shows a schematic diagram of a 6DoF provided by the present application
  • FIG. 3 shows a schematic diagram of a 3DoF provided by an embodiment of the present application
  • FIG. 4 shows a schematic diagram of a 3DoF+ provided by the present application
  • Fig. 5 shows a schematic diagram of a medium track of a volumetric medium provided by the present application
  • FIG. 6 is a flow diagram of a volumetric media data processing method provided by the present application.
  • Fig. 7 shows a schematic diagram of a medium track of a volumetric medium provided by the present application
  • Fig. 8 shows a schematic diagram of a media track of a volumetric medium provided by the present application
  • FIG. 9 is a flow diagram of a volumetric media data processing method provided by the present application.
  • FIG. 10 is a flow diagram of a volumetric media data processing method provided by the present application.
  • FIG. 11 is a schematic diagram of a scenario for obtaining replacement relationship indication information and replacement attribute information provided by the present application.
  • FIG. 12 is a schematic diagram of a scenario for obtaining replacement relationship indication information and replacement attribute information provided by the present application.
  • FIG. 13 is a schematic structural diagram of a volumetric media data processing device provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a volumetric media data processing device provided by an embodiment of the present application.
  • Fig. 15 is a schematic structural diagram of a volumetric media data processing device provided by an embodiment of the present application.
  • FIG. 16 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • volumetric video-based coding media refers to immersive media that captures visual content in three-dimensional space and provides 3DoF+ and 6DoF viewing experience, is encoded with traditional video, and contains volumetric video-type tracks in the file package, including multiple Perspective video, video encoded point cloud, etc.
  • Fig. 1 shows a structure diagram of a volume media system provided by an exemplary embodiment of the present application; as shown in Fig. 1, the volume media system includes a service device and a playback terminal, and the service device may refer to the
  • the computer equipment used can be a terminal (such as a PC (Personal Computer, personal computer), an intelligent mobile device (such as a smart phone), etc.) or a server.
  • the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud Cloud servers for basic cloud computing services such as communications, middleware services, domain name services, security services, content delivery network (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
  • the playback terminal can refer to the computer equipment used by the user of the volume media, and the computer equipment can be a terminal (such as a PC (Personal Computer, personal computer), a smart mobile device (such as a smart phone), a VR device (such as a VR helmet, a VR glasses, etc.)).
  • the data processing process of the volumetric media includes the data processing process at the service device side and the data processing process at the playback terminal side.
  • the data processing process at the service device side mainly includes: (1) the acquisition and production process of the media content of the volumetric media; (2) the process of encoding and packaging files of the volumetric media.
  • the data processing process at the playback terminal mainly includes: (3) the process of decapsulating and decoding the volumetric media file; (4) the rendering process of the volumetric media.
  • the transmission process involving volumetric media between the service device and the playback terminal can be carried out based on various transmission protocols.
  • the transmission protocols here can include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming Streaming Media Transport) Protocol, MMT (Moving Picture Experts Group Media Transport, Dynamic Picture Experts Group Media Transport Protocol), HLS (HTTP Live Streaming, Dynamic Bit Rate Adaptive Transmission) Protocol, SMTP (Smart Media Transport Protocol, Intelligent Media Transport Protocol), TCP (Transmission Control Protocol, Transmission Control Protocol), etc.
  • DASH Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming Streaming Media Transport
  • MMT Motion Picture Experts Group Media Transport, Dynamic Picture Experts Group Media Transport Protocol
  • HLS HTTP Live Streaming, Dynamic Bit Rate Adaptive Transmission
  • SMTP Smart Media Transport Protocol, Intelligent Media Transport Protocol
  • TCP Transmission Control Protocol
  • the media content of volumetric media is obtained by capturing real-world audio-visual scenes through capture devices.
  • the capture device may refer to a hardware component provided in the service device, for example, the capture device refers to a microphone, camera, sensor, etc. of the terminal.
  • the capture device may also be a hardware device connected to the service device, for example, a camera connected to the service device; used to provide the service device with an acquisition service of volumetric media content.
  • the capture device may include but not limited to: audio device, camera device and sensor device. Wherein, the audio device may include an audio sensor, a microphone, and the like.
  • the camera device may include a common camera, a stereo camera, a light field camera, and the like.
  • Sensing devices may include laser devices, radar devices, and the like.
  • the number of capture devices can be multiple, such as camera arrays. These capture devices are deployed at some specific positions in the real space to simultaneously capture audio and video content from different angles in the space.
  • the audio content and video content in the audio and video content Synchronized both in time and space.
  • the captured audio content is itself content suitable for performing audio encoding of volumetric media.
  • the captured video content can only become the content suitable for video encoding of the volumetric media after undergoing a series of production processes.
  • the production process includes:
  • splicing refers to splicing the video content shot at these various angles into a complete video that can reflect the 360-degree visual panorama of the real space, that is, the spliced video is a panoramic video (or spherical video) represented in three-dimensional space.
  • Projection refers to the process of mapping a three-dimensional video formed by splicing onto a two-dimensional (3-Dimension, 2D) image.
  • the 2D image formed by projection is called a projection image; projection methods may include but are not limited to: latitude and longitude map projection, Hexahedral projection.
  • the projected image can be encoded directly, or the projected image can be encoded after area encapsulation.
  • the encoding of two-dimensional projection images after region packing can greatly improve the video coding efficiency of volumetric media, so region packing technology is widely used in the video processing process of volumetric media .
  • area encapsulation refers to the process of transforming the projected image by area, and the area encapsulation process converts the projected image into an encapsulated image.
  • the process of area encapsulation specifically includes: dividing the projected image into multiple mapping areas, and then transforming the multiple mapping areas to obtain multiple encapsulation areas, and mapping the multiple encapsulation areas into a 2D image to obtain an encapsulation image.
  • the mapping area refers to the area obtained by dividing in the projected image before performing area encapsulation;
  • the encapsulating area refers to the area located in the encapsulating image after performing area encapsulation. Transformation processing may include, but is not limited to: mirroring, rotation, rearrangement, up-sampling, down-sampling, changing the resolution of an area, and moving.
  • the capture device can only capture panoramic video, after such video is processed by the service device and transmitted to the playback terminal for corresponding data processing, the user on the playback terminal side can only perform some specific actions (such as heading body rotation) to watch 360-degree video information, but performing non-specific actions (such as moving the head) cannot obtain corresponding video changes, and the VR experience is not good, so it is necessary to provide additional depth information that matches the panoramic video to make the Users get better immersion and better VR experience, which involves 6DoF (Six Degrees of Freedom, six degrees of freedom) production technology. When the user can move more freely in the simulated scene, it is called 6DoF.
  • the capture device When 6DoF production technology is used to produce volumetric media video content, the capture device generally uses light field cameras, laser equipment, radar equipment, etc. to capture point cloud data or light field data in space, and execute the above production process1-
  • the process of 3 also needs to carry out some specific processing, such as the process of cutting and mapping point cloud data, the calculation process of depth information, etc.
  • FIG. 2 shows a schematic diagram of 6DoF provided by an exemplary embodiment of the present application
  • 6DoF is divided into window 6DoF, omnidirectional 6DoF and 6DoF
  • window 6DoF means that the user's rotation and movement on the X-axis and Y-axis are limited, and Translation in the Z axis is limited; for example, the user cannot see outside the window frame, and the user cannot walk through the window.
  • Omni-directional 6DoF means that the user's rotation and movement on the X-axis, Y-axis and Z-axis are limited. For example, the user cannot freely pass through the three-dimensional 360-degree VR content in the restricted movement area.
  • 6DoF means that users can freely translate along the X-axis, Y-axis, and Z-axis. For example, users can move freely in three-dimensional 360-degree VR content. Similar to 6DoF, there are 3DoF and 3DoF+ production technologies.
  • Figure 3 shows a schematic diagram of 3DoF provided by an exemplary embodiment of the present application; as shown in Figure 3, 3DoF means that the user is fixed at the center point of a three-dimensional space, and the user's head is along the X axis, Y axis and Z axis Rotate to view the footage provided by the media content.
  • Figure 4 shows a schematic diagram of 3DoF+ provided by an exemplary embodiment of the present application. As shown in Figure 4, 3DoF+ means that when the virtual scene provided by the volumetric media has certain depth information, the user's head can Move within the space to watch the screen provided by the media content.
  • the captured audio content can be directly encoded to form an audio track of the volumetric media.
  • video encoding is performed on the video content to obtain the video track of the volumetric media.
  • a volumetric video track carrying the mapping relationship between 2D and 3D is obtained;
  • a packaged image is encoded to obtain a volumetric video track carrying packaged data.
  • the video stream carrying the position data is encoded to obtain the volumetric video track carrying the position data; the position data refers to the three-dimensional position information of the shooting object.
  • video coding includes AVC (Advanced Video Coding, Advanced Video Coding), HVC (High Video Coding, high-definition video coding), AVS (Audio Video Coding Standard, China National Video Coding Standard), HEVC/H.265 (High Efficiency Video Coding, International Video Coding Standard), VVC/H.266 (Versatile Video Coding, International Video Coding Standard).
  • AVC Advanced Video Coding, Advanced Video Coding
  • HVC High Video Coding, high-definition video coding
  • AVS Analog Video Coding Standard
  • HEVC/H.265 High Efficiency Video Coding, International Video Coding Standard
  • VVC/H.266 Very Low Video Coding, International Video Coding Standard
  • the service device can use different video encoding methods to encode video content, so that media track pairs with replacement relationship can be obtained.
  • Media track pairs with replacement relationship use different video encoding methods for the same video content.
  • the track type of the volumetric video track in the media track pair that can be called the replacement relationship is the same
  • FIG. 5 is a schematic diagram of the replacement relationship between volumetric video tracks in volumetric media V3C.
  • Video track 1 is a volumetric video track carrying 2D and 3D mapping relationships.
  • the track types of volumetric video track 2 and volumetric video track 5 are both volumetric video tracks carrying attribute data, and the encoding methods of volumetric video track 2 and volumetric video track 5 are avc1 (that is, advanced video coding) and hvc1 (that is, high-definition video coding ). Therefore, volumetric video track 2 and volumetric video track 5 have a replacement relationship, and the replacement identifier of volumetric video track 2 and volumetric video track 5 is 10, that is, volumetric video track 2 and volumetric video track 5 constitute media track pair 10.
  • volumetric video track 3 and volumetric video track 6 are both volumetric video tracks carrying position data, and the encoding methods of volumetric video track 3 and volumetric video track 6 are avc1 and hvc1 respectively. Therefore, there is a replacement relationship between the volume video track 3 and the volume video track 6, and the replacement identification of the volume video track 3 and the volume video track 6 is 11, that is, the volume video track 3 and the volume video track 6 can form a media track pair 11.
  • the track types of volumetric video track 4, volumetric video track 7, and volumetric video track 8 are all volumetric video tracks carrying placeholder map data, and the encoding methods of volumetric video track 4, volumetric video track 7, and volumetric video track 8 are avc1, hvc1, vvc1.
  • volumetric video track 4 there is a replacement relationship between any two volumetric video tracks among volumetric video track 4, volumetric video track 7, and volumetric video track 8, and the replacement identifiers of volumetric video track 4, volumetric video track 7, and volumetric video track 8 are 12, Any two volumetric video tracks among the volumetric video track 4 , the volumetric video track 7 , and the volumetric video track 8 form a media track pair 12 .
  • each volumetric video track can be grouped to obtain multiple playback groups.
  • the volumetric video tracks in the playback group have a joint playback relationship, that is, the volumetric video tracks in the playback group can form a complete volumetric media (that is, volumetric video), and the playback group includes only one volume in the media pair with a replacement relationship video track.
  • Each volumetric track in Fig. 5 can form a plurality of play groups, as in Fig.
  • volume video track 1-4 can form play group 1
  • volume video track 1 can form play group 2
  • volume video track 1 can form play group 3
  • volume video tracks 5-6 and volumetric video track 8 can form play group 3.
  • the combination of other volumetric video tracks can also constitute a playback group.
  • audio coding includes but is not limited to: MP3 (Moving Picture Experts Group Audio Layer III, moving picture experts compression standard audio level 3), AAC (Advanced Audio Coding, advanced audio coding), FLAC (Free Lossless Audio Codec, lossless audio compression encoding).
  • the service device can use different audio encoding methods to generate audio track pairs with a replacement relationship, and can add an audio track (one of the audio track pairs with a replacement relationship) to each play group, and generate two play groups
  • the replacement relationship indication information and the replacement attribute information, the replacement relationship indication information is used to indicate the media track pair with the replacement relationship in every two play groups, and the replacement attribute information is used to reflect the media track in the media track pair with the replacement relationship (such as volumetric video tracks or audio tracks).
  • the attribute difference may include the coding mode of the media track in the media track pair and so on.
  • the playback group, the replacement relationship indication information and the replacement attribute information can be encapsulated in the file container to form the media file resource of the volume media according to the file format of the volume media (such as ISOBMFF (ISO Base Media File Format, ISO Base Media File Format))
  • the media file resource may be a media file or a media file that forms a volumetric media from a media segment.
  • MPD Media presentation description
  • the metadata here is a general term for the information related to the presentation of the volumetric media.
  • the metadata may include description information of the media content, description information of the window, signaling information related to the presentation of the media content, and the like.
  • the service device stores media presentation description information and media file resources formed after data processing.
  • the playback terminal can dynamically obtain the media file resources of the volumetric media and the corresponding media presentation description information from the service device through the recommendation of the service device or adaptively and dynamically according to the processing capability of the playback terminal.
  • the playback terminal may request the service device to obtain corresponding media file resources according to the decoding capability of the playback terminal.
  • Media file resources and media presentation description information are transmitted from the service device to the playback terminal through a transmission mechanism (such as DASH, SMT).
  • the file decapsulation process at the playback terminal is opposite to the file encapsulation process at the service device.
  • the playback terminal decapsulates the media file resources according to the file format requirements of the volumetric media to obtain audio tracks and volumetric video tracks.
  • the decoding process of the playback terminal is opposite to the encoding process of the server device.
  • the playback terminal performs audio decoding on the audio track to restore the audio content.
  • the playback terminal decodes the volumetric video track to obtain video content.
  • the playback terminal renders the audio content obtained by audio decoding and the 3D image obtained by video decoding according to the metadata related to rendering and window in the media presentation description information. After the rendering is completed, the playback and output of the 3D image is realized.
  • the playback terminal mainly renders the 3D image based on the current viewpoint, disparity, depth information, etc.; if the production technology of 6DoF is used, the playback terminal mainly renders the 3D image in the window based on the current viewpoint. rendering.
  • the viewpoint refers to the viewing position of the user
  • the parallax refers to the visual difference caused by the user's binoculars or due to movement
  • the window refers to the viewing area.
  • the volumetric media system supports a data box (Box).
  • a data box refers to a data block or object that includes the playback group of the volumetric media, the indication information of the replacement relationship between each two playback groups, the replacement attribute information, and the metadata. .
  • the playback terminal by sending the replacement relationship indication information and replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing capability or user needs, so as to reconstruct and present the volumetric media. Personalization is possible. In addition, there is no need to perform decoding processing on all media tracks, which can save processing resources of the playback terminal.
  • FIG. 6 is a schematic flowchart of a volumetric media data processing method provided by an embodiment of the present application. As shown in FIG. 6, the method may be performed by the application service device in FIG. 1, where the method may at least include the following S101-S102:
  • the replacement relationship indication information is used to indicate the media that has a replacement relationship in each of the two playback groups
  • the replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with a replacement relationship.
  • the media track in the media track pair may refer to volumetric video track, audio track, text track, etc.; every two playback groups of volumetric media include a media track pair or at least two media tracks with a replacement relationship. track pair.
  • the attribute difference between the media tracks in the media track pair can be used to reflect the quality difference of the playback group, and the attribute difference between the media tracks in the media track pair can include the attribute information of each media track in the media track pair, and the media track
  • the attribute information includes one or more items of encoding method, transmission rate, quality level, and encoding type of the encoding method.
  • the coding method may include a video coding method and an audio coding method, and the coding type of the coding method may include a lossy coding method, a lossless coding method, and the like.
  • the attribute information of the media track may be used to reflect the playback quality of the media track, and further, may be used to reflect the playback quality of the playback group to which the media track belongs.
  • the playback quality of a media track whose encoding type is lossless encoding is better than that of a media track whose encoding type is lossy encoding.
  • the transmission resources (such as network bandwidth) consumed by the channel to transmit the media track and the resources consumed by the playback terminal to process the media track are related to the playback quality of the media track. For example, the higher the quality level of the media track, the more transmission resources are consumed by the channel to transmit the media track, and the resources consumed by the playback terminal to process the media track are more. On the contrary, the lower the quality level of the media track is, the less transmission resources are consumed by the channel to transmit the media track, and the resources consumed by the playback terminal to process the media track are less.
  • the media track pair includes a first media track belonging to the playing group P m and a second media track belonging to the playing group P n
  • the replacement attribute information between the playing group P m and the playing group P n includes the The attribute information of the first media track, and the attribute information of the second media track, m and n are all positive integers less than Z, and Z is the grouping quantity of the at least two playing groups;
  • the attribute information of the first media track includes The encoding mode of the first media track
  • the attribute information of the second media track includes the encoding mode of the second media track, the encoding mode of the first media track is different from the encoding mode of the second media track.
  • both the first media track and the second media track are volumetric video tracks
  • the encoding mode of the first media track is avc1
  • the encoding mode of the second media track is hvc1.
  • the attribute information of the first media track also includes one or more of the encoding type corresponding to the transmission rate, structural organization, quality level and encoding mode of the first media track; the attribute information of the second media track
  • the information also includes one or more of the transmission rate, structural organization, quality level, and encoding type corresponding to the encoding method of the second media track, and the transmission rate includes one or both of the frame rate or code rate of the media track .
  • the replacement relationship indication information also includes a replacement level field; when the replacement level field in the replacement relationship indication information between the play group P m and the play group P n is the first value (such as 1), the The replacement level field is used to reflect that the replacement level between the playing group P m and the playing group P n is the media content level; when the replacement level field in the replacement relationship indication information of the playing group P m and the playing group P n is When the second value (such as 0), the replacement level field is used to reflect that the replacement level between the playgroup P m and the playgroup Pn is a track level.
  • the replacement level field in the replacement relationship indication information between the play group P m and the play group P n is the first value (such as 1)
  • the replacement level field is used to reflect that the replacement level between the playing group P m and the playing group P n is the media content level
  • the replacement level field in the replacement relationship indication information of the playing group P m and the playing group P n is When the second value (such as 0), the replacement level
  • the replacement level between the play group P m and the play group P n is the media content level
  • the number of media track pairs with a replacement relationship between the play group P m and the play group P n is R
  • R is the number of media tracks in the play group P m
  • the number of media tracks in the play group P m is the same as the number of tracks in the play group P n ; that is to say, when the play group P m and the play group P
  • the replacement level between n is the media content level, there is a one-to-one replacement relationship between the play group P m and the media tracks of the play group P n .
  • volume media includes two playing groups, namely playing group 1 and playing group 2.
  • Playing group 1 and playing group 2 both include media tracks carrying location data and media tracks carrying attribute data.
  • the encoding type of each media track in the playback group 1 is a lossless encoding type
  • the encoding type of each media track in the playback group 2 is a lossy encoding type. Therefore, there is a replacement relationship between the media track carrying position data in play group 1 and the media track carrying position data in play group 2, and a media track pair can be formed.
  • the number of tracks in playgroup 1 and playgroup 2 is 2, and the media track pair between playgroup 1 and playgroup 2 is 2. Therefore, the replacement level between playgroup 1 and playgroup 2 can be called Media content level.
  • the replacement level between the play group P m and the play group P n is a track level
  • the number of media track pairs with a replacement relationship between the play group P m and the play group P n is less than R positive integer of . That is to say, when the replacement level between the playing group P m and the playing group P n is the track level, there is no substitution relationship between the media tracks in the playing group P m and the media tracks in the playing group P n , that is, there is a replacement relationship between some media tracks in the playgroup Pm and media tracks in the playgroup Pn .
  • the volumetric medium includes 6 volumetric video tracks, which are respectively volumetric video tracks 1-6, and volumetric video track 1 is a volumetric video track carrying a mapping relationship between 2D and 3D.
  • the play group of volume media can at least include play group 1 and play group 2, play group 1 can include volume video track 1, volume video track 2, volume video track 3, play group 2 can include volume video track 1, volume video track 5 , Volume video track6.
  • the track types of volumetric video track 2 and volumetric video track 5 are both volumetric video tracks carrying attribute data, and the encoding methods of volumetric video track 2 and volumetric video track 5 are avc1 and hvc1 respectively.
  • volumetric video track 2 and volumetric video track 5 have a replacement relationship, and the replacement identifier of volumetric video track 2 and volumetric video track 5 is 10, that is, volumetric video track 2 and volumetric video track 5 constitute media track pair 10.
  • the track types of volumetric video track 3 and volumetric video track 6 are both volumetric video tracks carrying position data, and the encoding methods of volumetric video track 3 and volumetric video track 6 are avc1 and hvc1 respectively. Therefore, there is a replacement relationship between the volume video track 3 and the volume video track 6, and the replacement identification of the volume video track 3 and the volume video track 6 is 11, that is, the volume video track 3 and the volume video track 6 can form a media track pair 11.
  • the replacement level between playgroup 1 and playgroup 2 can be called track level.
  • the encoding methods of the media tracks in the play group P m are all the first encoding methods; the play group P The coding modes of the media tracks in n are all the second coding mode. Both use the same encoding method to encode the media tracks in the playback group, which is beneficial for the playback terminal to quickly decode the media tracks, quickly obtain volumetric media, and improve the efficiency of reconstructing volumetric media.
  • the track class is the same between the media tracks in the media track pair of the relationship.
  • the track category can be used to reflect the content contained in the media track. If the track category includes audio track, video track, text track, etc., further, the track category of the video track can be subdivided according to the data carried by the video track.
  • the track of the video track Categories include volumetric video tracks that carry mappings between 2D and 3D, volumetric video tracks that carry attribute data, volumetric video tracks that carry position data, and more.
  • the media tracks with the joint playback relationship can refer to the volume media that can be reconstructed according to the media tracks with the joint playback relationship, that is, constitute a complete volume medium.
  • the replacement relationship indication information includes replacement identifiers of media track pairs that have a replacement relationship in each of the two playing groups.
  • the service device can obtain the replacement relationship indication information and replacement attribute information between every two playback groups in at least two playback groups of the volumetric media through the data box or transmission protocol of the volumetric media.
  • the service device may acquire the data box of the volume media; the data box of the volume media includes the indication information of the replacement relationship and the replacement attribute information between every two playback groups.
  • the data box may also include at least two playback groups of volume media, that is, the data box may be obtained by encapsulating at least two playback groups of volume media, replacement relationship indication information, and replacement attribute information.
  • the syntax of a data box based on a volumetric medium can be found in Table 1 below:
  • is_content_alternative in Table 1 is an alternative level field between two play groups, and the value of this field is 1, indicating that the alternative level between the two play groups is a media content level.
  • the value of this field is 0, indicating that the replacement level between the two play groups is the track level, that is, there is a replacement relationship between some tracks in the two play groups.
  • alternative_info_flag replaces the identification field, the value of this field is 1, and the media track in the play group carries differential information, that is, the media track is a media track in the media track pair, and the value of this field is 0, indicating that the media track does not carry Differentiation information, that is, the media track does not belong to a media track in the media track pair.
  • PlayoutAlternativeInfoBox is a data box field
  • alternative_type the difference attribute type of an alternative track.
  • the track can have one or more difference attributes.
  • the quality_ranking quality level field the smaller the value of this field, the higher the quality of the corresponding track.
  • lossless_flag The value of this field is 0, indicating that the encoding type of the encoding method corresponding to the media track is a lossy encoding type; the value of this field is 1, indicating that the encoding type of the encoding method of the corresponding media track is a lossless encoding type.
  • Bitrate is the bit rate information of the media track, indicating the bit rate of the corresponding media track.
  • Framerate is the frame rate information of the media track, and indicates the frame rate of the corresponding media track.
  • codec_type is the encoding method of the media track
  • is_tile_based is the structural organization field of the track. The value of this field is 1, indicating that the media track is organized in a tile structure; the value of this field is 0, indicating that the media track is not divided into tiles. When this field exists, the value of is_content_alternative must be 1.
  • the service device may obtain the transmission protocol of the volume media, and the transmission protocol includes the replacement relationship indication information and the replacement attribute information between each two playback groups.
  • the transmission protocol is a dynamic adaptive streaming media transmission protocol
  • the replacement relationship indication information and replacement attribute information between each two play groups are located in the track description field of the dynamic adaptive streaming media transmission protocol;
  • the transmission protocol is the dynamic picture experts group media transmission protocol
  • the replacement relationship indication information and the replacement attribute information between every two playing groups are located in the resource group of the dynamic picture experts group media transmission protocol.
  • the indication information that is, the replacement relationship indication information and the replacement attribute information
  • the playback terminal can request the matching playback group according to the transmission protocol, which can avoid the problem of wasting network bandwidth caused by sending all the playback groups to the playback terminal, and can save network bandwidth resources.
  • the transmission protocol is a dynamic adaptive streaming media transmission protocol
  • the fields about replacement relationship indication information and replacement attribute information added in the dynamic adaptive streaming media transmission protocol are shown in Table 2:
  • the dynamic adaptive streaming media transmission protocol includes three extension fields, namely: alternative_Id, quality_ranking, lossless_flag, alternative_Id represents the replacement identifier of the media track pair, and the data type is an unsigned integer.
  • quality_ranking indicates the quality level of the media track, the smaller the value of this field, the higher the quality of the media track, and the data type is unsigned byte type.
  • lossless_flag indicates whether the coding type of the coding method of the media track is lossless coding, and the value of this field is 0, indicating that the coding type of the coding method of the media track is a lossy coding type; indicating that the coding type of the coding method of the media track is Lossless encoding type, the data type is Boolean.
  • the O corresponding to alternative_Id, quality_ranking, and lossless_flag indicates that these fields are optional.
  • the transmission protocol is a dynamic adaptive streaming media transmission protocol
  • the fields about replacement relationship indication information and replacement attribute information added in the dynamic adaptive streaming media transmission protocol are shown in Table 3:
  • alternative_content_flag represents the replacement level field, and the value of this field is 1, indicating that the corresponding media resource group (ie, the playback group) forms an alternative relationship with other media resource groups, and the content between the media resource group and other resource groups
  • the replacement level is the media content level.
  • the value of this field is 0, indicating that the corresponding media resource group does not form a replacement relationship with any media resource group.
  • alternative_asset_flag substitution flag the value of this field is 1, indicating that the media track of the corresponding media resource group constitutes an alternative relationship with the media tracks in other groups.
  • the value of this field is 0, indicating that there is no replaceable media track in the corresponding media resource group.
  • alternative_type Indicates the difference attribute type of an alternative media resource (media track).
  • the track may have one or more difference attributes.
  • quality_ranking indicates the quality level information of the media resource, and the smaller the value of this field, the higher the quality of the corresponding media resource.
  • lossless_flag indicates the encoding type of the encoding method of the media resource. The value of this field is 0, indicating that the encoding type of the corresponding media resource is a lossy encoding type; the value of this field is 1, indicating that the encoding type of the corresponding media resource is a lossless encoding type.
  • Bitrate represents code rate information, indicating the code rate of the corresponding media resource.
  • Framerate represents frame rate information, indicating the frame rate of the corresponding media resource.
  • codec_type indicates the encoding method, indicating the encoding method of the corresponding media resource.
  • is_tile_based indicates the structural organization field of the media resource. The value of this field is 1, indicating that the media resources in the media resource group are organized in a tile structure; the value of this field is 0, indicating that the media resources in the media resource group do not have a tile structure. .
  • the service device may perform encapsulation processing on the at least two playback groups, the replacement relationship indication information and the replacement attribute information to obtain a media resource file, and send the media resource file to the playback terminal.
  • the playback terminal can perform decapsulation processing from the media resource file to obtain the at least two playback groups, the replacement relationship indication information and the replacement attribute information.
  • the playback terminal can select a target playback group that is compatible with the processing capability of the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and decode the target playback group. Get volumetric media.
  • the playback terminal may select a target playback group that matches user needs from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and decode the target playback group to obtain volume media.
  • the service device can obtain the replacement relationship indication information and replacement attribute information between each of the at least two playback groups of the volume media, and send the replacement relationship indication information, replacement attribute information, and at least two playback groups to the playback terminal. Since the replacement relationship indication information is used to indicate the media track pair with the replacement relationship in every two playing groups, the replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair; that is, according to the replacement relationship indication information and replacement attribute information, the playback quality of each playback group can be known.
  • the playback terminal By sending the replacement relationship indication information and replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing capability or user needs, so as to reconstruct and present the volumetric media, and realize personalized presentation. In addition, there is no need to perform decoding processing on all media tracks, which can save processing resources of the playback terminal.
  • FIG. 9 is a schematic flowchart of a volumetric media data processing method provided by an embodiment of the present application. As shown in FIG. 9, the method may be performed by the service device in FIG. 1, where the method may at least include the following S201-S203:
  • step S201 may include the following steps s11-s12.
  • each media track divides the at least two media tracks to obtain at least two play groups; there is a joint play relationship between the media tracks in each play group in the at least two play groups, the The track category differs between media tracks within each playgroup.
  • the service device can obtain at least two media tracks of the volumetric media, and the track category of each media track, and divide the media tracks with different track categories into the same playback group according to the track category of each media track , to get at least two playgroups.
  • the service device can mark the media track pairs that have a replacement relationship in each of the at least two playback groups, and obtain the replacement relationship indication information between each two playback groups; obtain the media track pair The attribute difference between the attribute information of the media tracks in the media track, and the replacement attribute information between each two play groups is obtained.
  • step S202 may include the following steps s21-s26.
  • s21 according to the track category of the media track of the play group P m and the track category of the media track in the play group P n , determine the candidate media track pair; have the same track category between the media tracks in the candidate media track pair, m and n is a positive integer less than Z, and Z is the grouping quantity of the at least two play groups.
  • the difference identification result indicates that the attribute information of the media tracks in the candidate media track pair has a difference, take the candidate media track pair as a media track pair with a replacement relationship, and generate a replacement identifier for the media track pair.
  • the service device may identify candidate media track pairs with the same track type from the playback group P m and playback group P n , and obtain attribute information of the media tracks in the candidate media track pairs.
  • the attribute information difference between the media tracks in the candidate media track pair is identified to obtain a difference identification result, and the difference identification result is used to reflect whether there is a difference in the attribute information between the media tracks in the candidate media track pair.
  • There is no difference in the attribute information between the media tracks in the candidate media track pair indicating that the attribute information of the media tracks in the candidate media track pair is the same, that is, the playback quality of the media tracks in the candidate media track pair is the same, then the candidate media track There is no substitution relationship between media tracks in a pair.
  • the candidate Media tracks in a media track pair have a replacement relationship.
  • the candidate media track pair may be used as a media track pair with a replacement relationship, and a replacement identifier of the media track pair may be generated.
  • the replacement identifier can be used as the replacement relationship indication information between the playback group P m and the playback group P n
  • the attribute information of the media track in the candidate media track pair can be used as the playback group P m and the playback group P m.
  • Replacement attribute information between groups Pn By marking the replacement relationship indication information and replacement attribute information between each two playback groups, it is beneficial for the playback terminal to dynamically and adaptively select the playback group for decoding processing according to its own processing capability or user needs, so as to reconstruct and present the volumetric media. Personalize your presentation.
  • the service device may perform encapsulation processing on the at least two playback groups, the replacement relationship indication information and the replacement attribute information to obtain a media resource file, and send the media resource file to the playback terminal.
  • the playback terminal can perform decapsulation processing from the media resource file to obtain the at least two playback groups, the replacement relationship indication information and the replacement attribute information. Further, the playback terminal can select a target playback group matching the playback terminal from at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and decode the target playback group to obtain volume media. The playback terminal does not need to decode all media tracks, which saves processing resources of the playback terminal.
  • step S203 the service device may send replacement relationship indication information and replacement attribute information to the playback terminal through a data box or a transmission protocol.
  • step S203 may include steps s31-s32.
  • the service device can perform the at least two playgroups, the connection between each two playgroups according to the file format of the volume media (such as ISOBMFF (ISO Base Media File Format, ISO base media file format)). Encapsulate the replacement relationship indication information and the replacement attribute information to obtain a data box of the volume medium, and send the data box including the at least two playback groups, the replacement relationship indication information and the replacement attribute information to the playback terminal. That is to say, the at least two playback groups, the replacement relationship indication information and the replacement attribute information are packaged into the same resource file, and the playback terminal can obtain at least two playback groups, the replacement relationship indication information and the at least two playback groups by downloading the resource file. Replacing the attribute information is beneficial to improving the convenience for the playback terminal to obtain these data.
  • the file format of the volume media such as ISOBMFF (ISO Base Media File Format, ISO base media file format)
  • step S203 may include steps s41-s45.
  • s44 Receive an acquisition request about a target playback group sent by the playback terminal and generated according to the replacement relationship indication information and replacement attribute information in the transmission protocol, where the target playback group is the at least two playback groups that match the playback terminal play group;
  • s45 In response to the acquisition request, send the target playback group to the playback terminal, so as to instruct the playback terminal to decode the target playback group to obtain volumetric media.
  • the service device can obtain the transmission protocol of the volumetric media, the transmission protocol includes a dynamic adaptive streaming media transmission protocol or a dynamic picture experts group media transmission protocol; information and replacement attribute information are added to the transport protocol. Then, the transmission protocol including the replacement relationship indication information and the replacement attribute information is sent to the playback terminal, and the playback terminal can determine the target playback group corresponding to the playback terminal according to the replacement relationship indication information and the replacement attribute information in the transmission protocol. Group ID, requesting the target play group corresponding to the group ID from the service device. That is to say, the at least two playgroups and the replacement relationship indication information are separately encapsulated into a file, so that all the playgroups do not need to be sent to the playback terminal, which can save network bandwidth resources.
  • step s42 may include: when the transmission protocol is a dynamic adaptive streaming media transmission protocol, adding the replacement relationship indication information and replacement attribute information between each two play groups to the dynamic adaptive streaming media transmission in the track description field of the protocol.
  • the transmission protocol is the dynamic picture experts group media transmission protocol
  • the replacement relationship indication information and the replacement attribute information between each two playing groups are added to the resource group of the dynamic picture experts group media transmission protocol.
  • the service device can mark the media track pairs that have a replacement relationship in every two playback groups of the volume media, and obtain the replacement relationship indication information between every two playback groups; and obtain the media track pairs in the media track pair
  • the attribute information of the track is to identify the difference between the attribute information of the media track in the media track pair, and obtain the replacement attribute information between each two playing groups.
  • FIG. 10 is a schematic flowchart of a volumetric media data processing method provided by an embodiment of the present application. As shown in FIG. 10, the method may be performed by the playback terminal in FIG. 1, wherein the method may at least include the following S301-S303:
  • the replacement relationship indication information is used to indicate the media that has a replacement relationship in each of the two playback groups
  • the replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with a replacement relationship.
  • the playback terminal may acquire the replacement relationship indication information and the replacement attribute information between every two playback groups in the at least two playback groups of the volumetric media from the service device.
  • the replacement relationship indication information and the replacement attribute information identify a target playback group that matches the playback terminal from the at least two playback groups.
  • the playback terminal can determine the processing resources required to process each playback packet according to the replacement relationship indication information and the replacement attribute information; then, the playback terminal can determine the processing resources required to process each playback packet , selecting a target playback group that matches the processing capability of the playback terminal from at least two playback groups.
  • the playback quality of each playback group can be determined according to the replacement relationship indication information and the replacement attribute information, and the playback quality that matches the user's playback quality requirements can be selected from at least two playback groups according to the playback quality of each playback group. target playgroup.
  • the playback terminal can decode the target playback group to obtain the volumetric media and play the volumetric media, that is, the playback terminal does not need to decode all the playback groups, which can save the processing resources of the playback terminal.
  • step S302 may include: receiving the transmission protocol of the volume media sent by the service device, and analyzing the transmission protocol , to obtain the replacement relationship indication information and the replacement attribute information.
  • the above-mentioned transmission protocol is analyzed to obtain the replacement relationship indication information and the replacement attribute information, including: when the transmission protocol is a dynamic adaptive streaming media transmission protocol, the track of the dynamic adaptive streaming media transmission protocol Read the replacement relationship indication information and the replacement attribute information in the description field; when the transmission protocol is the dynamic picture experts group media transmission protocol, read the substitution relationship indication information from the resource group of the dynamic picture experts group media transmission protocol and the replacement property information.
  • the playback terminal may receive the transmission protocol of the volumetric media sent by the service device, and the transmission protocol may refer to a dynamic adaptive streaming media transmission protocol or a dynamic picture experts group media transmission protocol.
  • the player terminal can obtain the replacement relationship indication information and replacement attribute information from the track description field of the dynamic adaptive streaming media transmission protocol.
  • the transmission protocol is the dynamic picture experts group media transmission protocol
  • the playback terminal can read the replacement relationship indication information and the replacement attribute information from the resource group whose transmission protocol is the dynamic picture experts group media transmission protocol.
  • step S303 may include steps s51-s54.
  • the replacement relationship indication information and the replacement attribute information determine a group identifier of a candidate playback group that matches the playback terminal from the at least two playback groups.
  • the candidate playing group is used as a target playing group matching the playing terminal.
  • the playback terminal can determine a candidate playback group that matches the processing capability of the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information or, the playback terminal may determine, from the at least two playback groups, the group ID of a candidate playback group that matches the user requirements corresponding to the playback terminal according to the replacement relationship indication information and the replacement attribute information. Then, an acquisition request carrying the group identifier of the candidate playback group can be generated, and the acquisition request is sent to the service device, and the service equipment receives the candidate playback group returned in response to the acquisition request, and uses the candidate playback group as the playback terminal that matches the playback terminal. target playgroup.
  • the playback terminal By pre-obtaining the transmission protocol carrying the replacement relationship indication information and replacement attribute information from the service device, it is beneficial for the playback terminal to determine the target playback group that matches the playback terminal according to the replacement relationship indication information and replacement attribute information. Obtaining the target playgroup in , does not need to download all the playgroups, which can save network bandwidth resources.
  • step s51 may include steps s61-s63.
  • the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information determine the processing difficulty level for processing each of the above-mentioned at least two playgroups; the above-mentioned processing difficulty level is determined according to the amount of processing resources consumed by processing the above-mentioned each playgroup .
  • the playback terminal can obtain the attribute information of the playback terminal, which includes storage space size, data processing speed, etc., and determine the volume media processing capacity of the playback terminal according to the attribute information of the playback terminal. Handling performance levels (such as decoding ability levels). Further, the amount of processing resources required to process each playback group may be determined according to the replacement relationship indication information and the replacement attribute information, and the processing resources for processing each playback group in the at least two playback groups may be generated according to the amount of processing resources required to process each playback group. difficulty level of processing.
  • the processing difficulty level of each playback group is proportional to the processing resources required to process each playback group. It becomes higher as the amount increases, and becomes lower as the amount of processing resources consumed to process each playback group decreases.
  • the playback group whose processing difficulty level matches the processing performance level of the playback terminal among the at least two playback groups may be used as a candidate playback group, and the group identifier of the candidate playback group may be acquired. According to the processing difficulty level of the playback group and the processing performance level of the playback terminal, the target playback group can be selected to achieve personalized presentation.
  • a target playback group with a high processing difficulty level can be obtained from the service device.
  • the playback quality of the target playback group is higher, and the playback effect of the volumetric media can be improved.
  • the processing performance level of the playback terminal is relatively low, the target playback group with low processing difficulty level can be acquired from the service device, which is beneficial to ensure the fluency of playing volumetric media.
  • step S301 may include: obtaining the data box of the volume medium from the service equipment, and the data of the volume medium The box performs decapsulation processing to obtain the at least two play groups, the replacement relationship indication information and the replacement attribute information between every two play groups in the at least two play groups.
  • step s51 may include: acquiring the first quality level of the volume media played by the playback terminal in history; Two quality levels: the playing group whose second quality level matches the first quality level among the at least two playing groups is used as a candidate playing group, and the group identifier of the candidate playing group is obtained.
  • the playback terminal can select a matching target playback group according to the historical playback behavior data.
  • the first quality level of the volume media played by the playback terminal history (such as the past week or the past month) can be obtained, and according to the replacement relationship indication information and
  • the replacement attribute information determines the second quality level of each playback group in the at least two playback groups; for example, the second quality level of each playback group is determined according to the encoding type of the encoding method of the media track pair, that is, the encoding type is a lossless encoding type
  • the quality level of the playback group corresponding to the media track of is higher than the quality level of the playback group corresponding to the media track whose encoding type is the lossy encoding type.
  • a playback group whose second quality level is the same as the first quality level among the at least two playback groups is used as a candidate playback group, and a group identifier of the candidate playback group is acquired.
  • the playback terminal may determine the second quality level of each playback group in the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and display the second quality level of each playback group.
  • the selected playing group is used as a candidate playing group, and the group identifier of the candidate playing group is obtained.
  • the playback terminal can obtain the replacement relationship attribute information, the replacement attribute information, and at least two playback groups through the data box.
  • the playback terminal can obtain the data box of the volume media from the service device, and perform decapsulation processing on the data box of the volume media, so as to obtain the at least two playback groups and each two playback groups in the at least two playback groups.
  • Substitution relationship indication information between groups and substitution attribute information By decapsulating the data box of the volume medium, the at least two playing groups, the replacement relationship indication information and the replacement attribute information can be obtained, which can improve the convenience of obtaining these information.
  • the above step S303 includes: performing decoding processing on the target play group to obtain the media content of the volumetric media; synchronizing the time of the media content to obtain the volumetric media.
  • the media content of the volumetric media includes at least one of audio content, video content, and text content.
  • the playback terminal can determine the encoding method of each media track in the target playback group from the description information of the target playback group, and determine the decoding method of each media track according to the encoding method of each media track in the target playback group.
  • the decoding method of each media track is used to decode each media track to obtain the media content of the volumetric media; time synchronization is performed on the audio content, the video content, and the text content to obtain the volumetric media.
  • the playback terminal only needs to decode one of the at least two playback groups to obtain the volumetric media, and does not need to decode all the playback groups, which can save the decoding resources of the playback terminal.
  • the playback terminal dynamically and adaptively selects the matching target playback group from at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and only needs to decode the target playback group to obtain the volume For media, there is no need to perform decoding operations on all playback groups, which can save processing resources of the playback terminal.
  • FIG. 13 is a schematic structural diagram of a volumetric media data processing device 1 provided by an embodiment of the present application.
  • the above-mentioned data processing device 1 for the volume medium may be a computer-readable instruction (including program code) running in a computer device, for example, the data processing device 1 for the volume medium is an application software; The corresponding steps in the method provided by the example.
  • the volumetric media data processing device 1 may include: an acquiring module 131 and a sending module 132 .
  • the obtaining module is used to obtain the replacement relationship indication information and replacement attribute information between every two play groups in the at least two play groups of the volume media;
  • a media track pair with a relationship the above-mentioned replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with a replacement relationship;
  • a sending module configured to send the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information to the playback terminal, so as to instruct the playback terminal to identify the target matching the playback terminal from at least two playback groups according to the replacement relationship indication information and the replacement attribute information
  • the playback group decodes the target playback group to obtain the volume media.
  • the acquisition module acquires replacement relationship indication information and replacement attribute information between every two playback groups in at least two playback groups of the volumetric media, including:
  • the above-mentioned data box of the above-mentioned volume medium includes the indication information of the replacement relationship between each two playing groups and the replacement attribute information.
  • the acquisition module acquires the replacement relationship indication information and replacement attribute information between each of the at least two playback groups of the volume media, including:
  • the above-mentioned transmission protocol includes the replacement relationship indication information and the replacement attribute information between each two playing groups.
  • the replacement relationship indication information and replacement attribute information between the above-mentioned every two playing groups are located in the track description field of the above-mentioned dynamic adaptive streaming media transmission protocol;
  • the replacement relationship indication information and the replacement attribute information between each two playback groups are located in the resource group of the dynamic picture experts group media transmission protocol.
  • the above-mentioned media track pair includes a first media track belonging to the playing group P m and a second media track belonging to the playing group P n
  • the replacement attribute information between the above-mentioned playing group P m and the above-mentioned playing group P n includes the above-mentioned
  • the attribute information of the first media track, and the attribute information of the above-mentioned second media track, m and n are all positive integers less than Z, and Z is the grouping quantity of the above-mentioned at least two playing groups;
  • the attribute information of the first media track includes the encoding method of the first media track
  • the attribute information of the second media track includes the encoding method of the second media track
  • the encoding method of the first media track is the same as the encoding method of the second media track.
  • the attribute information of the first media track further includes at least one of the transmission rate, structural organization, quality level, and encoding type corresponding to the encoding method of the first media track;
  • the attribute information of the second media track further includes the transmission rate, structural organization, quality level and encoding type corresponding to the encoding mode of the second media track.
  • the above-mentioned replacement relationship indication information also includes a replacement level field; when the replacement level field in the replacement relationship indication information between the above-mentioned playing group P m and the above-mentioned playing group P n is the first value, the above-mentioned replacement level field is used
  • the replacement level reflected between the playing group P m and the playing group P n is the media content level.
  • the above-mentioned replacement relationship indication information also includes a replacement level field; when the replacement level field in the replacement relationship indication information of the above-mentioned playing group P m and the above-mentioned playing group Pn is the second value, the above-mentioned replacement level field is used to reflect The replacement level between the playback group P m and the playback group P n is the track level.
  • the replacement level between the above-mentioned playing group P m and the above-mentioned playing group P n is the media content level
  • the number of media track pairs with a replacement relationship between the above-mentioned playing group P m and the above-mentioned playing group P n is R
  • R is the number of media tracks in the playgroup Pm
  • the number of media tracks in the playgroup Pm is the same as the number of tracks in the playgroup Pn .
  • the replacement level between the above-mentioned playing group P m and the above-mentioned playing group P n is a track level
  • the number of media track pairs having a replacement relationship in the above-mentioned playing group P m and the above-mentioned playing group P n is less than R positive integer of .
  • the encoding methods of the media tracks in the above-mentioned playing group P m are all the first encoding methods; the above-mentioned playing group P
  • the coding modes of the media tracks in n are all the second coding mode.
  • the track categories of the media tracks in the media track pairs having a replacement relationship in each two playing groups are the same.
  • the above replacement relationship indication information includes replacement identifiers of the media track pairs that have a replacement relationship in each of the two playing groups.
  • step S101 shown in FIG. 6 may be executed by the acquiring module 131 in FIG. 13
  • step S102 shown in FIG. 6 may be executed by the sending module 132 in FIG. 13 .
  • each module in the data processing device for volumetric media shown in FIG. Dividing into multiple functionally smaller subunits can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the above-mentioned modules are divided based on logical functions.
  • the functions of one module can also be realized by multiple units, or the functions of multiple modules can be realized by one unit.
  • the data processing apparatus for volumetric media may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented cooperatively by multiple units.
  • a general-purpose computer device such as a computer including processing elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM) and storage elements.
  • Computer-readable instructions capable of executing the steps involved in the corresponding method as shown in FIG. 6 to construct a volume medium data processing device as shown in FIG. 13 and to realize the embodiment of the present application Data processing methods for volumetric media.
  • the above-mentioned computer-readable instructions may be recorded in, for example, a computer-readable recording medium, loaded into the above-mentioned computing device via the computer-readable recording medium, and run there.
  • the service device can mark the media track pairs that have a replacement relationship in every two playback groups of the volume media, and obtain the replacement relationship indication information between every two playback groups; and obtain the media track pairs in the media track pair
  • the attribute information of the track is to identify the difference between the attribute information of the media track in the media track pair, and obtain the replacement attribute information between each two playing groups.
  • FIG. 14 is a schematic structural diagram of a volumetric media data processing device 2 provided by an embodiment of the present application.
  • the data processing device 2 of the above-mentioned volume medium may be a computer-readable instruction (including program code) running in the computer equipment, for example, the data processing device 2 of the volume medium is an application software; this device can be used to execute the implementation of the present application The corresponding steps in the method provided by the example.
  • the volume media data processing device 2 may include: an acquisition module 141 , a marking module 142 and a sending module 143 .
  • An acquisition module configured to acquire at least two playback groups of volume media
  • a marking module configured to mark each of the at least two play groups to obtain the replacement relationship indication information and replacement attribute information between the above-mentioned two play groups;
  • the replacement relationship indication information is used to indicate the above-mentioned There are media track pairs with a replacement relationship in every two play groups, and the above-mentioned replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pairs with a replacement relationship;
  • a sending module configured to send the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information to the playback terminal, so as to instruct the playback terminal to identify the target matching the playback terminal from at least two playback groups according to the replacement relationship indication information and the replacement attribute information
  • the playback group decodes the target playback group to obtain the volume media.
  • the acquisition module acquires at least two playback groups of volume media, including:
  • the above-mentioned at least two media tracks are divided to obtain at least two play groups; there is a joint play relationship between the media tracks in each play group in the above-mentioned at least two play groups, and the above-mentioned each play
  • the track category differs between media tracks within a group.
  • the above marking module marks each of the at least two play groups to obtain the replacement relationship indication information and replacement attribute information between the above two play groups, including:
  • the track class of the media track of the play group P m and the track class of the media track in the play group P n determine the candidate media track pair; There is the same track class between the media tracks in the above-mentioned candidate media track pair, m and n are equal Be a positive integer less than Z, and Z is the grouping quantity of the above-mentioned at least two playing groups;
  • the above-mentioned difference identification result indicates that the attribute information between the media tracks in the above-mentioned candidate media track pair has a difference, then the above-mentioned candidate media track pair is used as a media track pair with a replacement relationship, and the replacement identification of the above-mentioned media track pair is generated;
  • the attribute information of the media track in the candidate media track pair is used as the replacement attribute information between the playing group P m and the playing group P n .
  • the above-mentioned sending module sends the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information to the playback terminal, so as to instruct the playback terminal to identify from at least two playback groups that match the playback terminal according to the replacement relationship indication information and the replacement attribute information.
  • the target playback group decode the target playback group to obtain the volume media, including:
  • the above-mentioned sending module sends the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information to the playback terminal, so as to instruct the playback terminal to identify from at least two playback groups that match the playback terminal according to the replacement relationship indication information and the replacement attribute information.
  • the target playback group decode the target playback group to obtain the volume media, including:
  • the above-mentioned target playback group is sent to the above-mentioned playback terminal, so as to instruct the playback terminal to decode the target playback group to obtain volume media.
  • the above-mentioned sending module adds the above-mentioned replacement relationship indication information and replacement attribute information between each two play groups to the above-mentioned transmission protocol, including:
  • the above-mentioned transmission protocol is a dynamic adaptive streaming media transmission protocol
  • the above-mentioned replacement relationship indication information and replacement attribute information between each two playing groups are added to the track description field of the above-mentioned dynamic adaptive streaming media transmission protocol.
  • the above-mentioned sending module adds the above-mentioned replacement relationship indication information and replacement attribute information between each two play groups to the above-mentioned transmission protocol, including:
  • the above-mentioned transmission protocol is the dynamic picture experts group media transmission protocol
  • step S201 shown in FIG. 9 can be performed by the acquiring module 141 in FIG. 14
  • step S202 shown in FIG. 9 can be performed by the marking module 142 in FIG. 14
  • the above-mentioned modules are divided based on logical functions. In practical applications, the functions of one module can also be realized by multiple units, or the functions of multiple modules can be realized by one unit.
  • the data processing apparatus for volumetric media may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented cooperatively by multiple units.
  • a general-purpose computer device such as a computer including processing elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM) and storage elements.
  • Computer-readable instructions capable of executing the steps involved in the corresponding method as shown in FIG. 9 to construct a volume medium data processing device as shown in FIG. 14 and to realize the embodiment of the present application Data processing methods for volumetric media.
  • the above-mentioned computer-readable instructions may be recorded in, for example, a computer-readable recording medium, loaded into the above-mentioned computing device via the computer-readable recording medium, and run there.
  • the service device can mark the media track pairs that have a replacement relationship in every two playback groups of the volume media, and obtain the replacement relationship indication information between every two playback groups; and obtain the media track pairs in the media track pair
  • the attribute information of the track is to identify the difference between the attribute information of the media track in the media track pair, and obtain the replacement attribute information between each two playing groups.
  • FIG. 15 is a schematic structural diagram of a volumetric media data processing device 3 provided by an embodiment of the present application.
  • the data processing device 3 of the above-mentioned volume medium can be a computer-readable instruction (including program code) running in the computer equipment, for example, the data processing device 3 of the volume medium is an application software; this device can be used to execute the implementation of the present application The corresponding steps in the method provided by the example.
  • the volume media data processing device 3 may include: an acquisition module 151 , an identification module 152 and a decoding module 153 .
  • the obtaining module is used to obtain the replacement relationship indication information and replacement attribute information between every two play groups in the at least two play groups of the volume media;
  • a media track pair with a relationship the above-mentioned replacement attribute information is used to reflect the attribute difference between the media tracks in the media track pair with a replacement relationship;
  • An identification module configured to identify, from the at least two playback groups, a target playback group that matches the playback terminal according to the replacement relationship indication information and the replacement attribute information;
  • the decoding module is configured to perform decoding processing on the above-mentioned target play group to obtain the above-mentioned volume media.
  • the acquisition module acquires replacement relationship indication information and replacement attribute information between every two playback groups in at least two playback groups of the volumetric media, including:
  • the above-mentioned transmission protocol is analyzed to obtain the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information.
  • the above identification module identifies a target play group that matches the play terminal from the above at least two play groups according to the above replacement relationship indication information and the above replacement attribute information; including:
  • the above-mentioned candidate playing group is used as a target playing group matching the above-mentioned playing terminal.
  • the identification module determines the group identifier of the candidate playback group matching the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, including:
  • the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information determine the processing difficulty level of each play group in the above-mentioned at least two play groups; the above-mentioned processing difficulty level is determined according to the amount of processing resources consumed by processing the above-mentioned each play group;
  • the identification module determines the group identifier of the candidate playback group matching the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, including:
  • the above acquisition module parses the above transmission protocol to obtain the above replacement relationship indication information and the above replacement attribute information, including:
  • the above-mentioned transmission protocol is a dynamic adaptive streaming media transmission protocol
  • the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information are read in the track description field of the above-mentioned dynamic adaptive streaming media transmission protocol.
  • the above acquisition module parses the above transmission protocol to obtain the above replacement relationship indication information and the above replacement attribute information, including:
  • the above-mentioned transmission protocol is the dynamic picture experts group media transmission protocol
  • the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information are read from the resource group of the above-mentioned dynamic picture experts group media transmission protocol.
  • the acquisition module acquires replacement relationship indication information and replacement attribute information between every two playback groups in at least two playback groups of the volumetric media, including:
  • the above-mentioned decoding module decodes the above-mentioned target play group to obtain the above-mentioned volumetric media, and plays the above-mentioned volumetric media, including:
  • the media content is time-synchronized to obtain the above-mentioned volumetric media, and the above-mentioned volumetric media is played.
  • the media content of the volumetric media includes at least one of audio content, video content, and text content.
  • the steps involved in the data processing method for volume media shown in FIG. 10 may be executed by various modules in the data processing device for volume media shown in FIG. 15 .
  • step S301 shown in FIG. 10 can be performed by the acquisition module 151 in FIG. 15
  • step S302 shown in FIG. 10 can be performed by the identification module 152 in FIG. 15
  • each module of the data processing device 3 for volumetric media shown in FIG. Dividing into multiple functionally smaller subunits can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the above-mentioned modules are divided based on logical functions. In practical applications, the functions of one module can also be realized by multiple units, or the functions of multiple modules can be realized by one unit.
  • the data processing apparatus for volumetric media may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented cooperatively by multiple units.
  • a general-purpose computer device such as a computer including processing elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM) and storage elements.
  • Computer-readable instructions capable of executing the steps involved in the corresponding method as shown in FIG. 10 to construct a volume medium data processing device as shown in FIG. 15 and realize the embodiment of the present application Data processing methods for volumetric media.
  • the above-mentioned computer-readable instructions may be recorded in, for example, a computer-readable recording medium, loaded into the above-mentioned computing device via the computer-readable recording medium, and run there.
  • the playback terminal dynamically and adaptively selects the matching target playback group from at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and only needs to decode the target playback group to obtain the volume For media, there is no need to perform decoding operations on all playback groups, which can save processing resources of the playback terminal.
  • the computer device 1000 may include: a processor 1001 , a network interface 1004 and a memory 1005 .
  • the computer device 1000 may further include: a user interface 1003 and at least one communication bus 1002 .
  • the communication bus 1002 is used to realize connection and communication between these components.
  • the user interface 1003 may include a display screen (Display) and a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface and a wireless interface (such as a WI-FI interface).
  • the memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory.
  • the memory 1005 may also be at least one storage device located away from the aforementioned processor 1001 .
  • the memory 1005 as one or more computer-readable storage media may include an operating system, a network communication module, a user interface module, and a device control application program.
  • the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 program to implement the steps of the method in each embodiment of the present application.
  • the computer device 1000 described in the embodiment of the present application can execute the description of the data processing method for the above-mentioned volume media in the embodiment corresponding to the above-mentioned Figure 6, Figure 9, and Figure 10 above, and can also execute the above-mentioned Figure 13-figure 15 is a description of the data processing device for the above-mentioned volume media in the embodiment corresponding to 15. .
  • the embodiment of the present application also provides one or more computer-readable storage media, and the computer-readable storage medium executed by the data processing device that stores the above-mentioned volume medium in the above-mentioned computer-readable storage medium can Read instructions, and the above-mentioned computer-readable instructions include program instructions.
  • the above-mentioned processor executes the above-mentioned program instructions, it can execute the description of the data processing method of the above-mentioned volume medium in the embodiment corresponding to the previous FIG. 6 , FIG. 9 and the previous FIG. 10 , Therefore, details will not be repeated here. .
  • the technical details not disclosed in the embodiments of the computer-readable storage medium involved in the present application please refer to the description of the method embodiments of the present application.
  • the above-mentioned program instructions may be deployed to execute on one computer device, or deployed to be executed on multiple computer devices located at one location, or distributed across multiple locations and interconnected by a communication network Executed on the Internet, multiple computer devices distributed in multiple locations and interconnected through a communication network can form a blockchain network.
  • the embodiment of the present application also provides a computer program product, including computer readable instructions, and when the computer readable instructions are executed by one or more processors, the steps of the methods in the various embodiments of the present application are realized.
  • the above-mentioned storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)

Abstract

本申请实施例公开了一种容积媒体的数据处理方法，方法包括：获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；所述替换关系指示信息用于指示所述每两个播放组中具有替换关系的媒体轨道对，所述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异(S101)；将所述替换关系指示信息和所述替换属性信息发送至播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体(S102)。

Description

容积媒体的数据处理方法、装置、设备以及存储介质
本申请要求于2021年07月12日提交中国专利局，申请号为2021107858282、发明名称为“容积媒体的数据处理方法、装置、设备以及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，具体涉及点云技术领域，尤其涉及一种容积媒体的数据处理方法、装置、设备以及存储介质。
背景技术
随着三维场景捕捉和渲染技术的发展，促进了以高自由度、真三维为特征的三维容积媒体的创新。例如，点云技术作为容积媒体内容和服务的典型代表已得到广泛应用，其中，点云是空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集。点云中的每个点至少具有三维位置信息，根据应用场景的不同，还可能具有色彩、材质或其他信息。此外，作为容积媒体的另一项应用，多视角视频以及自由视点使用真实或者虚拟相机捕获3D场景信息，支持在有限的观看位置和方向范围内以六自由度(6DoF)呈现三维3D场景。
容积媒体通常包括多个媒体轨道，如视频轨道、音频轨道以及文字轨道等等；实践中发现，当容积媒体的媒体轨道之间存在某种关联关系时，播放终端只需要对容积媒体的部分媒体轨道进行解码操作，即可重建和呈现容积媒体。例如，容积媒体包括两个具有替换关系的视频轨道时，只需要播放终端对其中一个视频轨道进行解码操作。但是，由于播放终端无法识别这些媒体轨道之间的关联关系，播放终端需要对容积媒体的所有媒体轨道进行解码操作，导致播放终端的处理资源浪费。
发明内容
根据本申请提供的各种实施例，提供一种容积媒体的数据处理方法、装置、设备以及存储介质。
本申请实施例一方面提供一种容积媒体的数据处理方法，由服务设备执行，包括：
获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
本申请实施例一方面提供一种容积媒体的数据处理方法，由服务设备执行，包括：
获取容积媒体的至少两个播放组；
对上述至少两个播放组中的每两个播放组进行标记，得到上述每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
本申请实施例一方面提供一种容积媒体的数据处理方法，由播放终端执行，包括：
获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
根据上述替换关系指示信息和上述替换属性信息，从上述至少两个播放组中识别出与播放终端匹配的目标播放组；
对上述目标播放组进行解码处理，得到上述容积媒体。
本申请实施例一方面提供一种容积媒体的数据处理装置，包括：
获取模块，用于获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
发送模块，用于将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
本申请实施例一方面提供一种容积媒体的数据处理装置，包括：
获取模块，用于获取容积媒体的至少两个播放组；
标记模块，用于对上述至少两个播放组中的每两个播放组进行标记，得到上述每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
发送模块，用于将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
本申请实施例一方面提供一种容积媒体的数据处理装置，包括：
获取模块，用于获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
识别模块，用于根据上述替换关系指示信息和上述替换属性信息，从上述至少两个播放组中识别出与播放终端匹配的目标播放组；
解码模块，用于对上述目标播放组进行解码处理，得到上述容积媒体。
本申请一方面提供了一种计算机设备，包括：一个或多个处理器及存储器；
其中，上述存储器用于存储计算机可读指令，上述处理器用于调用上述计算机可读指令，以执行实现本申请各方法实施例中的步骤。
本申请一方面提供了一种计算机程序产品，包括计算机可读指令，所述计算机可读指令被一个或多个处理器执行时实现本申请各方法实施例中的步骤。
本申请实施例一方面提供了一个或多个计算机可读存储介质，上述计算机可读存储介质存储有计算机可读指令，上述计算机可读指令包括程序指令，上述程序指令当被处理器执行时，以执行实现本申请各方法实施例中的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请提供的一种容积媒体系统的架构图；
图2示出了本申请提供的一种6DoF的示意图；
图3示出了本申请实施例提供的一种3DoF的示意图；
图4示出了本申请提供的一种3DoF+的示意图；
图5示出了本申请提供的一种容积媒体的媒体轨道的示意图；
图6是本申请提供的一种容积媒体的数据处理方法的流程意图；
图7示出了本申请提供的一种容积媒体的媒体轨道的示意图；
图8示出了本申请提供的一种容积媒体的媒体轨道的示意图；
图9是本申请提供的一种容积媒体的数据处理方法的流程意图；
图10是本申请提供的一种容积媒体的数据处理方法的流程意图；
图11是本申请提供的一种获取替换关系指示信息以及替换属性信息的场景示意图；
图12是本申请提供的一种获取替换关系指示信息以及替换属性信息的场景示意图；
图13是本申请实施例提供的一种容积媒体的数据处理装置的结构示意图；
图14是本申请实施例提供的一种容积媒体的数据处理装置的结构示意图；
图15是本申请实施例提供的一种容积媒体的数据处理装置的结构示意图；
图16是本申请实施例提供的一种计算机设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例涉及容积媒体的数据处理技术。所谓容积媒体(visual volumetricvideo-based coding media)是指捕获自三维空间视觉内容并提供3DoF+、6DoF观看体验的，以传统视频编码的，在文件封装中包含容积视频类型轨道的沉浸式媒体，包括多视角视频、视频编码点云等。
图1示出了本申请一个示例性实施例提供的一种容积媒体系统的架构图；如图1所示，容积媒体系统包括服务设备和播放终端，服务设备可以是指容积媒体的提供者所使用的计算机设备，该计算机设备可以是终端(如PC(Personal Computer，个人计算机)、智能移动设备(如智能手机)等)或服务器。其中，服务器可以是独立的一个物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。播放终端可以是指容积媒体的使用者所使用的计算机设备，该计算机设备可以是终端(如PC(Personal Computer，个人计算机)、智能移动设备(如智能手机)、VR设备(如VR头盔、VR眼镜等))。容积媒体的数据处理过程包括在服务设备侧的数据处理过程及在播放终端侧的数据处理过程。
在服务设备端的数据处理过程主要包括：(1)容积媒体的媒体内容的获取与制作过程；(2)容积媒体的编码及文件封装的过程。在播放终端端的数据处理过程主要包括：(3)容积媒体的文件解封装及解码的过程；(4)容积媒体的渲染过程。另外，服务设备与播放终端之间涉及容积媒体的传输过程，该传输过程可以基于各种传输协议来进行，此处的传输协议可包括但不限于：DASH(Dynamic Adaptive Streaming over HTTP，动态自适应流媒体传输)协议、MMT(Moving Picture Experts GroupMedia Transport，动态图像专家组媒体传输协议)、HLS(HTTP Live Streaming，动态码率自适应传输)协议、SMTP(Smart Media TransportProtocaol，智能媒体传输协议)、TCP(Transmission Control Protocol，传输控制协议)等。
下面将结合图1，分别对容积媒体的数据处理过程中涉及的各个过程进行详细介绍。
一、在服务设备端的数据处理过程：
(1)容积媒体的媒体内容的获取与制作过程。
1)容积媒体的媒体内容的获取过程。
容积媒体的媒体内容是通过捕获设备采集现实世界的声音-视觉场景获得的。在一种实现中，捕获设备可以是指设于服务设备中的硬件组件，例如捕获设备是指终端的麦克风、摄像头、传感器等。另一种实现中，该捕获设备也可以是与服务设备相连接的硬件装置，例如与服务设备相连接摄像头；用于为服务设备提供容积媒体的媒体内容的获取服务。该捕获设备可以包括但不限于：音频设备、摄像设备及传感设备。其中，音频设备可以包括音频传感器、麦克风等。摄像设备可以包括普通摄像头、立体摄像头、光场摄像头等。传感设备可以包括激光设备、雷达设备等。捕获设备的数量可以为多个，如摄像机阵列，这些捕获设备被部署在现实空间中的一些特定位置以同时捕获该空间内不同角度的音视频内容，该音视频内容中的音频内容和视频内容在时间和空间上均保持同步。
2)容积媒体的媒体内容的制作过程。
捕获到的音频内容本身就是适合被执行容积媒体的音频编码的内容。捕获到的视频内容进行一系列制作流程后才可成为适合被执行容积媒体的视频编码的内容，该制作流程包括：
①拼接。由于捕获到的视频内容是捕获设备在不同角度下拍摄得到的，拼接就是指对这些各个角度拍摄的视频内容拼接成一个完整的、能够反映现实空间360度视觉全景的视频，即拼接后的视频是一个在三维空间表示的全景视频(或球面视频)。
②投影。投影就是指将拼接形成的一个三维视频映射到一个二维(3-Dimension，2D)图像上的过程，投影形成的2D图像称为投影图像；投影的方式可包括但不限于：经纬图投影、正六面体投影。
③区域封装。投影图像可以被直接进行编码，也可以对投影图像进行区域封装之后再进行编码。实践中发现，在容积媒体的数据处理过程中，对于二维投影图像进行区域封装之后再进行编码能够大幅提升容积媒体的视频编码效率，因此区域封装技术被广泛应用到容积媒体的视频处理过程中。所谓区域封装是指将投影图像按区域执行变换处理的过程，区域封装过程使投影图像被转换为封装图像。区域封装的过程具体包括：将投影图像划分为多个映射区域，然后再对多个映射区域分别进行变换处理得到多个封装区域，将多个封装区域映射到一个2D图像中得到封装图像。其中，映射区域是指执行区域封装前在投影图像中经划分得到的区域；封装区域是指执行区域封装后位于封装图像中的区域。变换处理可以包括但不限于：镜像、旋转、重新排列、上采样、下采样、改变区域的分辨率及移动等处理。
需要说明的是，由于采用捕获设备只能捕获到全景视频，这样的视频经服务设备处理并传输至播放终端进行相应的数据处理后，播放终端侧的用户只能通过执行一些特定动作(如头部旋转)来观看360度的视频信息，而执行非特定动作(如移动头部)并不能获得相应的视频变化，VR体验不佳，因此需要额外提供与全景视频相匹配的深度信息，来使用户获得更优的沉浸度和更佳的VR体验，这就涉及6DoF(Six Degrees of Freedom，六自由度)制作技术。当用户可以在模拟的场景中较自由的移动时，称为6DoF。采用6DoF制作技术进行容积媒体的视频内容的制作时，捕获设备一般会选用光场摄像头、激光设备、雷达设备等，捕获空间中的点云数据或光场数据，并且在执行上述制作流程①-③的过程中还需要进行一些特定处理，例如对点云数据的切割、映射等过程，深度信息的计算过程等。图2示出了本申请一个示例性实施例提供的6DoF的示意图；6DoF分为窗口6DoF、全方向6DoF和6DoF，其中，窗口6DoF是指用户在X轴、Y轴的旋转移动受限，以及在Z轴的平移受限；例如，用户不能够看到窗户框架外的景象，以及用户无法穿过窗户。全方向6DoF是指用户在X轴、Y轴和Z轴的旋转移动受限，例如，用户在受限的移动区域中不能自由的穿过三维的360度VR内容。6DoF是指用户可以沿着X轴、Y轴、Z轴自由平移，例如，用户可以在三维的360度VR内容中自由的走动。与6DoF相类似的，还有3DoF和3DoF+制作技术。图3示出了本申请一个示例性实施例提供的3DoF的示意图；如图3所示，3DoF是指用户在一个三维空间的中心点固定，用户头部沿着X轴、Y轴和Z轴旋转来观看媒体内容提供的画面。图4示出了本申请一个示例性实施例提供的3DoF+的示意图，如图4所示，3DoF+是指当容积媒体提供的虚拟场景具有一定的深度信息，用户头部可以基于3DoF在一个有限的空间内移动来观看媒体内容提供的画面。
(2)容积媒体的编码及文件封装的过程。
捕获到的音频内容可直接进行音频编码形成容积媒体的音频轨道。经过上述制作流程①-②或①-③之后，对视频内容进行视频编码，得到容积媒体的视频轨道。例如，如投影图形进行编码得到携带2D与3D映射关系的容积视频轨道；对封装图像进行编码得到携带封装数据的容积视频轨道。从三维空间表示的全景视频中获取携带属性数据的视频流，以及携带位置数据的视频流；对携带属性数据的视频流进行编码，得到携带属性数据的容积视频轨道；属性数据是指拍摄对象的颜色信息、材质等属性。对携带位置数据的视频流进行编码，得到携带位置数据的容积视频轨道；位置数据是指拍摄对象的三维位置信息。
此处需要说明的是，上述视频编码方式可包括但不限于：视频编码包括AVC(Advanced Video Coding，高级视频编码)、HVC(High Video Coding，高清视频编码)、AVS(Audio Video Coding Standard，中国国家视频编码标准)、HEVC/H.265(High Efficiency Video Coding，国际视频编码标准)、VVC/H.266 (Versatile Video Coding，国际视频编码标准)。为了适应不同的应用场景，服务设备可以采用不同的视频编码方式对视频内容进行编码，这样可以得到具有替换关系的媒体轨道对，具有替换关系的媒体轨道对是对同一视频内容采用不同的视频编码方式得到的，即可以称为具有替换关系的媒体轨道对中的容积视频轨道的轨道类别相同。通常采用轨道标识来区分不同容积视频轨道，采用替换标识来标识具有替换关系的容积视频轨道。
例如，如图5所示，图5为容积媒体V3C中的容积视频轨道之间的替换关系示意图，该容积媒体包括8个容积视频轨道，各个容积视频轨道的轨道标识分别为1-8，容积视频轨道1为携带2D与3D映射关系的容积视频轨道。容积视频轨道2、容积视频轨道5的轨道类别均为携带属性数据的容积视频轨道，容积视频轨道2、容积视频轨道5的编码方式分别为avc1(即高级视频编码)、hvc1(即高清视频编码)。因此，容积视频轨道2、容积视频轨道5之间具有替换关系，容积视频轨道2、容积视频轨道5的替换标识为10，即容积视频轨道2、容积视频轨道5构成媒体轨道对10。容积视频轨道3、容积视频轨道6的轨道类别均为携带位置数据的容积视频轨道，容积视频轨道3、容积视频轨道6的编码方式分别为avc1、hvc1。因此，容积视频轨道3、容积视频轨道6之间具有替换关系，容积视频轨道3、容积视频轨道6的替换标识为11，即容积视频轨道3、容积视频轨道6可构成媒体轨道对11。容积视频轨道4、容积视频轨道7、容积视频轨道8的轨道类别均为携带占位图数据的容积视频轨道，容积视频轨道4、容积视频轨道7、容积视频轨道8的编码方式分别为avc1、hvc1、vvc1。因此，容积视频轨道4、容积视频轨道7、容积视频轨道8中任一两个容积视频轨道之间具有替换关系，容积视频轨道4、容积视频轨道7、容积视频轨道8的替换标识为12，容积视频轨道4、容积视频轨道7、容积视频轨道8中任一两个容积视频轨道构成媒体轨道对12。
如果播放终端对这些容积视频轨道进行解码，以重建和呈现容积媒体，容易导致播放终端的处理资源的浪费。基于此，为了使播放终端选择对具有替换关系的媒体对中的一个容积视频轨道进行解码，可以对各个容积视频轨道进行分组，得到多个播放组。播放组内的容积视频轨道之间具有联合播放关系，即播放组内的容积视频轨道可以构成一个完整的容积媒体(即容积视频)，播放组中仅包括具有替换关系的媒体对中的一个容积视频轨道。图5中各个容积轨道可以构成多个播放组，如图5中，容积视频轨道1-4可以构成播放组1，容积视频轨道1、容积视频轨道5-7可构成播放组2，容积视频轨道1、容积视频轨道5-6、容积视频轨道8可构成播放组3。当然，其他容积视频轨道的组合也可以构成播放组。
需要说明的是，音频编码包括但不限于：MP3(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、AAC(Advanced Audio Coding，高级音频编码)、FLAC(Free Lossless Audio Codec，无损音频压缩编码)。服务设备可以采用不同的音频编码方式生成具有替换关系的音频轨道对，可以将音频轨道(具有替换关系的音频轨道对中的一个音频轨道)添加至各个播放组中，并生成每两个播放组的替换关系指示信息以及替换属性信息，该替换关系指示信息用于指示每两个播放组中具有替换关系的媒体轨道对，该替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道(如容积视频轨道或音频轨道)之间的属性差异。此处属性差异可以包括媒体轨道对中的媒体轨道的编码方式等等。进一步，可以将播放组、替换关系指示信息以及替换属性信息按照容积媒体的文件格式(如ISOBMFF(ISO Base Media File Format，ISO基媒体文件格式))封装在文件容器中形成容积媒体的媒体文件资源，该媒体文件资源可以是媒体文件或媒体片段形成容积媒体的媒体文件。并按照容积媒体的文件格式要求采用媒体呈现描述信息(Media presentation description，MPD)记录该容积媒体的媒体文件资源的元数据，此处的元数据是对与容积媒体的呈现有关的信息的总称，该元数据可包括对媒体内容的描述信息、对视窗的描述信息以及对媒体内容呈现相关的信令信息等等。如图1所示，服务设备会存储经过数据处理过程之后形成的媒体呈现描述信息和媒体文件资源。
二、在播放终端端的数据处理过程：
(3)容积媒体的文件解封装及解码的过程；
播放终端可以通过服务设备的推荐或按照播放终端端的处理能力自适应动态从服务设备获得容积媒体的媒体文件资源和相应的媒体呈现描述信息。例如，播放终端可以根据该播放终端的解码能力， 向服务设备请求获得相应的媒体文件资源。媒体文件资源和媒体呈现描述信息通过传输机制(如DASH、SMT)由服务设备传输给播放终端。播放终端端的文件解封装的过程与服务设备端的文件封装过程是相逆的，播放终端按照容积媒体的文件格式要求对媒体文件资源进行解封装，得到音频轨道和容积视频轨道。播放终端端的解码过程与服务设备端的编码过程是相逆的，播放终端对音频轨道进行音频解码，还原出音频内容。另外，播放终端对容积视频轨道的解码得到视频内容。
(4)容积媒体的渲染过程。
播放终端根据媒体呈现描述信息中与渲染、视窗相关的元数据对音频解码得到的音频内容及视频解码得到的3D图像进行渲染，渲染完成即实现了对该3D图像的播放输出。特别地，如果采用3DoF和3DoF+的制作技术，播放终端主要基于当前视点、视差、深度信息等对3D图像进行渲染，如果采用6DoF的制作技术，播放终端主要基于当前视点对视窗内的3D图像进行渲染。其中，视点指用户的观看位置点，视差是指用户的双目产生的视线差或由于运动产生的视线差，视窗是指观看区域。
容积媒体系统支持数据盒(Box)，数据盒是指包括对容积媒体的播放组、每两个播放组之间的替换关系指示信息、替换属性信息、元数据的数据块或对象进行封装得到的。
综上，本申请中，通过向播放终端发送替换关系指示信息以及替换属性信息，有利于播放终端根据自身处理能力或用户需求，自适应地选择播放组进行解码处理，以重建和呈现容积媒体，可实现个性化呈现。另外，不需要对所有的媒体轨道进行解码处理，可节省播放终端的处理资源。
进一步地，请参见图6，是本申请实施例提供的一种容积媒体的数据处理方法的流程示意图。如图6所示，该方法可以由图1中的应用服务设备来执行，其中，该方法至少可以包括以下S101-S102：
S101、获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；该替换关系指示信息用于指示该每两个播放组中具有替换关系的媒体轨道对，该替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异。
本申请中，媒体轨道对中的媒体轨道可以是指容积视频轨道、音频轨道、文本轨道等等；容积媒体的每两个播放组之间包括具有替换关系的一个媒体轨道对或者至少两个媒体轨道对。媒体轨道对中的媒体轨道之间的属性差异可以用于反映播放组的质量差异，媒体轨道对中的媒体轨道之间的属性差异可以包括媒体轨道对中的各个媒体轨道的属性信息，媒体轨道属性信息包括编码方式、传输速率、质量等级、编码方式的编码类型中的一项或多项。编码方式可以包括视频编码方式和音频编码方式，编码方式的编码类型可以包括有损编码方式、无损编码方式等等。
需要说明的是，媒体轨道的属性信息可用于反映该媒体轨道的播放质量，进而，可以用于反映该媒体轨道所属的播放组的播放质量。例如，编码方式的编码类型为无损编码类型的媒体轨道的播放质量优于，编码方式的编码类型为有损编码类型的媒体轨道的播放质量。同时，信道传输媒体轨道所消耗的传输资源(如网络带宽)、播放终端处理媒体轨道所消耗的资源与媒体轨道的播放质量相关。如媒体轨道的质量等级越高，信道传输媒体轨道所消耗的传输资源、播放终端处理媒体轨道所消耗的资源越多。相反，媒体轨道的质量等级越低，信道传输媒体轨道所消耗的传输资源、播放终端处理媒体轨道所消耗的资源越少。
可选的，该媒体轨道对包括属于播放组P m的第一媒体轨道和属于播放组P n的第二媒体轨道，该播放组P m与该播放组P n之间的替换属性信息包括该第一媒体轨道的属性信息，以及该第二媒体轨道的属性信息，m和n均为小于Z的正整数，Z为该至少两个播放组的分组数量；该第一媒体轨道的属性信息包括该第一媒体轨道的编码方式，该第二媒体轨道的属性信息包括该第二媒体轨道的编码方式，该第一媒体轨道的编码方式与该第二媒体轨道的编码方式不相同。例如，第一媒体轨道、第二媒体轨道均为容积视频轨道，第一媒体轨道的编码方式为avc1，第二媒体轨道的编码方式为hvc1。通过标记媒体轨道的编码方式，有利于更加精细地获知各个播放组之间的质量差异，进而，有利于播放终端快速地，识别出与该播放终端的处理能力匹配的播放组，或者，识别出与用户需求匹配的播放组。
可选的，该第一媒体轨道的属性信息还包括该第一媒体轨道的传输速率、结构组织、质量等级和编码方式对应的编码类型中的一种或多种；该第二媒体轨道的属性信息还包括该第二媒体轨道的传输速率、结构组织、质量等级和编码方式对应的编码类型中的一种或多种，传输速率包括媒体轨道的帧 率或码率中的一种或两种。通过多个维度标记媒体轨道的属性信息，为播放终端提供更多关于播放组之间的差异属性，有利于播放终端更加准确地选择出与自身匹配的播放组。
可选的，该替换关系指示信息还包括替换级别字段；当该播放组P m和该播放组P n之间的替换关系指示信息中的替换级别字段为第一数值(如1)时，该替换级别字段用于反映该播放组P m与该播放组P n之间的替换级别为媒体内容级别；当该播放组P m和该播放组P n的替换关系指示信息中的替换级别字段为第二数值(如0)时，该替换级别字段用于反映该播放组P m与该播放组P n之间的替换级别为轨道级别。
可选的，当该播放组P m与该播放组P n之间的替换级别为媒体内容级别时，该播放组P m与该播放组P n中具有替换关系的媒体轨道对的数量为R，R为该播放组P m中的媒体轨道数量，该播放组P m中的媒体轨道数量与该播放组P n的轨道数量相同；也就是说，当该播放组P m与该播放组P n之间的替换级别为媒体内容级别时，该播放组P m与该播放组P n的媒体轨道之间具有一对一的替换关系。
例如，如图7所示，容积媒体包括两个播放组，分别为播放组1和播放组2，播放组1和播放组2中均包括携带位置数据的媒体轨道以及携带属性数据的媒体轨道。播放组1中的各个媒体轨道的编码方式的编码类型均为无损编码类型，播放组2中的各个媒体轨道的编码方式的编码类型为有损编码类型。因此，播放组1中携带位置数据的媒体轨道与播放组2中携带位置数据的媒体轨道之间具有替换关系，可构成媒体轨道对。同理，播放组1中携带属性数据的媒体轨道与播放组2中携带属性数据的媒体轨道之间具有替换关系，可构成媒体轨道对。可知，播放组1和播放组2中的轨道数量均为2，播放组1和播放组2之间的媒体轨道对为2，因此，可以称播放组1和播放组2之间的替换级别为媒体内容级别。
可选的，当该播放组P m与该播放组P n之间的替换级别为轨道级别时，该播放组P m与该播放组P n中具有替换关系的媒体轨道对的数量为小于R的正整数。也就是说，当该播放组P m与该播放组P n之间的替换级别为轨道级别时，该播放组P m存在媒体轨道与该播放组P n中的媒体轨道之间不具有替换关系，即该播放组P m部分媒体轨道与该播放组P n中的媒体轨道之间具有替换关系。
例如，如图8所示，容积媒体包括6个容积视频轨道，分别为容积视频轨道1-6，容积视频轨道1为携带2D与3D映射关系的容积视频轨道。容积媒体的播放组至少可包括播放组1和播放组2，播放组1可包括容积视频轨道1、容积视频轨道2、容积视频轨道3，播放组2可包括容积视频轨道1、容积视频轨道5、容积视频轨道6。容积视频轨道2、容积视频轨道5的轨道类别均为携带属性数据的容积视频轨道，容积视频轨道2、容积视频轨道5的编码方式分别为avc1、hvc1。因此，容积视频轨道2、容积视频轨道5之间具有替换关系，容积视频轨道2、容积视频轨道5的替换标识为10，即容积视频轨道2、容积视频轨道5构成媒体轨道对10。容积视频轨道3、容积视频轨道6的轨道类别均为携带位置数据的容积视频轨道，容积视频轨道3、容积视频轨道6的编码方式分别为avc1、hvc1。因此，容积视频轨道3、容积视频轨道6之间具有替换关系，容积视频轨道3、容积视频轨道6的替换标识为11，即容积视频轨道3、容积视频轨道6可构成媒体轨道对11。可知，播放组1和播放组2之间的媒体轨道对数量为2，播放组1和播放组2中的轨道数量均为3，即播放组1和播放组2之间的媒体轨道对数量小于播放组1中的轨道数量。因此，可以称播放组1和播放组2之间的替换级别为轨道级别。
可选的，当该播放组P m与该播放组P n之间的替换级别为媒体内容级别时，该播放组P m中的媒体轨道的编码方式均为第一编码方式；该播放组P n中的媒体轨道的编码方式均为第二编码方式。均采用同一种编码方式对播放组内的媒体轨道进行编码，有利于播放终端快速地对媒体轨道进行解码，快速地获取容积媒体，提高重建容积媒体的效率。
可选的，该至少两个播放组中各个播放组内的媒体轨道之间具有联合播放关系，该各个播放组内的媒体轨道之间的轨道类别不相同；该每两个播放组中具有替换关系的媒体轨道对中的媒体轨道之间的轨道类别相同。轨道类别可以用于反映媒体轨道所包含的内容，如果轨道类别包括音频轨道、视频轨道、文本轨道等等，进一步，可以根据视频轨道所携带的数据细分视频轨道的轨道类别，视频轨道的轨道类别包括携带2D与3D之间的映射关系的容积视频轨道，携带属性数据的容积视频轨道、携带位置数据的容积视频轨道等等。具有联合播放关系的媒体轨道可以是指根据具有联合播放关系的媒体 轨道可重建容积媒体，即构成完整的容积媒体。
可选的，该替换关系指示信息包括该每两个播放组中具有替换关系的媒体轨道对的替换标识。
需要说明的是，服务设备可以通过容积媒体的数据盒或传输协议，获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息。
例如，服务设备可以获取该容积媒体的数据盒；该容积媒体的数据盒中包括该每两个播放组之间的替换关系指示信息和替换属性信息。该数据盒中还可以包括容积媒体的至少两个播放组，即该数据盒可以是指对容积媒体的至少两个播放组、替换关系指示信息和替换属性信息进行封装得到的。如基于容积媒体的数据盒的语法可参见下述表1：
表1
Figure PCTCN2022099706-appb-000001
Figure PCTCN2022099706-appb-000002
其中，表1中is_content_alternative为两个播放组之间的替换级别字段，该字段取值为1，表明该两个播放组之间替换级别为媒体内容级别。该字段取值为0，表明该两个播放组之间替换级别为轨道级别，即两个播放组内的部分轨道之间具有替换关系。alternative_info_flag替换标识字段，该字段取值为1，播放组中的媒体轨道携带差异化信息，即该媒体轨道为媒体轨道对中的一个媒体轨道，该字段取值为0，表明该媒体轨道不携带差异化信息，即该媒体轨道不属于媒体轨道对中的一个媒体轨道。PlayoutAlternativeInfoBox为数据盒字段，alternative_type：可替换轨道的差异属性类型，根据该差异类型的取值，该轨道可以具备一个或多个差异属性。quality_ranking质量等级字段，该字段取值越小，表明对应轨道的质量越高。lossless_flag：该字段取值为0，表明对应媒体轨道的编码方式的编码类型为有损编码类型；该字段取值为1，表明对应媒体轨道的编码方式的编码类型为无损编码类型。Bitrate为媒体轨道的码率信息，指示对应媒体轨道的码率。Framerate为媒体轨道的帧率信息，指示对应媒体轨道的帧率。codec_type为媒体轨道的编码方式，is_tile_based为轨道的结构组织字段，该字段取值为1，表明媒体轨道以tile结构组织；该字段取值为0，表明媒体轨道未划分tile。存在该字段时，is_content_alternative取值必须为1。
可选的，服务设备可获取该容积媒体的传输协议，该传输协议中包括该每两个播放组之间的替换关系指示信息和替换属性信息。具体的，当该传输协议为动态自适应流媒体传输协议时，该每两个播放组之间的替换关系指示信息和替换属性信息位于该动态自适应流媒体传输协议的轨道描述字段中；当该传输协议为动态图像专家组媒体传输协议时，该每两个播放组之间的替换关系指示信息和替换属性信息位于该动态图像专家组媒体传输协议的资源组内。通过将替换关系指示信息和替换属性信息添加至传输协议中，即可实现对指示信息(即替换关系指示信息和替换属性信息)与至少两个播放组进行单独封装，这样可以优先将传输信息发送至播放终端，播放终端可根据传输协议请求与其匹配的播放组，可避免将所有的播放组发送至播放终端，造成网络带宽浪费的问题，可节省网络带宽资源。例如，当该传输协议为动态自适应流媒体传输协议时，在动态自适应流媒体传输协议添加的关于替换关系指示信息和替换属性信息的字段如表2所示：
表2
Figure PCTCN2022099706-appb-000003
其中，表2中，动态自适应流媒体传输协议中包括三个扩展字段，分别为：alternative_Id、quality_ranking、lossless_flag，alternative_Id表示媒体轨道对的替换标识，数据类型为无符号整数型。quality_ranking表示媒体轨道的质量等级，该字段取值越小，表明该媒体轨道的质量越高，数据类型为无符号字节类型。lossless_flag指示该媒体轨道的编码方式的编码类型是否为无损编码，该字段取值为0，表明该媒体轨道的编码方式的编码类型为有损编码类型；表明该媒体轨道的编码方式 的编码类型为无损编码类型，数据类型为布尔类型。alternative_Id、quality_ranking、lossless_flag对应的O表示，这几个字段均为可选的字段。
例如，当该传输协议为动态自适应流媒体传输协议时，在动态自适应流媒体传输协议添加的关于替换关系指示信息和替换属性信息的字段如表3所示：
表3
Figure PCTCN2022099706-appb-000004
Figure PCTCN2022099706-appb-000005
其中，表3中，alternative_content_flag表示替换级别字段，该字段取值为1，表明对应媒体资源组(即播放组)与其他媒体资源组构成可替代关系，该媒体资源组与其他资源组之间的替换级别为媒体内容级别。该字段取值为0，表明对应媒体资源组不与任何媒体资源组构成替代关系。alternative_asset_flag替换标识，该字段取值为1，表明对应媒体资源组的媒体轨道与其他组内的媒体轨道之间构成替代关系。该字段取值为0，表明对应媒体资源组内不存在可替换的媒体轨道。alternative_type：表示替换媒体资源(媒体轨道)的差异属性类型，根据该差异类型的取值，该轨道可以具备一个或多个差异属性。quality_ranking表示媒体资源的质量等级信息，该字段取值越小，表明对应媒体资源的质量越高。lossless_flag表示媒体资源的编码方式的编码类型，该字段取值为0，表明对应媒体资源的编码类型为有损编码类型；该字段取值为1，表明对应媒体资源的编码类型为无损 编码类型。Bitrate表示码率信息，指示对应媒体资源的码率。Framerate表示帧率信息，指示对应媒体资源的帧率。codec_type表示编码方式，指示对应媒体资源的编码方式。is_tile_based表示媒体资源的结构组织字段，该字段取值为1，表明该媒体资源组内的媒体资源以tile结构组织；该字段取值为0，表明该媒体资源组内的媒体资源不存在tile结构。
S102、将该替换关系指示信息和该替换属性信息发送至播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体。
本申请中，服务设备可以对该至少两个播放组、该替换关系指示信息和该替换属性信息进行封装处理，得到媒体资源文件，将该媒体资源文件发送至播放终端。播放终端可以从媒体资源文件进行解封处理，得到该至少两个播放组、该替换关系指示信息和该替换属性信息。在一个实施例中，播放终端可根据该替换关系指示信息和该替换属性信息，从该至少两个播放组中选择与该播放终端的处理能力适配的目标播放组，对目标播放组进行解码得到容积媒体。在一个实施例中，播放终端可以根据该替换关系指示信息和该替换属性信息，从该至少两个播放组中选择与用户需求匹配的目标播放组，对目标播放组进行解码得到容积媒体。
本申请中，服务设备可以获取容积媒体的至少两个播放组中每个播放组之间的替换关系指示信息以及替换属性信息，将该替换关系指示信息、替换属性信息以及至少两个播放组发送至播放终端。由于替换关系指示信息用于指示每两个播放组中具有替换关系的媒体轨道对，替换属性信息用于反映媒体轨道对中的媒体轨道之间的属性差异；也就是说，根据替换关系指示信息和替换属性信息可知各个播放组的播放质量。通过向播放终端发送替换关系指示信息以及替换属性信息，有利于播放终端根据自身处理能力或用户需求，自适应地选择播放组进行解码处理，以重建和呈现容积媒体，可实现个性化呈现。另外，不需要对所有的媒体轨道进行解码处理，可节省播放终端的处理资源。
进一步地，请参见图9，是本申请实施例提供的一种容积媒体的数据处理方法的流程示意图。如图9所示，该方法可以由图1中的服务设备来执行，其中，该方法至少可以包括以下S201-S203：
S201、获取容积媒体的至少两个播放组。
可选的，步骤S201可包括如下步骤s11～s12。
s11、获取容积媒体的至少两个媒体轨道，以及该至少两个媒体轨道中的各个媒体轨道的轨道类别；
s12、根据该各个媒体轨道的轨道类别，对该至少两个媒体轨道进行划分，得到至少两个播放组；该至少两个播放组中各个播放组内的媒体轨道之间具有联合播放关系，该各个播放组内的媒体轨道之间的轨道类别不相同。
在步骤s11～s12中，服务设备可以获取容积媒体的至少两个媒体轨道，以及各个媒体轨道的轨道类别，根据各个媒体轨道的轨道类别，将具有不同轨道类别的媒体轨道划分至同一个播放组，得到至少两个播放组。
S202、对该至少两个播放组中的每两个播放组进行标记，得到该每两个播放组之间的替换关系指示信息和替换属性信息；该替换关系指示信息用于指示该每两个播放组中具有替换关系的媒体轨道对，该替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异。
本申请中，服务设备可以对该至少两个播放组中的每两个播放组中具有替换关系的媒体轨道对进行标记，得到每两个播放组之间的替换关系指示信息；获取媒体轨道对中的媒体轨道的属性信息之间的属性差异，得到每两个播放组之间的替换属性信息。通过标记每两个播放组之间的替换关系指示信息和替换属性信息，有利于播放终端根据自身处理能力或用户需求，动态自适应地选择播放组进行解码处理，以重建和呈现容积媒体，可实现个性化呈现。
可选的，步骤S202可包括如下步骤s21～s26。
s21、根据播放组P m的媒体轨道的轨道类别与播放组P n中媒体轨道的轨道类别，确定候选媒体轨道对；该候选媒体轨道对中的媒体轨道之间具有相同的轨道类别，m和n均为小于Z的正整数，Z为该 至少两个播放组的分组数量。
s22、获取该候选媒体轨道对中的媒体轨道的属性信息。
s23、对该候选媒体轨道对中的媒体轨道的属性信息之间进行差异识别，得到差异识别结果。
s24、若该差异识别结果指示该候选媒体轨道对中的媒体轨道的属性信息具有差异，则将该候选媒体轨道对作为具有替换关系的媒体轨道对，生成该媒体轨道对的替换标识。
s25、将该替换标识作为该播放组P m和该播放组P n之间的替换关系指示信息。
s26、将候选媒体轨道对中的媒体轨道的属性信息作为该播放组P m和该播放组P n之间的替换属性信息。
在步骤s21～s26中，服务设备可以从播放组P m与播放组P n中识别出具有相同轨道类别的候选媒体轨道对，获取候选媒体轨道对中的媒体轨道的属性信息。将候选媒体轨道对中的媒体轨道之间的属性信息进行差异识别，得到差异识别结果，该差异识别结果用于反映候选媒体轨道对中的媒体轨道之间的属性信息是否存在差异。候选媒体轨道对中的媒体轨道之间的属性信息不存在差异，表明候选媒体轨道对中的媒体轨道的属性信息相同，即候选媒体轨道对中的媒体轨道的播放质量相同，则该候选媒体轨道对中的媒体轨道之间不具有替换关系。若候选媒体轨道对中的媒体轨道之间的属性信息存在差异，表明候选媒体轨道对中的媒体轨道的属性信息不相同，即候选媒体轨道对中的媒体轨道的播放质量不相同，则该候选媒体轨道对中的媒体轨道之间具有替换关系。进一步，可以将该候选媒体轨道对作为具有替换关系的媒体轨道对，生成该媒体轨道对的替换标识。进一步，可以将该替换标识作为该播放组P m和该播放组P n之间的替换关系指示信息，并可以将候选媒体轨道对中的媒体轨道的属性信息作为该播放组P m和该播放组P n之间的替换属性信息。通过标记每两个播放组之间的替换关系指示信息和替换属性信息，有利于播放终端根据自身处理能力或用户需求，动态自适应地选择播放组进行解码处理，以重建和呈现容积媒体，可实现个性化呈现。
S203、将该替换关系指示信息和该替换属性信息发送至播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体。
本申请中，服务设备可以对该至少两个播放组、该替换关系指示信息和该替换属性信息进行封装处理，得到媒体资源文件，将该媒体资源文件发送至播放终端。播放终端可以从媒体资源文件进行解封处理，得到该至少两个播放组、该替换关系指示信息和该替换属性信息。进一步，播放终端可以根据该替换关系指示信息和该替换属性信息，从至少两个播放组中选择与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体。播放终端不需要对所有的媒体轨道进行解码处理，可节省播放终端的处理资源。
可选的，步骤S203中服务设备可以通过数据盒或传输协议向播放终端发送替换关系指示信息以及替换属性信息。具体的，当服务设备可以通过数据盒向播放终端发送替换关系指示信息以及替换属性信息时，步骤S203可包括步骤s31-s32。
s31、对该至少两个播放组、该每两个播放组之间的替换关系指示信息和替换属性信息进行封装，得到该容积媒体的数据盒。
s32、将包括该至少两个播放组、该替换关系指示信息和该替换属性信息的数据盒发送至该播放终端。
在步骤s31-s32中，服务设备可以按照容积媒体的文件格式(如ISOBMFF(ISO Base Media File Format，ISO基媒体文件格式))对该至少两个播放组、该每两个播放组之间的替换关系指示信息和替换属性信息进行封装，得到该容积媒体的数据盒，将包括该至少两个播放组、该替换关系指示信息和该替换属性信息的数据盒发送至该播放终端。也就是说，将该至少两个播放组、该替换关系指示信息和该替换属性信息封装至同一个资源文件中，播放终端通过下载该资源文件可以获取至少两个播放组、替换关系指示信息以及替换属性信息，有利于提高播放终端获取这些数据的便捷性。
可选的，当服务设备可以通过传输协议向播放终端发送替换关系指示信息以及替换属性信息时，步骤S203可包括步骤s41-s45。
s41、获取该容积媒体的传输协议。
s42、将该每两个播放组之间的替换关系指示信息和替换属性信息添加至该传输协议中。
s43、将包括上述替换关系指示信息和上述替换属性信息的传输协议发送至播放终端。
s44、接收上述播放终端发送的、且根据传输协议中的替换关系指示信息和替换属性信息生成的关于目标播放组的获取请求，上述目标播放组为上述至少两个播放组中与上述播放终端匹配的播放组；
s45、响应于获取请求，将上述目标播放组发送至上述播放终端，以指示播放终端对目标播放组进行解码处理，得到容积媒体。
在步骤s41-s45中，服务设备可以获取容积媒体的传输协议，该传输协议包括动态自适应流媒体传输协议或动态图像专家组媒体传输协议；将该每两个播放组之间的替换关系指示信息和替换属性信息添加至该传输协议中。然后，将包括该替换关系指示信息和该替换属性信息的传输协议发送至播放终端，播放终端可以根据传输协议中的替换关系指示信息以及替换属性信息，确定与播放终端匹配的目标播放组对应的分组标识，从服务设备中请求该分组标识对应的目标播放组。也就是说，将该至少两个播放组和该替换关系指示信息单独封装至一个文件中，这样不需要将所有的播放组发送至播放终端，可以节省网络带宽资源。
可选的，步骤s42可包括：当该传输协议为动态自适应流媒体传输协议时，将该每两个播放组之间的替换关系指示信息和替换属性信息添加至该动态自适应流媒体传输协议的轨道描述字段中。当该传输协议为动态图像专家组媒体传输协议时，将该每两个播放组之间的替换关系指示信息和替换属性信息添加至该动态图像专家组媒体传输协议的资源组内。通过将替换关系指示信息和替换属性信息添加至动态自适应流媒体传输协议中，或者，该动态自适应流媒体传输协议中，有利于播放终端提前预知替换关系指示信息和替换属性信息，能够在一定程度上能够提升对播放组的解码效率，从而带来更好的关于容积媒体的呈现效果。
本申请中，服务设备可以对容积媒体的每两个播放组中具有替换关系的媒体轨道对进行标记，得到每两个播放组之间的替换关系指示信息；并获取该媒体轨道对中的媒体轨道的属性信息，对媒体轨道对中的媒体轨道的属性信息进行差异识别，得到每两个播放组之间的替换属性信息。将至少两个播放组、替换关系指示信息以及替换属性信息发送至播放终端。可见，通过向播放终端发送替换关系指示信息以及替换属性信息，有利于播放终端根据自身处理能力或用户需求，自适应地选择播放组进行解码处理，以重建和呈现容积媒体，可实现个性化呈现。另外，不需要对所有的媒体轨道进行解码处理，可节省播放终端的处理资源。
进一步地，请参见图10，是本申请实施例提供的一种容积媒体的数据处理方法的流程示意图。如图10所示，该方法可以由图1中的播放终端来执行，其中，该方法至少可以包括以下S301-S303：
S301、获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；该替换关系指示信息用于指示该每两个播放组中具有替换关系的媒体轨道对，该替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异。
播放终端可以从服务设备中获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息。
S302、根据该替换关系指示信息和该替换属性信息，从该至少两个播放组中识别出与播放终端匹配的目标播放组。
在一个实施例中，播放终端可以根据该替换关系指示信息以及替换属性信息，确定处理各个播放分组所需要消耗的处理资源等等；然后，播放终端可以根据处理各个播放分组所需要消耗的处理资源，从至少两个播放组中选择与该播放终端的处理能力匹配的目标播放组。在一个实施例中，可以根据该替换关系指示信息以及替换属性信息，确定各个播放分组的播放质量，根据各个播放分组的播放质量，从至少两个播放组中选择与用户的播放质量需求匹配的目标播放组。
S303、对该目标播放组进行解码处理，得到该容积媒体。
播放终端可以对该目标播放组进行解码处理，得到该容积媒体，播放容积媒体，即不需要播放终 端对所有的播放组进行解码处理，可以节省播放终端的处理资源。
可选的，如图11所示，当服务设备通过传输协议传输替换关系属性信息和替换属性信息时，步骤S302可包括：接收服务设备发送的该容积媒体的传输协议，对该传输协议进行解析，得到该替换关系指示信息和该替换属性信息。
可选的，上述对该传输协议进行解析，得到该替换关系指示信息和该替换属性信息，包括：当该传输协议为动态自适应流媒体传输协议时，该动态自适应流媒体传输协议的轨道描述字段中读取该替换关系指示信息和该替换属性信息；当该传输协议为动态图像专家组媒体传输协议时，从该动态图像专家组媒体传输协议的资源组内读取该替换关系指示信息和该替换属性信息。播放终端可以接收服务设备发送的该容积媒体的传输协议，该传输协议可以是指动态自适应流媒体传输协议或动态图像专家组媒体传输协议。当该传输协议为动态自适应流媒体传输协议时，播放终端可以从动态自适应流媒体传输协议的轨道描述字段中获取该替换关系指示信息和替换属性信息。当该传输协议为动态图像专家组媒体传输协议时，播放终端可以从传输协议为动态图像专家组媒体传输协议的资源组内读取该替换关系指示信息和替换属性信息。
当服务设备通过传输协议传输替换关系属性信息和替换属性信息时，步骤S303可包括步骤s51-s54。
s51、根据该替换关系指示信息和该替换属性信息，从该至少两个播放组中确定出与播放终端匹配的候选播放组的分组标识。
s52、生成携带该候选播放分组的分组标识的获取请求，向该服务设备发送该获取请求。
s53、接收该服务设备响应该获取请求所返回的候选播放组。
s54、将该候选播放组作为与该播放终端匹配的目标播放组。
在步骤s51-s54中，如图11所示，播放终端可以根据该替换关系指示信息和该替换属性信息，从该至少两个播放组中确定出与该播放终端的处理能力匹配的候选播放组的分组标识；或者，播放终端可以根据该替换关系指示信息和该替换属性信息，从该至少两个播放组中确定出与该播放终端对应的用户需求匹配的候选播放组的分组标识。然后，可以生成携带该候选播放分组的分组标识的获取请求，向该服务设备发送该获取请求，接收服务设备响应该获取请求返回的候选播放组，将该候选播放组作为与该播放终端匹配的目标播放组。通过预先从服务设备中获取携带替换关系指示信息和替换属性信息的传输协议，有利于播放终端根据该替换关系指示信息和替换属性信息确定出与播放终端匹配的目标播放组，只需要从服务设备中获取目标播放组，不需要下载所有的播放组，可节省网络带宽资源。
可选的，步骤s51可包括步骤s61-s63。
s61、获取该播放终端的属性信息，根据该播放终端的属性信息确定该播放终端处理容积媒体的处理性能等级。
s62、根据上述替换关系指示信息和上述替换属性信息，确定处理上述至少两个播放组中各个播放组的处理难度等级；上述处理难度等级是根据处理上述各个播放组所消耗的处理资源量确定的。
s63、将上述至少两个播放组中处理难度等级与上述播放终端的处理性能等级匹配的播放组，作为候选播放组，获取上述候选播放组的分组标识。
在步骤s61-s63中，播放终端可以获取该播放终端的属性信息，该播放终端的属性信息包括存储空间大小、数据处理速度等等，根据该播放终端的属性信息确定该播放终端处理容积媒体的处理性能等级(如解码能力等级)。进一步，可以根据替换关系指示信息和替换属性信息确定处理各个播放组所需要消耗的处理资源量，根据处理各个播放组所需要消耗的处理资源量，生成处理该至少两个播放组中各个播放组的处理难度等级。即处理各个播放组所需要消耗的处理资源量，与处理各个播放组的处理难度等级之间具有正相关关系，即处理各个播放组的处理难度等级，随处理各个播放组所需要消耗的处理资源量增加而变高，随处理各个播放组所需要消耗的处理资源量减少而变低。进一步，可以将上述至少两个播放组中处理难度等级与上述播放终端的处理性能等级匹配的播放组，作为候选播放组，获取上述候选播放组的分组标识。通过播放组的处理难度等级与播放终端的处理性能等级，选择目标 播放组，可实现个性化呈现。例如，该播放终端的处理性能等级比较高，则可以从服务设备中获取处理难度等级高的目标播放组，此时该目标播放组的播放质量越高，可以提高容积媒体的播放效果。相反，该播放终端的处理性能等级比较低，则可以从服务设备中获取处理难度等级低的目标播放组，这样有利于确保播放容积媒体的流畅性。
可选的，如图12所示，当服务设备通过数据盒传输替换关系属性信息和替换属性信息时，步骤S301可包括：从服务设备中获取该容积媒体的数据盒，对该容积媒体的数据盒进行解封装处理，得到该至少两个播放组、该至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息。
可选的，步骤s51可包括：获取该播放终端历史所播放的容积媒体的第一质量等级；根据该替换关系指示信息和该替换属性信息，确定该至少两个播放组中各个播放组的第二质量等级；将该至少两个播放组中第二质量等级与该第一质量等级匹配的播放组，作为候选播放组，获取该候选播放组的分组标识。
播放终端可以根据历史播放行为数据选择匹配的目标播放组，具体的，可以获取播放终端历史(如近一周、近一个月)所播放的容积媒体的第一质量等级，根据该替换关系指示信息和该替换属性信息，确定该至少两个播放组中各个播放组的第二质量等级；如根据媒体轨道对的编码方式的编码类型确定各个播放组的第二质量等级，即编码类型为无损编码类型的媒体轨道对应的播放组的质量等级，高于编码类型为有损编码类型的媒体轨道对应的播放组的质量等级。将该至少两个播放组中第二质量等级与该第一质量等级相同的播放组，作为候选播放组，获取该候选播放组的分组标识。通过根据历史播放行为数据选择匹配的目标播放组，有利于为用户呈现满意的容积媒体。
可选的，播放终端可以根据该替换关系指示信息和该替换属性信息，确定该至少两个播放组中各个播放组的第二质量等级，展示各个播放组的第二质量等级，响应于根据上述第二质量等级从至少两个播放组中播放组的选择操作，将所选择的播放组，作为候选播放组，获取该候选播放组的分组标识。通过展示各个播放组的第二质量等级，有利于用户根据自己的喜好选择播放组，有利于实现对容积媒体的个性化呈现。当服务设备通过数据盒传输替换关系属性信息和替换属性信息时，播放终端可以通过数据盒获取替换关系属性信息和替换属性信息、至少两个播放组。具体的，播放终端可以从服务设备中获取该容积媒体的数据盒，对该容积媒体的数据盒进行解封装处理，得到该至少两个播放组、该至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息。通过对容积媒体的数据盒进行解封装处理，即可得到该至少两个播放组、替换关系指示信息和替换属性信息，可提高获取这些信息的便捷性。
可选的，上述步骤S303中包括：对该目标播放组进行解码处理，得到该容积媒体的媒体内容；对媒体内容时间同步，得到该容积媒体。
可选的，该容积媒体的媒体内容包括音频内容、视频内容、以及文本内容中的至少一种。
播放终端可以从目标播放组的描述信息中确定目标播放组中的各个媒体轨道的编码方式，根据目标播放组中的各个媒体轨道的编码方式确定各个媒体轨道的解码方式。采用各个媒体轨道的解码方式对各个媒体轨道进行解码，得到容积媒体的媒体内容；对该音频内容、该视频内容、以及该文本内容进行时间同步，得到该容积媒体。只需要播放终端对至少两个播放组中的一个播放组进行解码操作，即可得到容积媒体，不需要对所有的播放组进行解码操作，可节省播放终端的解码资源。本申请中，播放终端通过根据替换关系指示信息和替换属性信息，从至少两个播放组中动态自适应地选择与其匹配的目标播放组，只需要对目标播放组进行解码操作，即可得到容积媒体，不需要对所有的播放组进行解码操作，可以节省播放终端的处理资源。
请参见图13，是本申请实施例提供的一种容积媒体的数据处理装置1的结构示意图。上述容积媒体的数据处理装置1可以是运行于计算机设备中的一个计算机可读指令(包括程序代码)，例如该容积媒体的数据处理装置1为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的相应步骤。如图13所示，该容积媒体的数据处理装置1可以包括：获取模块131以及发送模块132。
获取模块，用于获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述 替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
发送模块，用于将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体。
可选的，上述获取模块获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
获取上述容积媒体的数据盒；上述容积媒体的数据盒中包括上述每两个播放组之间的替换关系指示信息和替换属性信息。
可选的，获取模块获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
获取上述容积媒体的传输协议，上述传输协议中包括上述每两个播放组之间的替换关系指示信息和替换属性信息。
可选的，当上述传输协议为动态自适应流媒体传输协议时，上述每两个播放组之间的替换关系指示信息和替换属性信息位于上述动态自适应流媒体传输协议的轨道描述字段中；
可选的，当上述传输协议为动态图像专家组媒体传输协议时，上述每两个播放组之间的替换关系指示信息和替换属性信息位于上述动态图像专家组媒体传输协议的资源组内。
可选的，上述媒体轨道对包括属于播放组P m的第一媒体轨道和属于播放组P n的第二媒体轨道，上述播放组P m与上述播放组P n之间的替换属性信息包括上述第一媒体轨道的属性信息，以及上述第二媒体轨道的属性信息，m和n均为小于Z的正整数，Z为上述至少两个播放组的分组数量；
上述第一媒体轨道的属性信息包括上述第一媒体轨道的编码方式，上述第二媒体轨道的属性信息包括上述第二媒体轨道的编码方式，上述第一媒体轨道的编码方式与上述第二媒体轨道的编码方式不相同。
可选的，上述第一媒体轨道的属性信息还包括上述第一媒体轨道的传输速率、结构组织、质量等级和编码方式对应的编码类型中的至少一种；
上述第二媒体轨道的属性信息还包括上述第二媒体轨道的传输速率、结构组织、质量等级和编码方式对应的编码类型。
可选的，上述替换关系指示信息还包括替换级别字段；当上述播放组P m和上述播放组P n之间的替换关系指示信息中的替换级别字段为第一数值时，上述替换级别字段用于反映上述播放组P m与上述播放组P n之间的替换级别为媒体内容级别。
可选的，上述替换关系指示信息还包括替换级别字段；当上述播放组P m和上述播放组P n的替换关系指示信息中的替换级别字段为第二数值时，上述替换级别字段用于反映上述播放组P m与上述播放组P n之间的替换级别为轨道级别。
可选的，当上述播放组P m与上述播放组P n之间的替换级别为媒体内容级别时，上述播放组P m与上述播放组P n中具有替换关系的媒体轨道对的数量为R，R为上述播放组P m中的媒体轨道数量，上述播放组P m中的媒体轨道数量与上述播放组P n的轨道数量相同。
可选的，当上述播放组P m与上述播放组P n之间的替换级别为轨道级别时，上述播放组P m与上述播放组P n中具有替换关系的媒体轨道对的数量为小于R的正整数。
可选的，当上述播放组P m与上述播放组P n之间的替换级别为媒体内容级别时，上述播放组P m中的媒体轨道的编码方式均为第一编码方式；上述播放组P n中的媒体轨道的编码方式均为第二编码方式。
可选的，上述至少两个播放组中各个播放组内的媒体轨道之间具有联合播放关系，上述各个播放组内的媒体轨道之间的轨道类别不相同；
上述每两个播放组中具有替换关系的媒体轨道对中的媒体轨道之间的轨道类别相同。
可选的，上述替换关系指示信息包括上述每两个播放组中具有替换关系的媒体轨道对的替换标识。
根据本申请的一个实施例，图6所示的容积媒体的数据处理方法所涉及的步骤可由图13所示的容 积媒体的数据处理装置中的各个模块来执行。例如，图6中所示的步骤S101可由图13中的获取模块131来执行，图6中所示的步骤S102可由图13中的发送模块132来执行。
根据本申请的一个实施例，图13所示的容积媒体的数据处理装置中的各个模块可以分别或全部合并为一个或若干个单元来构成，或者其中的某个(些)单元还可以再拆分为功能上更小的多个子单元，可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述模块是基于逻辑功能划分的，在实际应用中，一个模块的功能也可以由多个单元来实现，或者多个模块的功能由一个单元实现。在本申请的其它实施例中，容积媒体的数据处理装置也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。
根据本申请的一个实施例，可以通过在包括中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的例如计算机的通用计算机设备上运行能够执行如图6中所示的相应方法所涉及的各步骤的计算机可读指令(包括程序代码)，来构造如图13中所示的容积媒体的数据处理装置，以及来实现本申请实施例的容积媒体的数据处理方法。上述计算机可读指令可以记载于例如计算机可读记录介质上，并通过计算机可读记录介质装载于上述计算设备中，并在其中运行。
本申请中，服务设备可以对容积媒体的每两个播放组中具有替换关系的媒体轨道对进行标记，得到每两个播放组之间的替换关系指示信息；并获取该媒体轨道对中的媒体轨道的属性信息，对媒体轨道对中的媒体轨道的属性信息进行差异识别，得到每两个播放组之间的替换属性信息。将至少两个播放组、替换关系指示信息以及替换属性信息发送至播放终端。可见，通过向播放终端发送替换关系指示信息以及替换属性信息，有利于播放终端根据自身处理能力或用户需求，自适应地选择播放组进行解码处理，以重建和呈现容积媒体，可实现个性化呈现。另外，不需要对所有的媒体轨道进行解码处理，可节省播放终端的处理资源。
请参见图14，是本申请实施例提供的一种容积媒体的数据处理装置2的结构示意图。上述容积媒体的数据处理装置2可以是运行于计算机设备中的一个计算机可读指令(包括程序代码)，例如该容积媒体的数据处理装置2为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的相应步骤。如图14所示，该容积媒体的数据处理装置2可以包括：获取模块141、标记模块142以及发送模块143。
获取模块，用于获取容积媒体的至少两个播放组；
标记模块，用于对上述至少两个播放组中的每两个播放组进行标记，得到上述每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
发送模块，用于将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体。
可选的，上述获取模块获取容积媒体的至少两个播放组，包括：
获取容积媒体的至少两个媒体轨道，以及上述至少两个媒体轨道中的各个媒体轨道的轨道类别；
根据上述各个媒体轨道的轨道类别，对上述至少两个媒体轨道进行划分，得到至少两个播放组；上述至少两个播放组中各个播放组内的媒体轨道之间具有联合播放关系，上述各个播放组内的媒体轨道之间的轨道类别不相同。
可选的，上述标记模块对上述至少两个播放组中的每两个播放组进行标记，得到上述每两个播放组之间的替换关系指示信息和替换属性信息，包括：
根据播放组P m的媒体轨道的轨道类别与播放组P n中媒体轨道的轨道类别，确定候选媒体轨道对；上述候选媒体轨道对中的媒体轨道之间具有相同的轨道类别，m和n均为小于Z的正整数，Z为上述至少两个播放组的分组数量；
获取上述候选媒体轨道对中的媒体轨道的属性信息；
对上述候选媒体轨道对中的媒体轨道的属性信息之间进行差异识别，得到差异识别结果；
若上述差异识别结果指示上述候选媒体轨道对中的媒体轨道之间的属性信息具有差异，则将上述 候选媒体轨道对作为具有替换关系的媒体轨道对，生成上述媒体轨道对的替换标识；
将上述替换标识作为上述播放组P m和上述播放组P n之间的替换关系指示信息；
将上述候选媒体轨道对中的媒体轨道的属性信息作为上述播放组P m和上述播放组P n之间的替换属性信息。
可选的，上述发送模块将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体，包括：
对上述至少两个播放组、上述每两个播放组之间的替换关系指示信息和替换属性信息进行封装，得到上述容积媒体的数据盒；
将包括上述至少两个播放组、上述替换关系指示信息和上述替换属性信息的数据盒发送至上述播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体。
可选的，上述发送模块将上述替换关系指示信息和上述替换属性信息发送至播放终端，以指示播放终端根据替换关系指示信息和替换属性信息，从至少两个播放组中识别出与播放终端匹配的目标播放组，对目标播放组进行解码处理，得到容积媒体，包括：
获取上述容积媒体的传输协议；
将上述每两个播放组之间的替换关系指示信息和替换属性信息添加至上述传输协议中；
将包括上述替换关系指示信息和上述替换属性信息的传输协议发送至播放终端；
接收上述播放终端发送的、且根据传输协议中的替换关系指示信息和替换属性信息生成的关于目标播放组的获取请求，上述目标播放组为上述至少两个播放组中与上述播放终端匹配的播放组；
响应于获取请求，将上述目标播放组发送至上述播放终端，以指示播放终端对目标播放组进行解码处理，得到容积媒体。
可选的，上述发送模块将上述每两个播放组之间的替换关系指示信息和替换属性信息添加至上述传输协议中，包括：
当上述传输协议为动态自适应流媒体传输协议时，将上述每两个播放组之间的替换关系指示信息和替换属性信息添加至上述动态自适应流媒体传输协议的轨道描述字段中。
可选的，上述发送模块将上述每两个播放组之间的替换关系指示信息和替换属性信息添加至上述传输协议中，包括：
当上述传输协议为动态图像专家组媒体传输协议时，将上述每两个播放组之间的替换关系指示信息和替换属性信息添加至上述动态图像专家组媒体传输协议的资源组内。
根据本申请的一个实施例，图9所示的容积媒体的数据处理方法所涉及的步骤可由图14所示的容积媒体的数据处理装置中的各个模块来执行。例如，图9中所示的步骤S201可由图14中的获取模块141来执行，图9中所示的步骤S202可由图14中的标记模块142来执行；图9中所示的步骤S203可由图14中的发送模块143来执行。
根据本申请的一个实施例，图14所示的容积媒体的数据处理装置2的各个模块可以分别或全部合并为一个或若干个单元来构成，或者其中的某个(些)单元还可以再拆分为功能上更小的多个子单元，可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述模块是基于逻辑功能划分的，在实际应用中，一个模块的功能也可以由多个单元来实现，或者多个模块的功能由一个单元实现。在本申请的其它实施例中，容积媒体的数据处理装置也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。
根据本申请的一个实施例，可以通过在包括中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的例如计算机的通用计算机设备上运行能够执行如图9中所示的相应方法所涉及的各步骤的计算机可读指令(包括程序代码)，来构造如图14中所示的容积媒体的数据处理装置，以及来实现本申请实施例的容积媒体的数据处理方法。上述计算机可读指令可以记载于例如计算机可读记录介质上，并通过计算机可读记录介质装载于上述计算设备中，并在其中运行。
本申请中，服务设备可以对容积媒体的每两个播放组中具有替换关系的媒体轨道对进行标记，得到每两个播放组之间的替换关系指示信息；并获取该媒体轨道对中的媒体轨道的属性信息，对媒体轨道对中的媒体轨道的属性信息进行差异识别，得到每两个播放组之间的替换属性信息。将至少两个播放组、替换关系指示信息以及替换属性信息发送至播放终端。可见，通过向播放终端发送替换关系指示信息以及替换属性信息，有利于播放终端根据自身处理能力或用户需求，自适应地选择播放组进行解码处理，以重建和呈现容积媒体，可实现个性化呈现。另外，不需要对所有的媒体轨道进行解码处理，可节省播放终端的处理资源。
请参见图15，是本申请实施例提供的一种容积媒体的数据处理装置3的结构示意图。上述容积媒体的数据处理装置3可以是运行于计算机设备中的一个计算机可读指令(包括程序代码)，例如该容积媒体的数据处理装置3为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的相应步骤。如图15所示，该容积媒体的数据处理装置3可以包括：获取模块151、识别模块152以及解码模块153。
获取模块，用于获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；上述替换关系指示信息用于指示上述每两个播放组中具有替换关系的媒体轨道对，上述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
识别模块，用于根据上述替换关系指示信息和上述替换属性信息，从上述至少两个播放组中识别出与播放终端匹配的目标播放组；
解码模块，用于对上述目标播放组进行解码处理，得到上述容积媒体。
可选的，上述获取模块获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
接收服务设备发送的上述容积媒体的传输协议；
对上述传输协议进行解析，得到上述替换关系指示信息和上述替换属性信息。
可选的，上述识别模块根据上述替换关系指示信息和上述替换属性信息，从上述至少两个播放组中识别出与播放终端匹配的目标播放组；包括：
根据上述替换关系指示信息和上述替换属性信息，从上述至少两个播放组中确定出与播放终端匹配的候选播放组的分组标识；
生成携带上述候选播放分组的分组标识的获取请求，向上述服务设备发送上述获取请求；
接收上述服务设备响应上述获取请求所返回的候选播放组；
将上述候选播放组作为与上述播放终端匹配的目标播放组。
可选的，上述识别模块根据上述替换关系指示信息和上述替换属性信息，从上述至少两个播放组中确定出与播放终端匹配的候选播放组的分组标识，包括：
获取上述播放终端的属性信息，根据上述播放终端的属性信息确定上述播放终端处理容积媒体的处理性能等级；
根据上述替换关系指示信息和上述替换属性信息，确定处理上述至少两个播放组中各个播放组的处理难度等级；上述处理难度等级是根据处理上述各个播放组所消耗的处理资源量确定的；
将上述至少两个播放组中处理难度等级与上述播放终端的处理性能等级匹配的播放组，作为候选播放组，获取上述候选播放组的分组标识。
可选的，上述识别模块根据上述替换关系指示信息和上述替换属性信息，从上述至少两个播放组中确定出与播放终端匹配的候选播放组的分组标识，包括：
获取上述播放终端历史所播放的容积媒体的第一质量等级；
根据上述替换关系指示信息和上述替换属性信息，确定上述至少两个播放组中各个播放组的第二质量等级；
将上述至少两个播放组中第二质量等级与上述第一质量等级匹配的播放组，作为候选播放组，获取上述候选播放组的分组标识。
可选的，上述获取模块对上述传输协议进行解析，得到上述替换关系指示信息和上述替换属性信息，包括：
当上述传输协议为动态自适应流媒体传输协议时，上述动态自适应流媒体传输协议的轨道描述字段中读取上述替换关系指示信息和上述替换属性信息。
可选的，上述获取模块对上述传输协议进行解析，得到上述替换关系指示信息和上述替换属性信息，包括：
当上述传输协议为动态图像专家组媒体传输协议时，从上述动态图像专家组媒体传输协议的资源组内读取上述替换关系指示信息和上述替换属性信息。
可选的，上述获取模块获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
从服务设备中获取上述容积媒体的数据盒；
对上述容积媒体的数据盒进行解封装处理，得到上述至少两个播放组、上述至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息。
可选的，上述解码模块对上述目标播放组进行解码处理，得到上述容积媒体，播放上述容积媒体，包括：
对上述目标播放组进行解码处理，得到上述容积媒体的媒体内容；
对媒体内容进行时间同步，得到上述容积媒体，播放上述容积媒体。
可选的，容积媒体的媒体内容包括音频内容、视频内容、以及文本内容中的至少一种。
根据本申请的一个实施例，图10所示的容积媒体的数据处理方法所涉及的步骤可由图15所示的容积媒体的数据处理装置中的各个模块来执行。例如，图10中所示的步骤S301可由图15中的获取模块151来执行，图10中所示的步骤S302可由图15中的识别模块152来执行；图10中所示的步骤S303可由图15中的解码模块153来执行。
根据本申请的一个实施例，图15所示的容积媒体的数据处理装置3的各个模块可以分别或全部合并为一个或若干个单元来构成，或者其中的某个(些)单元还可以再拆分为功能上更小的多个子单元，可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述模块是基于逻辑功能划分的，在实际应用中，一个模块的功能也可以由多个单元来实现，或者多个模块的功能由一个单元实现。在本申请的其它实施例中，容积媒体的数据处理装置也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。
根据本申请的一个实施例，可以通过在包括中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的例如计算机的通用计算机设备上运行能够执行如图10中所示的相应方法所涉及的各步骤的计算机可读指令(包括程序代码)，来构造如图15中所示的容积媒体的数据处理装置，以及来实现本申请实施例的容积媒体的数据处理方法。上述计算机可读指令可以记载于例如计算机可读记录介质上，并通过计算机可读记录介质装载于上述计算设备中，并在其中运行。
本申请中，播放终端通过根据替换关系指示信息和替换属性信息，从至少两个播放组中动态自适应地选择与其匹配的目标播放组，只需要对目标播放组进行解码操作，即可得到容积媒体，不需要对所有的播放组进行解码操作，可以节省播放终端的处理资源。
请参见图16，是本申请实施例提供的一种计算机设备的结构示意图。如图16所示，上述计算机设备1000可以包括：处理器1001，网络接口1004和存储器1005，此外，上述计算机设备1000还可以包括：用户接口1003，和至少一个通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。其中，用户接口1003可以包括显示屏(Display)、键盘(Keyboard)，可选用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可选的可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器，也可以是非易失性的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1005可选的还可以是至少一个位于远离前述处理器1001的存储装置。如图16所示，作为一个或多个计算机可读存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及设备控制应用程序。
在图16所示的计算机设备1000中，网络接口1004可提供网络通讯功能；而用户接口1003主要 用于为用户提供输入的接口；而处理器1001可以用于调用存储器1005中存储的设备控制应用程序，以实现本申请各实施例的方法的步骤。
应当理解，本申请实施例中所描述的计算机设备1000可执行前文图6、图9以及前文图10所对应实施例中对上述容积媒体的数据处理方法的描述，也可执行前文图13-图15所对应实施例中对上述容积媒体的数据处理装置的描述。。
此外，这里需要指出的是：本申请实施例还提供了一个或多个计算机可读存储介质，且上述计算机可读存储介质中存储有前文提及的容积媒体的数据处理装置所执行的计算机可读指令，且上述计算机可读指令包括程序指令，当上述处理器执行上述程序指令时，能够执行前文图6、图9以及前文图10对应实施例中对上述容积媒体的数据处理方法的描述，因此，这里将不再进行赘述。。对于本申请所涉及的计算机可读存储介质实施例中未披露的技术细节，请参照本申请方法实施例的描述。
作为示例，上述程序指令可被部署在一个计算机设备上执行，或者被部署位于一个地点的多个计算机设备上执行，又或者，在分布在多个地点且通过通信网络互连的多个计算机设备上执行，分布在多个地点且通过通信网络互连的多个计算机设备可以组成区块链网络。
本申请实施例还提供了一种计算机程序产品，包括计算机可读指令，计算机可读指令被一个或多个处理器执行时实现本申请各实施例中方法的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，上述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，上述的存储介质可为磁盘、光盘、只读存储器(Read-Only Memory，ROM)或随机存储器(Random Access Memory，RAM)等。
以上所揭露的仅为本申请较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (38)

  1. 一种容积媒体的数据处理方法，其特征在于，由服务设备执行，包括：
    获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；所述替换关系指示信息用于指示所述每两个播放组中具有替换关系的媒体轨道对，所述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
    将所述替换关系指示信息和所述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
  2. 如权利要求1所述的方法，其特征在于，所述获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
    获取所述容积媒体的数据盒；所述容积媒体的数据盒中包括所述每两个播放组之间的替换关系指示信息和替换属性信息。
  3. 如权利要求1所述的方法，其特征在于，所述获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
    获取所述容积媒体的传输协议，所述传输协议中包括所述每两个播放组之间的替换关系指示信息和替换属性信息。
  4. 如权利要求3所述的方法，其特征在于，当所述传输协议为动态自适应流媒体传输协议时，所述每两个播放组之间的替换关系指示信息和替换属性信息位于所述动态自适应流媒体传输协议的轨道描述字段中。
  5. 如权利要求3所述的方法，其特征在于，当所述传输协议为动态图像专家组媒体传输协议时，所述每两个播放组之间的替换关系指示信息和替换属性信息位于所述动态图像专家组媒体传输协议的资源组内。
  6. 如权利要求1所述的方法，其特征在于，所述媒体轨道对包括属于播放组P m的第一媒体轨道和属于播放组P n的第二媒体轨道，所述播放组P m与所述播放组P n之间的替换属性信息包括所述第一媒体轨道的属性信息，以及所述第二媒体轨道的属性信息，m和n均为小于Z的正整数，Z为所述至少两个播放组的分组数量。
  7. 如权利要求6所述的方法，其特征在于，所述第一媒体轨道的属性信息包括所述第一媒体轨道的编码方式，所述第二媒体轨道的属性信息包括所述第二媒体轨道的编码方式，所述第一媒体轨道的编码方式与所述第二媒体轨道的编码方式不相同。
  8. 如权利要求7所述的方法，其特征在于，所述第一媒体轨道的属性信息还包括所述第一媒体轨道的传输速率、结构组织、质量等级和编码方式对应的编码类型中的至少一种；
    所述第二媒体轨道的属性信息还包括所述第二媒体轨道的传输速率、结构组织、质量等级和编码方式对应的编码类型中的至少一种。
  9. 如权利要求6所述的方法，其特征在于，所述替换关系指示信息包括替换级别字段；当所述播放组P m和所述播放组P n之间的替换关系指示信息中的替换级别字段为第一数值时，所述替换级别字段用于反映所述播放组P m与所述播放组P n之间的替换级别为媒体内容级别。
  10. 如权利要求6所述的方法，其特征在于，所述替换关系指示信息包括替换级别字段；当所述播放组P m和所述播放组P n的替换关系指示信息中的替换级别字段为第二数值时，所述替换级别字段用于反映所述播放组P m与所述播放组P n之间的替换级别为轨道级别。
  11. 如权利要求9所述的方法，其特征在于，当所述播放组P m与所述播放组P n之间的替换级别为媒体内容级别时，所述播放组P m与所述播放组P n中具有替换关系的媒体轨道对的数量为R，R为所述播放组P m中的媒体轨道数量，所述播放组P m中的媒体轨道数量与所述播放组P n的轨道数量相同。
  12. 如权利要求10所述的方法，其特征在于，当所述播放组P m与所述播放组P n之间的替换级别为轨道级别时，所述播放组P m与所述播放组P n中具有替换关系的媒体轨道对的数量为小于R的正整数，R为所述播放组P m中的媒体轨道数量，所述播放组P m中的媒体轨道数量与所述播放组P n的轨道数量相 同。
  13. 如权利要求9所述的方法，其特征在于，当所述播放组P m与所述播放组P n之间的替换级别为媒体内容级别时，所述播放组P m中的媒体轨道的编码方式均为第一编码方式；所述播放组P n中的媒体轨道的编码方式均为第二编码方式。
  14. 如权利要求1所述的方法，其特征在于，所述至少两个播放组中各个播放组内的媒体轨道之间具有联合播放关系，所述各个播放组内的媒体轨道之间的轨道类别不相同；
    所述每两个播放组中具有替换关系的媒体轨道对中的媒体轨道之间的轨道类别相同。
  15. 如权利要求1所述的方法，其特征在于，所述替换关系指示信息包括所述每两个播放组中具有替换关系的媒体轨道对的替换标识。
  16. 一种容积媒体的数据处理方法，其特征在于，由服务设备执行，包括：
    获取容积媒体的至少两个播放组；
    对所述至少两个播放组中的每两个播放组进行标记，得到所述每两个播放组之间的替换关系指示信息和替换属性信息；所述替换关系指示信息用于指示所述每两个播放组中具有替换关系的媒体轨道对，所述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
    将所述替换关系指示信息和所述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
  17. 如权利要求16所述的方法，其特征在于，所述获取容积媒体的至少两个播放组，包括：
    获取容积媒体的至少两个媒体轨道，以及所述至少两个媒体轨道中的各个媒体轨道的轨道类别；
    根据所述各个媒体轨道的轨道类别，对所述至少两个媒体轨道进行划分，得到至少两个播放组；所述至少两个播放组中各个播放组内的媒体轨道之间具有联合播放关系，所述各个播放组内的媒体轨道之间的轨道类别不相同。
  18. 如权利要求16所述的方法，其特征在于，所述对所述至少两个播放组中的每两个播放组进行标记，得到所述每两个播放组之间的替换关系指示信息和替换属性信息，包括：
    根据播放组P m的媒体轨道的轨道类别与播放组P n中媒体轨道的轨道类别，确定候选媒体轨道对；所述候选媒体轨道对中的媒体轨道之间具有相同的轨道类别，m和n均为小于Z的正整数，Z为所述至少两个播放组的分组数量；
    获取所述候选媒体轨道对中的媒体轨道的属性信息；
    对所述候选媒体轨道对中的媒体轨道的属性信息之间进行差异识别，得到差异识别结果；
    若所述差异识别结果指示所述候选媒体轨道对中的媒体轨道之间的属性信息具有差异，则将所述候选媒体轨道对作为具有替换关系的媒体轨道对，生成所述媒体轨道对的替换标识；
    将所述替换标识作为所述播放组P m和所述播放组P n之间的替换关系指示信息；
    将所述候选媒体轨道对中的媒体轨道的属性信息作为所述播放组P m和所述播放组P n之间的替换属性信息。
  19. 如权利要求16-18任一项所述的方法，其特征在于，所述将所述替换关系指示信息和所述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体，包括：
    对所述至少两个播放组、所述每两个播放组之间的替换关系指示信息和替换属性信息进行封装，得到所述容积媒体的数据盒；
    将包括所述至少两个播放组、所述替换关系指示信息和所述替换属性信息的数据盒发送至所述播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
  20. 如权利要求16-18任一项所述的方法，其特征在于，所述将所述替换关系指示信息和所述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息， 从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体，包括：
    获取所述容积媒体的传输协议；
    将所述每两个播放组之间的替换关系指示信息和替换属性信息添加至所述传输协议中；
    将包括所述替换关系指示信息和所述替换属性信息的传输协议发送至播放终端；
    接收所述播放终端发送的、且根据所述传输协议中的所述替换关系指示信息和所述替换属性信息生成的关于目标播放组的获取请求，所述目标播放组为所述至少两个播放组中与所述播放终端匹配的播放组；
    响应于所述获取请求，将所述目标播放组发送至所述播放终端，以指示所述播放终端对所述目标播放组进行解码处理，得到所述容积媒体。
  21. 如权利要求20所述的方法，其特征在于，所述将所述每两个播放组之间的替换关系指示信息和替换属性信息添加至所述传输协议中，包括：
    当所述传输协议为动态自适应流媒体传输协议时，将所述每两个播放组之间的替换关系指示信息和替换属性信息添加至所述动态自适应流媒体传输协议的轨道描述字段中。
  22. 如权利要求20所述的方法，其特征在于，所述将所述每两个播放组之间的替换关系指示信息和替换属性信息添加至所述传输协议中，包括：
    当所述传输协议为动态图像专家组媒体传输协议时，将所述每两个播放组之间的替换关系指示信息和替换属性信息添加至所述动态图像专家组媒体传输协议的资源组内。
  23. 一种容积媒体的数据处理方法，其特征在于，由播放终端执行，包括：
    获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；所述替换关系指示信息用于指示所述每两个播放组中具有替换关系的媒体轨道对，所述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
    根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与播放终端匹配的目标播放组；
    对所述目标播放组进行解码处理，得到所述容积媒体。
  24. 如权利要求23所述的方法，其特征在于，所述获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
    接收服务设备发送的所述容积媒体的传输协议；
    对所述传输协议进行解析，得到所述替换关系指示信息和所述替换属性信息。
  25. 如权利要求23所述的方法，其特征在于，所述根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与播放终端匹配的目标播放组；包括：
    根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中确定出与播放终端匹配的候选播放组的分组标识；
    生成携带所述候选播放分组的分组标识的获取请求，向所述服务设备发送所述获取请求；
    接收所述服务设备响应所述获取请求所返回的候选播放组；
    将所述候选播放组作为与所述播放终端匹配的目标播放组。
  26. 如权利要求25所述的方法，其特征在于，所述根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中确定出与播放终端匹配的候选播放组的分组标识，包括：
    获取所述播放终端的属性信息，根据所述播放终端的属性信息确定所述播放终端处理容积媒体的处理性能等级；
    根据所述替换关系指示信息和所述替换属性信息，确定处理所述至少两个播放组中各个播放组的处理难度等级；所述处理难度等级是根据处理所述各个播放组所消耗的处理资源量确定的；
    将所述至少两个播放组中处理难度等级与所述播放终端的处理性能等级匹配的播放组，作为候选播放组，获取所述候选播放组的分组标识。
  27. 如权利要求25所述的方法，其特征在于，所述根据所述替换关系指示信息和所述替换属性 信息，从所述至少两个播放组中确定出与播放终端匹配的候选播放组的分组标识，包括：
    获取所述播放终端历史所播放的容积媒体的第一质量等级；
    根据所述替换关系指示信息和所述替换属性信息，确定所述至少两个播放组中各个播放组的第二质量等级；
    将所述至少两个播放组中第二质量等级与所述第一质量等级匹配的播放组，作为候选播放组，获取所述候选播放组的分组标识。
  28. 如权利要求24所述的方法，其特征在于，所述对所述传输协议进行解析，得到所述替换关系指示信息和所述替换属性信息，包括：
    当所述传输协议为动态自适应流媒体传输协议时，所述动态自适应流媒体传输协议的轨道描述字段中读取所述替换关系指示信息和所述替换属性信息。
  29. 如权利要求24所述的方法，其特征在于，所述对所述传输协议进行解析，得到所述替换关系指示信息和所述替换属性信息，包括：
    当所述传输协议为动态图像专家组媒体传输协议时，从所述动态图像专家组媒体传输协议的资源组内读取所述替换关系指示信息和所述替换属性信息。
  30. 如权利要求23所述的方法，其特征在于，所述获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息，包括：
    从服务设备中获取所述容积媒体的数据盒；
    对所述容积媒体的数据盒进行解封装处理，得到所述至少两个播放组、所述至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息。
  31. 如权利要求23所述的方法，其特征在于，所述对所述目标播放组进行解码处理，得到所述容积媒体，包括：
    对所述目标播放组进行解码处理，得到所述容积媒体的媒体内容；
    对所述媒体内容进行时间同步，得到所述容积媒体。
  32. 如权利要求31所述的方法，其特征在于，所述容积媒体的媒体内容包括音频内容、视频内容、以及文本内容中的至少一种。
  33. 一种容积媒体的数据处理装置，其特征在于，包括：
    获取模块，用于获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；所述替换关系指示信息用于指示所述每两个播放组中具有替换关系的媒体轨道对，所述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
    发送模块，用于将所述替换关系指示信息和所述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
  34. 一种容积媒体的数据处理装置，其特征在于，包括：
    获取模块，用于获取容积媒体的至少两个播放组；
    标记模块，用于对所述至少两个播放组中的每两个播放组进行标记，得到所述每两个播放组之间的替换关系指示信息和替换属性信息；所述替换关系指示信息用于指示所述每两个播放组中具有替换关系的媒体轨道对，所述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
    发送模块，用于将所述替换关系指示信息和所述替换属性信息发送至播放终端，以指示所述播放终端根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与所述播放终端匹配的目标播放组，对所述目标播放组进行解码处理，得到所述容积媒体。
  35. 一种容积媒体的数据处理装置，其特征在于，包括：
    获取模块，用于获取容积媒体的至少两个播放组中的每两个播放组之间的替换关系指示信息和替换属性信息；所述替换关系指示信息用于指示所述每两个播放组中具有替换关系的媒体轨道对，所述替换属性信息用于反映具有替换关系的媒体轨道对中的媒体轨道之间的属性差异；
    识别模块，用于根据所述替换关系指示信息和所述替换属性信息，从所述至少两个播放组中识别出与播放终端匹配的目标播放组；
    解码模块，用于对所述目标播放组进行解码处理，得到所述容积媒体。
  36. 一种计算机设备，其特征在于，包括：
    一个或多个处理器以及存储器；
    所述处理器与所述存储器相连，其中，所述存储器用于存储程序代码，所述处理器用于调用所述程序代码，以执行如权利要求1-32任一项所述的方法。
  37. 一个或多个计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机可读指令，该计算机可读指令适于由处理器加载并执行如权利要求1-32任一项所述的方法。
  38. 一种计算机程序产品，包括计算机可读指令，其特征在于，所述计算机可读指令被一个或多个处理器执行时实现权利要求1-32任一项所述的方法的步骤。
PCT/CN2022/099706 2021-07-12 2022-06-20 容积媒体的数据处理方法、装置、设备以及存储介质 WO2023284487A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/224,165 US20230360678A1 (en) 2021-07-12 2023-07-20 Data processing method and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110785828.2A CN115623183A (zh) 2021-07-12 2021-07-12 容积媒体的数据处理方法、装置、设备以及存储介质
CN202110785828.2 2021-07-12

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/224,165 Continuation US20230360678A1 (en) 2021-07-12 2023-07-20 Data processing method and storage medium

Publications (1)

Publication Number Publication Date
WO2023284487A1 true WO2023284487A1 (zh) 2023-01-19

Family

ID=84855981

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/099706 WO2023284487A1 (zh) 2021-07-12 2022-06-20 容积媒体的数据处理方法、装置、设备以及存储介质

Country Status (3)

Country Link
US (1) US20230360678A1 (zh)
CN (1) CN115623183A (zh)
WO (1) WO2023284487A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116347118A (zh) * 2023-03-07 2023-06-27 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法及相关设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102388613A (zh) * 2009-04-09 2012-03-21 瑞典爱立信有限公司 媒体容器文件管理
CN112804256A (zh) * 2021-02-09 2021-05-14 腾讯科技（深圳）有限公司 多媒体文件中轨道数据的处理方法、装置、介质及设备
CN112887733A (zh) * 2021-01-25 2021-06-01 中兴通讯股份有限公司 一种容积媒体处理方法、装置、存储介质及电子装置
US20210209807A1 (en) * 2020-01-08 2021-07-08 Lg Electronics Inc. Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102388613A (zh) * 2009-04-09 2012-03-21 瑞典爱立信有限公司 媒体容器文件管理
US20210209807A1 (en) * 2020-01-08 2021-07-08 Lg Electronics Inc. Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method
CN112887733A (zh) * 2021-01-25 2021-06-01 中兴通讯股份有限公司 一种容积媒体处理方法、装置、存储介质及电子装置
CN112804256A (zh) * 2021-02-09 2021-05-14 腾讯科技（深圳）有限公司 多媒体文件中轨道数据的处理方法、装置、介质及设备

Also Published As

Publication number Publication date
US20230360678A1 (en) 2023-11-09
CN115623183A (zh) 2023-01-17

Similar Documents

Publication Publication Date Title
CN109691094B (zh) 发送全向视频的方法、接收全向视频的方法、发送全向视频的装置和接收全向视频的装置
KR102246002B1 (ko) 가상 현실 미디어 콘텐트의 스트리밍을 개선하는 방법, 디바이스, 및 컴퓨터 프로그램
CN110876051B (zh) 视频数据的处理，传输方法及装置，视频数据的处理系统
CN111149368A (zh) 用于浸入式媒体数据的内容来源描述
CN109644262A (zh) 发送全向视频的方法、接收全向视频的方法、发送全向视频的装置和接收全向视频的装置
WO2019202207A1 (en) Processing video patches for three-dimensional content
WO2018068236A1 (zh) 一种视频流传输方法、相关设备及系统
JP2020503792A (ja) 情報処理方法および装置
CN113891117B (zh) 沉浸媒体的数据处理方法、装置、设备及可读存储介质
CN111869222B (zh) 基于http的dash客户端网元、方法及介质
WO2021190221A1 (zh) 沉浸式媒体提供方法、获取方法、装置、设备及存储介质
US12107908B2 (en) Media file encapsulating method, media file decapsulating method, and related devices
CN113852829A (zh) 点云媒体文件的封装与解封装方法、装置及存储介质
CN115396647B (zh) 一种沉浸媒体的数据处理方法、装置、设备及存储介质
WO2023061131A1 (zh) 媒体文件封装方法、装置、设备及存储介质
US20230360678A1 (en) Data processing method and storage medium
CN110351492B (zh) 一种视频数据处理方法、装置及介质
CN110999312B (zh) 发送360度视频的方法、接收360度视频的方法、发送360度视频的装置和接收360度视频的装置
CN115733576B (zh) 点云媒体文件的封装与解封装方法、装置及存储介质
US20230421819A1 (en) Media file unpacking method and apparatus, device, and storage medium
CN116137664A (zh) 点云媒体文件封装方法、装置、设备及存储介质
CN118200634A (zh) 一种三维点云的传输方法、传输装置、接收方法和接收装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22841127

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17.05.2024)