WO2021125117A1 - Systèmes et procédés pour signaler des informations pour un mesh dans un support omnidirectionnel - Google Patents

Systèmes et procédés pour signaler des informations pour un mesh dans un support omnidirectionnel Download PDF

Info

Publication number
WO2021125117A1
WO2021125117A1 PCT/JP2020/046443 JP2020046443W WO2021125117A1 WO 2021125117 A1 WO2021125117 A1 WO 2021125117A1 JP 2020046443 W JP2020046443 W JP 2020046443W WO 2021125117 A1 WO2021125117 A1 WO 2021125117A1
Authority
WO
WIPO (PCT)
Prior art keywords
mesh
syntax element
box
specifies
eye
Prior art date
Application number
PCT/JP2020/046443
Other languages
English (en)
Inventor
Sachin G. Deshpande
Original Assignee
Sharp Kabushiki Kaisha
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sharp Kabushiki Kaisha filed Critical Sharp Kabushiki Kaisha
Publication of WO2021125117A1 publication Critical patent/WO2021125117A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/23614—Multiplexing of additional data and video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2362—Generation or processing of Service Information [SI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365—Multiplexing of several video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/85406—Content authoring involving a specific file format, e.g. MP4 format

Definitions

  • This disclosure relates to the field of interactive video distribution and more particularly to techniques for signaling information for a mesh in a virtual reality application.
  • Digital media playback capabilities may be incorporated into a wide range of devices, including digital televisions, including so-called “smart” televisions, set-top boxes, laptop or desktop computers, tablet computers, digital recording devices, digital media players, video gaming devices, cellular phones, including so-called “smart” phones, dedicated video streaming devices, and the like.
  • Digital media content (e.g., video and audio programming) may originate from a plurality of sources including, for example, over-the-air television providers, satellite television providers, cable television providers, online media service providers, including, so-called streaming service providers, and the like.
  • Digital media content may be delivered over packet-switched networks, including bidirectional networks, such as Internet Protocol (IP) networks and unidirectional networks, such as digital broadcast networks.
  • IP Internet Protocol
  • Digital video included in digital media content may be coded according to a video coding standard.
  • Video coding standards may incorporate video compression techniques. Examples of video coding standards include ISO/IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC) and High-Efficiency Video Coding (HEVC).
  • Video compression techniques enable data requirements for storing and transmitting video data to be reduced. Video compression techniques may reduce data requirements by exploiting the inherent redundancies in a video sequence.
  • Video compression techniques may sub-divide a video sequence into successively smaller portions (i.e., groups of frames within a video sequence, a frame within a group of frames, slices within a frame, coding tree units (e.g., macroblocks) within a slice, coding blocks within a coding tree unit, etc.).
  • Prediction coding techniques may be used to generate difference values between a unit of video data to be coded and a reference unit of video data. The difference values may be referred to as residual data.
  • Residual data may be coded as quantized transform coefficients.
  • Syntax elements may relate residual data and a reference coding unit. Residual data and syntax elements may be included in a compliant bitstream. Compliant bitstreams and associated metadata may be formatted according to data structures.
  • Compliant bitstreams and associated metadata may be transmitted from a source to a receiver device (e.g., a digital television or a smart phone) according to a transmission standard.
  • a transmission standard include Digital Video Broadcasting (DVB) standards, Integrated Services Digital Broadcasting Standards (ISDB) standards, and standards developed by the Advanced Television Systems Committee (ATSC), including, for example, the ATSC 2.0 standard.
  • the ATSC is currently developing the so-called ATSC 3.0 suite of standards.
  • a method of determining mesh information associated with an omnidirectional video comprises: receiving a mesh box specifying a three dimensional mesh consisting of one or more mesh element; parsing a number of mesh elements syntax element specifying a number of mesh elements described in the mesh box; parsing an explicit mesh identifier syntax element indicating whether a mesh element identifier syntax element is present in the mesh box; and parsing the mesh element identifier syntax element, which specifies an identifier to uniquely identify a mesh element within the mesh box, by using a value of the explicit mesh identifier syntax element for the number of mesh elements.
  • a method of signaling mesh information associated with an omnidirectional video comprises: signaling a number of mesh elements syntax element specifying a number of mesh elements described in a mesh box; signaling an explicit mesh identifier syntax element indicating whether a mesh element identifier syntax element is present in the mesh box; and signaling the mesh element identifier syntax element, which specifies an identifier to uniquely identify a mesh element within the mesh box, by using a value of the explicit mesh identifier syntax element for the number of mesh elements, wherein the mesh box specifies a three dimensional mesh consisting of one or more mesh element.
  • a device comprises one or more processors configured to: receive a mesh box specifying a three dimensional mesh consisting of one or more mesh element; parse a number of mesh elements syntax element specifying a number of mesh elements described in the mesh box; parse an explicit mesh identifier syntax element indicating whether a mesh element identifier syntax element is present in the mesh box; and parse the mesh element identifier syntax element, which specifies an identifier to uniquely identify a mesh element within the mesh box, by using a value of the explicit mesh identifier syntax element for the number of mesh elements.
  • a device comprises one or more processors configured to: signal a number of mesh elements syntax element specifying a number of mesh elements described in a mesh box; signal an explicit mesh identifier syntax element indicating whether a mesh element identifier syntax element is present in the mesh box; and signal the mesh element identifier syntax element, which specifies an identifier to uniquely identify a mesh element within the mesh box, by using a value of the explicit mesh identifier syntax element for the number of mesh elements, wherein the mesh box specifies a three dimensional mesh consisting of one or more mesh element.
  • FIG. 1 is a block diagram illustrating an example of a system that may be configured to transmit coded video data according to one or more techniques of this disclosure.
  • FIG. 2A is a conceptual diagrams illustrating coded video data and corresponding data structures according to one or more techniques of this disclosure.
  • FIS. 2B is a conceptual diagrams illustrating coded video data and corresponding data structures according to one or more techniques of this disclosure.
  • FIG. 3 is a conceptual diagram illustrating coded video data and corresponding data structures according to one or more techniques of this disclosure.
  • FIG. 4 is a conceptual diagram illustrating an example of a coordinate system according to one or more techniques of this disclosure.
  • FIG. 5A is conceptual diagrams illustrating examples of specifying regions on a sphere according to one or more techniques of this disclosure.
  • FIG. 5B is conceptual diagrams illustrating examples of specifying regions on a sphere according to one or more techniques of this disclosure.
  • FIG. 6 is a conceptual drawing illustrating an example of parallelogram corresponding to a mesh according to one or more techniques of this disclosure.
  • FIG. 7 is a conceptual drawing illustrating an example of a rectangular region that may be mapped to a mesh according to one or more techniques of this disclosure.
  • FIG. 8 is a conceptual drawing illustrating an example of a rectangular region that may be mapped to a mesh according to one or more techniques of this disclosure.
  • FIG. 9 is a conceptual drawing illustrating an example of components that may be included in an implementation of a system that may be configured to transmit coded video data according to one or more techniques of this disclosure.
  • FIG. 10 is a block diagram illustrating an example of a data encapsulator that may implement one or more techniques of this disclosure.
  • FIG. 11 is a block diagram illustrating an example of a receiver device that may implement one or more techniques of this disclosure.
  • this disclosure describes various techniques for signaling a mesh. It should be noted that although in some examples, the techniques of this disclosure are described with respect to transmission standards, the techniques described herein may be generally applicable. For example, the techniques described herein are generally applicable to any of DVB standards, ISDB standards, ATSC Standards, Digital Terrestrial Multimedia Broadcast (DTMB) standards, Digital Multimedia Broadcast (DMB) standards, Hybrid Broadcast and Broadband Television (HbbTV) standards, World Wide Web Consortium (W3C) standards, and Universal Plug and Play (UPnP) standard.
  • DTMB Digital Terrestrial Multimedia Broadcast
  • DMB Digital Multimedia Broadcast
  • HbbTV Hybrid Broadcast and Broadband Television
  • W3C World Wide Web Consortium
  • UPC Universal Plug and Play
  • ITU-T H.264 and ITU-T H.265 are generally applicable to video coding, including omnidirectional video coding.
  • the coding techniques described herein may be incorporated into video coding systems, (including video coding systems based on future video coding standards) including block structures, intra prediction techniques, inter prediction techniques, transform techniques, filtering techniques, and/or entropy coding techniques other than those included in ITU-T H.265.
  • reference to ITU-T H.264 and ITU-T H.265 is for descriptive purposes and should not be construed to limit the scope of the techniques described herein.
  • Video content typically includes video sequences comprised of a series of frames.
  • a series of frames may also be referred to as a group of pictures (GOP).
  • Each video frame or picture may include a one or more slices, where a slice includes a plurality of video blocks.
  • a video block may be defined as the largest array of pixel values (also referred to as samples) that may be predictively coded.
  • Video blocks may be ordered according to a scan pattern (e.g., a raster scan).
  • a video encoder performs predictive encoding on video blocks and sub-divisions thereof.
  • ITU-T H.264 specifies a macroblock including 16 x 16 luma samples.
  • ITU-T H.265 specifies an analogous Coding Tree Unit (CTU) structure where a picture may be split into CTUs of equal size and each CTU may include Coding Tree Blocks (CTB) having 16 x 16, 32 x 32, or 64 x 64 luma samples.
  • CTU Coding Tree Block
  • the term video block may generally refer to an area of a picture or may more specifically refer to the largest array of pixel values that may be predictively coded, sub-divisions thereof, and/or corresponding structures.
  • each video frame or picture may be partitioned to include one or more tiles, where a tile is a sequence of coding tree units corresponding to a rectangular area of a picture.
  • the CTBs of a CTU may be partitioned into Coding Blocks (CB) according to a corresponding quadtree block structure.
  • CB Coding Blocks
  • one luma CB together with two corresponding chroma CBs and associated syntax elements are referred to as a coding unit (CU).
  • a CU is associated with a prediction unit (PU) structure defining one or more prediction units (PU) for the CU, where a PU is associated with corresponding reference samples.
  • PU prediction unit
  • PU prediction unit
  • a PU may include luma and chroma prediction blocks (PBs), where square PBs are supported for intra prediction and rectangular PBs are supported for inter prediction.
  • Intra prediction data e.g., intra prediction mode syntax elements
  • inter prediction data e.g., motion data syntax elements
  • Residual data may include respective arrays of difference values corresponding to each component of video data (e.g., luma (Y) and chroma (Cb and Cr)). Residual data may be in the pixel domain.
  • a transform such as, a discrete cosine transform (DCT), a discrete sine transform (DST), an integer transform, a wavelet transform, or a conceptually similar transform, may be applied to pixel difference values to generate transform coefficients.
  • DCT discrete cosine transform
  • DST discrete sine transform
  • an integer transform e.g., a wavelet transform, or a conceptually similar transform
  • CUs may be further sub-divided into Transform Units (TUs).
  • an array of pixel difference values may be sub-divided for purposes of generating transform coefficients (e.g., four 8 x 8 transforms may be applied to a 16 x 16 array of residual values corresponding to a 16 x16 luma CB), such sub-divisions may be referred to as Transform Blocks (TBs).
  • Transform coefficients may be quantized according to a quantization parameter (QP).
  • Quantized transform coefficients (which may be referred to as level values) may be entropy coded according to an entropy encoding technique (e.g., content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), etc.).
  • CAVLC content adaptive variable length coding
  • CABAC context adaptive binary arithmetic coding
  • PIPE probability interval partitioning entropy coding
  • syntax elements such as, a syntax element indicating a prediction mode, may also be entropy coded. Entropy encoded quantized transform coefficients and corresponding entropy encoded syntax elements may form a compliant bitstream that can be used to reproduce video data.
  • a binarization process may be performed on syntax elements as part of an entropy coding process. Binarization refers to the process of converting a syntax value into a series of one or more bits. These bits may be referred to as “bins.”
  • VR applications may include video content that may be rendered with a head-mounted display, where only the area of the spherical video that corresponds to the orientation of the user’s head is rendered.
  • VR applications may be enabled by omnidirectional video, which is also referred to as 360 degree spherical video of 360 degree video.
  • Omnidirectional video is typically captured by multiple cameras that cover up to 360 degrees of a scene.
  • a distinct feature of omnidirectional video compared to normal video is that, typically only a subset of the entire captured video region is displayed, i.e., the area corresponding to the current user’s field of view (FOV) is displayed.
  • a FOV is sometimes also referred to as viewport.
  • a viewport may be described as part of the spherical video that is currently displayed and viewed by the user. It should be noted that the size of the viewport can be smaller than or equal to the field of view. Further, it should be noted that omnidirectional video may be captured using monoscopic or stereoscopic cameras. Monoscopic cameras may include cameras that capture a single view of an object. Stereoscopic cameras may include cameras that capture multiple views of the same object (e.g., views are captured using two lenses at slightly different angles). It should be noted that in some cases, the center point of a viewport may be referred to as a viewpoint.
  • the term viewpoint when associated with a camera may refer to information associated with a camera used to capture a view(s) of an object (e.g., camera parameters).
  • images for use in omnidirectional video applications may be captured using ultra wide-angle lens (i.e., so-called fisheye lens).
  • the process for creating 360 degree spherical video may be generally described as stitching together input images and projecting the stitched together input images onto a three-dimensional structure (e.g., a sphere or cube), which may result in so-called projected frames.
  • regions of projected frames may be transformed, resized, and relocated, which may result in a so-called packed frame.
  • Transmission systems may be configured to transmit omnidirectional video to one or more computing devices.
  • Computing devices and/or transmission systems may be based on models including one or more abstraction layers, where data at each abstraction layer is represented according to particular structures, e.g., packet structures, modulation schemes, etc.
  • An example of a model including defined abstraction layers is the so-called Open Systems Interconnection (OSI) model.
  • the OSI model defines a 7-layer stack model, including an application layer, a presentation layer, a session layer, a transport layer, a network layer, a data link layer, and a physical layer. It should be noted that the use of the terms upper and lower with respect to describing the layers in a stack model may be based on the application layer being the uppermost layer and the physical layer being the lowermost layer.
  • Layer 1 may be used to refer to a physical layer
  • Layer 2 may be used to refer to a link layer
  • Layer 3 or “L3” or “IP layer” may be used to refer to the network layer.
  • a physical layer may generally refer to a layer at which electrical signals form digital data.
  • a physical layer may refer to a layer that defines how modulated radio frequency (RF) symbols form a frame of digital data.
  • RF radio frequency
  • a data link layer which may also be referred to as a link layer, may refer to an abstraction used prior to physical layer processing at a sending side and after physical layer reception at a receiving side.
  • a link layer may refer to an abstraction used to transport data from a network layer to a physical layer at a sending side and used to transport data from a physical layer to a network layer at a receiving side.
  • a sending side and a receiving side are logical roles and a single device may operate as both a sending side in one instance and as a receiving side in another instance.
  • a link layer may abstract various types of data (e.g., video, audio, or application files) encapsulated in particular packet types (e.g., Motion Picture Expert Group - Transport Stream (MPEG-TS) packets, Internet Protocol Version 4 (IPv4) packets, etc.) into a single generic format for processing by a physical layer.
  • MPEG-TS Motion Picture Expert Group - Transport Stream
  • IPv4 Internet Protocol Version 4
  • a network layer may generally refer to a layer at which logical addressing occurs.
  • a network layer may generally provide addressing information (e.g., Internet Protocol (IP) addresses) such that data packets can be delivered to a particular node (e.g., a computing device) within a network.
  • IP Internet Protocol
  • the term network layer may refer to a layer above a link layer and/or a layer having data in a structure such that it may be received for link layer processing.
  • Each of a transport layer, a session layer, a presentation layer, and an application layer may define how data is delivered for use by a user application.
  • W18865 “Text of ISO/IEC CD 23090-2 2nd edition OMAF,” October 2019, Geneva, CH, is incorporated by reference and referred to herein as W18865 defines a media application format that enables omnidirectional media applications.
  • W18865 specifies a coordinate system for omnidirectional video; projection and rectangular region-wise packing methods that may be used for conversion of a spherical video sequence or image into a two-dimensional rectangular video sequence or image, respectively; storage of omnidirectional media and the associated metadata using the ISO Base Media File Format (ISOBMFF); encapsulation, signaling, and streaming of omnidirectional media in a media streaming system; and media profiles and presentation profiles.
  • ISOBMFF ISO Base Media File Format
  • W18865 provides media profiles where video is coded according to ITU-T H.265.
  • ITU-T H.265 is described in High Efficiency Video Coding (HEVC), Rec. ITU-T H.265 December 2016, which is incorporated by reference, and referred to herein as ITU-T H.265.
  • HEVC High Efficiency Video Coding
  • each video frame or picture may be partitioned to include one or more slices and further partitioned to include one or more tiles.
  • FIGS. 2A-2B are conceptual diagrams illustrating an example of a group of pictures including slices and further partitioning pictures into tiles. In the example illustrated in FIG.
  • Pic 4 is illustrated as including two slices (i.e., Slice 1 and Slice 2 ) where each slice includes a sequence of CTUs (e.g., in raster scan order).
  • Pic 4 is illustrated as including six tiles (i.e., Tile 1 to Tile 6 ), where each tile is rectangular and includes a sequence of CTUs.
  • a tile may consist of coding tree units contained in more than one slice and a slice may consist of coding tree units contained in more than one tile.
  • ITU-T H.265 provides that one or both of the following conditions shall be fulfilled: (1) All coding tree units in a slice belong to the same tile; and (2) All coding tree units in a tile belong to the same slice.
  • 360 degree spherical video may include regions.
  • the 360 degree spherical video includes Regions A, B, and C and as illustrated in FIG. 3, tiles (i.e., Tile 1 to Tile 6 ) may form a region of an omnidirectional video.
  • tiles i.e., Tile 1 to Tile 6
  • each of the regions are illustrated as including CTUs.
  • CTUs may form slices of coded video data and/or tiles of video data.
  • video coding techniques may code areas of a picture according to video blocks, sub-divisions thereof, and/or corresponding structures and it should be noted that video coding techniques enable video coding parameters to be adjusted at various levels of a video coding structure, e.g., adjusted for slices, tiles, video blocks, and/or at sub-divisions.
  • the 360 degree video illustrated in FIG. 3 may represent a sporting event where Region A and Region C include views of the stands of a stadium and Regions B includes a view of the playing field (e.g., the video is captured by a 360 degree camera placed at the 50-yard line).
  • a viewport may be part of the spherical video that is currently displayed and viewed by the user.
  • regions of omnidirectional video may be selectively delivered depending on the user’s viewport, i.e., viewport-dependent delivery may be enabled in omnidirectional video streaming.
  • source content is split into sub-picture sequences before encoding, where each sub-picture sequence covers a subset of the spatial area of the omnidirectional video content, and sub-picture sequences are then encoded independently from each other as a single-layer bitstream.
  • each of Region A, Region B, and Region C, or portions thereof may correspond to independently coded sub-picture bitstreams.
  • Each sub-picture bitstream may be encapsulated in a file as its own track and tracks may be selectively delivered to a receiver device based on viewport information. It should be noted that in some cases, it is possible that sub-pictures overlap. For example, referring to FIG. 3, Tile 1 , Tile 2 , Tile 4 , and Tile 5 may form a sub-picture and Tile 2 , Tile 3 , Tile 5 , and Tile 6 may form a sub-picture. Thus, a particular sample may be included in multiple sub-pictures.
  • a composition-aligned sample includes one of a sample in a track that is associated with another track, the sample has the same composition time as a particular sample in the another track, or, when a sample with the same composition time is not available in the another track, the closest preceding composition time relative to that of a particular sample in the another track.
  • a constituent picture includes part of a spatially frame-packed stereoscopic picture that corresponds to one view, or a picture itself when frame packing is not in use or the temporal interleaving frame packing arrangement is in use.
  • W18865 specifies a coordinate system for omnidirectional video.
  • the coordinate system consists of a unit sphere and three coordinate axes, namely the X (back-to-front) axis, the Y (lateral, side-to-side) axis, and the Z (vertical, up) axis, where the three axes cross at the center of the sphere.
  • the location of a point on the sphere is identified by a pair of sphere coordinates azimuth ( ⁇ ) and elevation ( ⁇ ).
  • FIG. 4 illustrates the relation of the sphere coordinates azimuth ( ⁇ ) and elevation ( ⁇ ) to the X, Y, and Z coordinate axes as specified in W18865.
  • W18865 specifies where a region on a sphere may be specified by four great circles, where a great circle (also referred to as a Riemannian circle) is an intersection of the sphere and a plane that passes through the center point of the sphere, where the center of the sphere and the center of a great circle are co-located.
  • a great circle also referred to as a Riemannian circle
  • W18865 further describes where a region on a sphere may be specified by two azimuth circles and two elevation circles, where a azimuth circle is a circle on the sphere connecting all points with the same azimuth value, and an elevation circle is a circle on the sphere connecting all points with the same elevation value.
  • the sphere region structure in W18865 forms the basis for signaling various types of metadata.
  • a sphere region structure (SphereRegionStruct) specifies a sphere region.
  • the sphere region structure defines a sphere region by defining parameters centre_azimuth, centre_elevation, centre_tilt with optional inclusion of azimuth_range and elevation_range, (and optional inclusion of an interpolation indication).
  • the sphere region sample metadata in turn is used to define initial viewing orientation and recommended viewport information. Additionally, the sphere region structure forms the basis of DASH descriptor signaling for content coverage descriptor, spherical region-wise quality ranking descriptor. Similarly, the sphere region structure forms the basis of VRROIGuide MMT message.
  • unsigned int(n) refers to an unsigned integer having n-bits.
  • bit(n) refers to a bit value having n-bits.
  • W18865 forms the basis for signaling various types of metadata.
  • W18865 includes a sphere region structure specifying a sphere region having the following definition, syntax, and semantics:
  • the sphere region structure (SphereRegionStruct) specifies a sphere region.
  • the sphere region specified by this structure is derived as follows:
  • the sphere region is defined as follows with reference to the shape type value specified in the semantics of the structure containing this instance of SphereRegionStruct:
  • the sphere region is firstly derived as above and then a tilt rotation is applied along the axis originating from the sphere origin passing through the centre point of the sphere region, where the angle value increases clockwise when looking from the origin towards the positive end of the axis.
  • the final sphere region is the one after applying the tilt rotation.
  • Shape type value equal to 0 specifies that the sphere region is specified by four great circles as illustrated in FIG. 5A.
  • Shape type value 1 specifies that the sphere region is specified by two azimuth circles and two elevation circles as illustrated in FIG. 5B. Shape type values greater than 1 are reserved.
  • centre_azimuth and centre_elevation specify the centre of the sphere region. centre_azimuth shall be in the range of -180 * 2 16 to 180 * 2 16 - 1, inclusive.
  • centre_elevation shall be in the range of -90 * 2 16 to 90 * 2 16 , inclusive.
  • centre_tilt specifies the tilt angle of the sphere region. centre_tilt shall be in the range of -180 * 2 16 to 180 * 2 16 - 1, inclusive.
  • azimuth_range and elevation_range when present, specify the azimuth and elevation ranges, respectively, of the sphere region specified by this structure in units of 2 -16 degrees.
  • azimuth_range and elevation_range specify the range through the centre point of the sphere region, as illustrated by FIG. 5A or FIG. 5B.
  • azimuth_range and elevation_range are not present in this instance of SphereRegionStruct, they are inferred as specified in the semantics of the structure containing this instance of SphereRegionStruct.
  • azimuth_range shall be in the range of 0 to 360 * 2 16 , inclusive.
  • elevation_range shall be in the range of 0 to 180 * 2 16 , inclusive.
  • interpolate The semantics of interpolate are specified by the semantics of the structure containing this instance of SphereRegionStruct. When interpolate is not present in this instance of SphereRegionStruct, it is inferred as specified in the semantics of the syntax structure containing this instance of SphereRegionStruct.
  • a 3D object may be represented using a plurality of polygons (e.g., triangles or rectangles) or elements which cover the surface of the 3D object. Such a representation is referred to as a mesh or a 3D mesh.
  • W18865 provides where a 3D mesh consisting of one or more mesh objects may be signaled and used to map samples of video to locations of a 3D space.
  • W18865 includes a mesh box data structure having the following definition, syntax and semantic:
  • MeshBox specifies a 3D mesh consisting of one or more mesh elements, each of which have a unique id. The different mesh elements are referenced by one or more tile mesh group entries (e.g.
  • TileMeshGroupEntry which define the relationship between a group of samples (or a complete tile track) and the position on the mesh. This information can be used for both tile selection as well as during the rendering process. In line with how typical rendering engines work with stereoscopic content, the same mesh is assumed to be used for both eyes in case of stereoscopic content. In other words, it is not possible to define distinct left-eye and right-eye meshes.
  • the MeshBox also provides implicit content coverage information.
  • num_mesh_elements specifies the number of mesh elements described in this box.
  • mesh_type specifies the type of the mesh description used all mesh elements described in this box. mesh_type shall be equal to 0 or 1. Other values of mesh_type are reserved.
  • SphereRegionStruct specifies a spherical mesh as the spherical region specified in the structure as defined above.
  • the SphereRegionStruct shall be inferred to have shape type value equal to 1 (specified by two azimuth circles and two elevation circles).
  • 3DParallelogramStruct specifies a parallelogram mesh in the 3D space. Two mesh elements described in this box must not overlap.
  • mesh_element_id specifies an identifier to uniquely identify a mesh element within this mesh box.
  • a mesh in W18865 may be formed using 3D parallelogram structures.
  • W18865 provides the following for a 3D parallelogram structure:
  • a 3DParallelogramStruct is a parallelogram in the local reference frame (O, X, Y, Z), as depicted in FIG. 6.
  • the parallelogram is defined using a special vertex named origin and with two vectors u_direction and v_direction representing the two characteristic directions of the parallelogram, as depicted in FIG. 6.
  • origin is a vector used to specify the starting vertex of the parallelogram. If O is the origin of the local reference frame, then the starting vertex v is equal to O + origin.
  • u_direction is a vector useDd to indicate the u direction.
  • v_direction is a vector used to indicate the v direction.
  • This structure specifies a vector in a 3D cartesian coordinate system.
  • the vector coordinates are expressed in the local coordinate system.
  • x, y, and z represent, respectively, the x, y, and z cartesian coordinate of a 3D vector expressed in the local coordinate system.
  • a mesh may be used to map samples of video to locations of a 3D space.
  • W18865 provides the following with respect to the mapping of rectangular regions to a 3D mesh.
  • the TileMeshGroupEntry describes one or more rectangular regions within the track, and describes how they should be mapped onto one or more mesh elements described by the MeshBox associated with this track.
  • num_regions specifies the number of packed regions.
  • guard_band_flag specifies the existence of the GuardBand struct for all regions.
  • eye[i] specifies the eye associated with the i-th rectangular region. When set to 0, the region corresponds to the left eye; when set to 1, the region corresponds to the right eye. For non-stereoscopic content, eye[i] shall be set to 0.
  • mesh_element_id[i] specifies to which mesh element from the associated MeshBox the i-th rectangular region is mapped to.
  • the mesh_element_id[i] is associated with that MeshBox.
  • the mesh_element_id[i] is associated with the MeshBox in that tile base track.
  • RectRegionStruct specifies a rectangular region. The regions described by any two instances of RectRegionStruct in the same TileMeshGroupEntry shall not overlap.
  • GuardBand(i) specifies the guard bands around the i-th rectangular region.
  • a RectRegionStruct is used in TileMeshGroupEntry to specify a rectangular region that is mapped onto 3D mesh.
  • region_width, region_height, region_start_x, and region_start_y specify the width, height, the vertical offset the start point and the horizontal offset of the start point, respectively, of a packed region within the packed picture. If region_width is positive, the region continues to the right side of the start point, otherwise it continues to the left side. If the region_height is positive, the region continues below the start point, otherwise it continues above the start point.
  • FIG. 7 and FIG. 8 illustrate examples of rectangular regions when region_width and region_height are positive and negative, respectively.
  • W18865 further provides the following for the projection of a sample location onto a 3D mesh.
  • an OMAF player In order to for an OMAF player to be able to use the MeshBox and the TileMeshGroupEntry, it needs to be able to map sample locations from a decoded picture to positions on the 3D mesh.
  • This clause specifies mapping of a sample location from a decoded picture to a position on the 3D mesh.
  • W18865 specifies encapsulation, signaling, and streaming of omnidirectional media in a media streaming system.
  • W18865 specifies how to encapsulate, signal, and stream omnidirectional media using dynamic adaptive streaming over Hypertext Transfer Protocol (HTTP) (DASH).
  • DASH is described in ISO/IEC: ISO/IEC 23009-1:2014, “Information technology - Dynamic adaptive streaming over HTTP (DASH) - Part 1: Media presentation description and segment formats,” International Organization for Standardization, 2nd Edition, 5/15/2014 (hereinafter, “ISO/IEC 23009-1:2014”), which is incorporated by reference herein.
  • a DASH media presentation may include data segments, video segments, and audio segments.
  • a DASH Media Presentation may correspond to a linear service or part of a linear service of a given duration defined by a service provider (e.g., a single TV program, or the set of contiguous linear TV programs over a period of time).
  • a Media Presentation Description is a document that includes metadata required by a DASH Client to construct appropriate HTTP-URLs to access segments and to provide the streaming service to the user.
  • a MPD document fragment may include a set of eXtensible Markup Language (XML)-encoded metadata fragments. The contents of the MPD provide the resource identifiers for segments and the context for the identified resources within the Media Presentation.
  • a MPD may include a MPD as described in ISO/IEC 23009-1:2014, currently proposed MPDs, and/or combinations thereof.
  • a media presentation as described in a MPD may include a sequence of one or more Periods, where each Period may include one or more Adaptation Sets. It should be noted that in the case where an Adaptation Set includes multiple media content components, then each media content component may be described individually. Each Adaptation Set may include one or more Representations.
  • each Representation is provided: (1) as a single Segment, where Subsegments are aligned across Representations with an Adaptation Set; and (2) as a sequence of Segments where each Segment is addressable by a template-generated Universal Resource Locator (URL).
  • the properties of each media content component may be described by an AdaptationSet element and/or elements within an Adaption Set, including for example, a ContentComponent element. It should be noted that the sphere region structure forms the basis of DASH descriptor signaling for various descriptors.
  • the mesh box data structure provided in W18865 may be less than ideal.
  • the mesh box data structure in W18865 explicitly signals mesh element identifiers for each mesh element, which is inefficient.
  • mesh element identifiers may be explicitly signaled in some cases and not signaled and inferred in some cases. This results in bit-savings.
  • FIG. 1 is a block diagram illustrating an example of a system that may be configured to code (i.e., encode and/or decode) video data according to one or more techniques of this disclosure.
  • System 100 represents an example of a system that may encapsulate video data according to one or more techniques of this disclosure.
  • system 100 includes source device 102, communications medium 110, and destination device 120.
  • source device 102 may include any device configured to encode video data and transmit encoded video data to communications medium 110.
  • Destination device 120 may include any device configured to receive encoded video data via communications medium 110 and to decode encoded video data.
  • Source device 102 and/or destination device 120 may include computing devices equipped for wired and/or wireless communications and may include, for example, set top boxes, digital video recorders, televisions, desktop, laptop or tablet computers, gaming consoles, medical imagining devices, and mobile devices, including, for example, smartphones, cellular telephones, personal gaming devices.
  • Communications medium 110 may include any combination of wireless and wired communication media, and/or storage devices.
  • Communications medium 110 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that may be useful to facilitate communications between various devices and sites.
  • Communications medium 110 may include one or more networks.
  • communications medium 110 may include a network configured to enable access to the World Wide Web, for example, the Internet.
  • a network may operate according to a combination of one or more telecommunication protocols. Telecommunications protocols may include proprietary aspects and/or may include standardized telecommunication protocols.
  • Examples of standardized telecommunications protocols include Digital Video Broadcasting (DVB) standards, Advanced Television Systems Committee (ATSC) standards, Integrated Services Digital Broadcasting (ISDB) standards, Data Over Cable Service Interface Specification (DOCSIS) standards, Global System Mobile Communications (GSM) standards, code division multiple access (CDMA) standards, 3rd Generation Partnership Project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards.
  • DVD Digital Video Broadcasting
  • ATSC Advanced Television Systems Committee
  • ISDB Integrated Services Digital Broadcasting
  • DOCSIS Data Over Cable Service Interface Specification
  • GSM Global System Mobile Communications
  • CDMA code division multiple access
  • 3GPP 3rd Generation Partnership Project
  • ETSI European Telecommunications Standards Institute
  • IP Internet Protocol
  • WAP Wireless Application Protocol
  • IEEE Institute of Electrical and Electronics Engineers
  • Storage devices may include any type of device or storage medium capable of storing data.
  • a storage medium may include a tangible or non-transitory computer-readable media.
  • a computer readable medium may include optical discs, flash memory, magnetic memory, or any other suitable digital storage media.
  • a memory device or portions thereof may be described as non-volatile memory and in other examples portions of memory devices may be described as volatile memory.
  • Examples of volatile memories may include random access memories (RAM), dynamic random access memories (DRAM), and static random access memories (SRAM).
  • Examples of non-volatile memories may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories.
  • Storage device(s) may include memory cards (e.g., a Secure Digital (SD) memory card), internal/external hard disk drives, and/or internal/external solid state drives. Data may be stored on a storage device according to a defined file format
  • FIG. 9 is a conceptual drawing illustrating an example of components that may be included in an implementation of system 100.
  • system 100 includes one or more computing devices 402A-402N, television service network 404, television service provider site 406, wide area network 408, local area network 410, and one or more content provider sites 412A-412N.
  • the implementation illustrated in FIG. 9 represents an example of a system that may be configured to allow digital media content, such as, for example, a movie, a live sporting event, etc., and data and applications and media presentations associated therewith to be distributed to and accessed by a plurality of computing devices, such as computing devices 402A-402N.
  • digital media content such as, for example, a movie, a live sporting event, etc.
  • computing devices 402A-402N such as computing devices 402A-402N.
  • computing devices 402A-402N may include any device configured to receive data from one or more of television service network 404, wide area network 408, and/or local area network 410.
  • computing devices 402A-402N may be equipped for wired and/or wireless communications and may be configured to receive services through one or more data channels and may include televisions, including so-called smart televisions, set top boxes, and digital video recorders.
  • computing devices 402A-402N may include desktop, laptop, or tablet computers, gaming consoles, mobile devices, including, for example, “smart” phones, cellular telephones, and personal gaming devices.
  • Television service network 404 is an example of a network configured to enable digital media content, which may include television services, to be distributed.
  • television service network 404 may include public over-the-air television networks, public or subscription-based satellite television service provider networks, and public or subscription-based cable television provider networks and/or over the top or Internet service providers.
  • television service network 404 may primarily be used to enable television services to be provided, television service network 404 may also enable other types of data and services to be provided according to any combination of the telecommunication protocols described herein.
  • television service network 404 may enable two-way communications between television service provider site 406 and one or more of computing devices 402A-402N.
  • Television service network 404 may comprise any combination of wireless and/or wired communication media.
  • Television service network 404 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that may be useful to facilitate communications between various devices and sites.
  • Television service network 404 may operate according to a combination of one or more telecommunication protocols.
  • Telecommunications protocols may include proprietary aspects and/or may include standardized telecommunication protocols. Examples of standardized telecommunications protocols include DVB standards, ATSC standards, ISDB standards, DTMB standards, DMB standards, Data Over Cable Service Interface Specification (DOCSIS) standards, HbbTV standards, W3C standards, and UPnP standards.
  • DOCSIS Data Over Cable Service Interface Specification
  • television service provider site 406 may be configured to distribute television service via television service network 404.
  • television service provider site 406 may include one or more broadcast stations, a cable television provider, or a satellite television provider, or an Internet-based television provider.
  • television service provider site 406 may be configured to receive a transmission including television programming through a satellite uplink/downlink.
  • television service provider site 406 may be in communication with wide area network 408 and may be configured to receive data from content provider sites 412A-412N. It should be noted that in some examples, television service provider site 406 may include a television studio and content may originate therefrom.
  • Wide area network 408 may include a packet based network and operate according to a combination of one or more telecommunication protocols.
  • Telecommunications protocols may include proprietary aspects and/or may include standardized telecommunication protocols. Examples of standardized telecommunications protocols include Global System Mobile Communications (GSM) standards, code division multiple access (CDMA) standards, 3 rd Generation Partnership Project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, European standards (EN), IP standards, Wireless Application Protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards, such as, for example, one or more of the IEEE 802 standards (e.g., Wi-Fi).
  • GSM Global System Mobile Communications
  • CDMA code division multiple access
  • 3GPP 3 rd Generation Partnership Project
  • ETSI European Telecommunications Standards Institute
  • EN European standards
  • IP standards European standards
  • WAP Wireless Application Protocol
  • IEEE Institute of Electrical and Electronics Engineers
  • Wide area network 408 may comprise any combination of wireless and/or wired communication media.
  • Wide area network 480 may include coaxial cables, fiber optic cables, twisted pair cables, Ethernet cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that may be useful to facilitate communications between various devices and sites.
  • wide area network 408 may include the Internet.
  • Local area network 410 may include a packet based network and operate according to a combination of one or more telecommunication protocols. Local area network 410 may be distinguished from wide area network 408 based on levels of access and/or physical infrastructure. For example, local area network 410 may include a secure home network.
  • content provider sites 412A-412N represent examples of sites that may provide multimedia content to television service provider site 406 and/or computing devices 402A-402N.
  • a content provider site may include a studio having one or more studio content servers configured to provide multimedia files and/or streams to television service provider site 406.
  • content provider sites 412A-412N may be configured to provide multimedia content using the IP suite.
  • a content provider site may be configured to provide multimedia content to a receiver device according to Real Time Streaming Protocol (RTSP), HTTP, or the like.
  • RTSP Real Time Streaming Protocol
  • content provider sites 412A-412N may be configured to provide data, including hypertext based content, and the like, to one or more of receiver devices computing devices 402A-402N and/or television service provider site 406 through wide area network 408.
  • Content provider sites 412A-412N may include one or more web servers. Data provided by data provider site 412A-412N may be defined according to data formats.
  • source device 102 includes video source 104, video encoder 106, data encapsulator 107, and interface 108.
  • Video source 104 may include any device configured to capture and/or store video data.
  • video source 104 may include a video camera and a storage device operably coupled thereto.
  • Video encoder 106 may include any device configured to receive video data and generate a compliant bitstream representing the video data.
  • a compliant bitstream may refer to a bitstream that a video decoder can receive and reproduce video data therefrom. Aspects of a compliant bitstream may be defined according to a video coding standard. When generating a compliant bitstream video encoder 106 may compress video data. Compression may be lossy (discernible or indiscernible to a viewer) or lossless.
  • data encapsulator 107 may receive encoded video data and generate a compliant bitstream, e.g., a sequence of NAL units according to a defined data structure.
  • a device receiving a compliant bitstream can reproduce video data therefrom.
  • conforming bitstream may be used in place of the term compliant bitstream.
  • data encapsulator 107 need not necessary be located in the same physical device as video encoder 106. For example, functions described as being performed by video encoder 106 and data encapsulator 107 may be distributed among devices illustrated in FIG. 9.
  • data encapsulator 107 may include a data encapsulator configured to receive one or more media components and generate media presentation based on DASH.
  • FIG. 10 is a block diagram illustrating an example of a data encapsulator that may implement one or more techniques of this disclosure.
  • Data encapsulator 500 may be configured to generate a media presentation according to the techniques described herein.
  • functional blocks of component encapsulator 500 correspond to functional blocks for generating a media presentation (e.g., a DASH media presentation).
  • component encapsulator 500 includes media presentation description generator 502, segment generator 504, and system memory 506.
  • Each of media presentation description generator 502, segment generator 504, and system memory 506 may be interconnected (physically, communicatively, and/or operatively) for inter-component communications and may be implemented as any of a variety of suitable circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable gate arrays
  • data encapsulator 500 is illustrated as having distinct functional blocks, such an illustration is for descriptive purposes and does not limit data encapsulator 500 to a particular hardware architecture. Functions of data encapsulator 500 may be realized using any combination of hardware, firmware and/or software implementations.
  • Media presentation description generator 502 may be configured to generate media presentation description fragments. Segment generator 504 may be configured to receive media components and generate one or more segments for inclusion in a media presentation.
  • System memory 506 may be described as a non-transitory or tangible computer-readable storage medium. In some examples, system memory 506 may provide temporary and/or long-term storage. In some examples, system memory 506 or portions thereof may be described as non-volatile memory and in other examples portions of system memory 506 may be described as volatile memory. System memory 506 may be configured to store information that may be used by data encapsulator during operation.
  • mesh element identifiers may be explicitly signaled in some cases and not signaled and inferred in some cases.
  • a flag may be included in the mesh box data structure to indicate if unique Ids (identifiers) for each mesh element are explicitly signaled or are implicitly inferred. In one example, if the value of the flag is equal to one, a mesh element Id for each mesh element in the mesh box is explicitly signaled. Otherwise (i.e. if flag is equal to zero), the mesh element id for each mesh element in the mesh box is inferred.
  • the meaning of the flag may be flipped such that: if the value of the flag is equal to zero, a mesh element Id for each mesh element in the mesh box is explicitly signaled. Otherwise (i.e. if flag is equal to one), the mesh element id for each mesh element in the mesh box is inferred.
  • data encapsulator 107 may be configured to signal a mesh box using and/or based on the following definition, syntax and semantics: MeshBox specifies a 3D mesh consisting of one or more mesh elements, each of which have a unique id. The different mesh elements are referenced by one or more tile mesh group entries (e.g.
  • TileMeshGroupEntry which define the relationship between a group of samples (or a complete tile track) and the position on the mesh. This information can be used for both tile selection as well as during the rendering process. In line with how typical rendering engines work with stereoscopic content, the same mesh is assumed to be used for both eyes in case of stereoscopic content. In other words, it is not possible to define distinct left-eye and right-eye meshes.
  • the MeshBox also provides implicit content coverage information.
  • num_mesh_elements specifies the number of mesh elements described in this box.
  • mesh_type specifies the type of the mesh description used for all mesh elements described in this box. mesh_type shall be equal to 0 or 1. Other values of mesh_type are reserved.
  • SphereRegionStruct specifies a spherical mesh as the spherical region specified in the structure as defined above.
  • the SphereRegionStruct shall be inferred to have shape type value equal to 1 (specified by two azimuth circles and two elevation circles).
  • 3DParallelogramStruct specifies a parallelogram mesh in the 3D space. Two mesh elements described in this box must not overlap.
  • mesh_element_id specifies an identifier to uniquely identify a mesh element within this mesh box. When explicit_mesh_id is equal to 0, mesh_element_id for the i-th mesh element is inferred to be equal to i.
  • some other inference based on the loop index may be done for the i-th mesh element. For example: when explicit_mesh_id is equal to 0, mesh_element_id for the i-th mesh element is inferred to be equal to i+1.
  • the mesh_element_id is not signaled and its value is always inferred.
  • data encapsulator 107 may be configured to signal a mesh box using and/or based on the following definition, syntax and semantics: num_mesh_elements specifies the number of mesh elements described in this box.
  • mesh_type specifies the type of the mesh description used for all mesh elements described in this box.
  • mesh_type shall be equal to 0 or 1. Other values of mesh_type are reserved.
  • SphereRegionStruct specifies a spherical mesh as the spherical region specified in the structure as defined above.
  • the SphereRegionStruct shall be inferred to have shape type value equal to 1 (specified by two azimuth circles and two elevation circles).
  • 3DParallelogramStruct specifies a parallelogram mesh in the 3D space. Two mesh elements described in this box must not overlap.
  • a value of mesh_element_id for the i-th mesh element is inferred to be equal to i.
  • some other inference based on the loop index may be done for the i-th mesh element. For example: when explicit_mesh_id is equal to 0, mesh_element_id for the i-th mesh element is inferred to be equal to i+1.
  • the mesh_element_id syntax element may be referred to be with an index i as mesh_element_id[i].
  • data encapsulator 107 may be configured to signal a mesh box using and/or based on the following definition, syntax and semantics: num_mesh_elements specifies the number of mesh elements described in this box.
  • mesh_type specifies the type of the mesh description used for all mesh elements described in this box.
  • mesh_type shall be equal to 0 or 1. Other values of mesh_type are reserved.
  • SphereRegionStruct specifies a spherical mesh as the spherical region specified in the structure as defined above.
  • the SphereRegionStruct shall be inferred to have shape type value equal to 1 (specified by two azimuth circles and two elevation circles).
  • 3DParallelogramStruct specifies a parallelogram mesh in the 3D space. Two mesh elements described in this box must not overlap.
  • mesh_element_id[i] specifies an identifier for the i-th mesh element to uniquely identify a mesh element within this mesh box. When explicit_mesh_id is equal to 0, mesh_element_id for the i-th mesh element is inferred to be equal to i. It should be noted that some other name than the name mesh_element_id may be used for example mesh_element_identifier or mesh_element_identifier[i].
  • W18865 provides a tile mesh sample grouping for the mapping of rectangular regions to a 3D mesh.
  • a flag may be signaled to indicate if the content of a tile mesh sample grouping is stereoscopic content or non-stereoscopic content and only if the content is stereoscopic content a flag eye[i] is signaled for each of the number of packed regions (for i in the range of 0 to num_regions-1, inclusive) to specify the eye associated with the region. Otherwise, the value of the flag eye[i] is inferred.
  • This signaling provides bit-savings by not signaling a flag eye[i] and 7 reserved bits in cases of non-stereoscopic content.
  • data encapsulator 107 may be configured to signal a tile mesh sample grouping using and/or based on the following definition, syntax and semantics:
  • the TileMeshGroupEntry describes one or more rectangular regions within the track, and describes how they should be mapped onto one or more mesh elements described by the MeshBox associated with this track.
  • num_regions specifies the number of packed regions.
  • guard_band_flag specifies the existence of the GuardBand struct for all regions.
  • stereo_content_flag (may also be named is_eye_present_flag) equal to 1 specifies that syntax element eye[i] is present. stereo_content_flag equal to 0 specifies that syntax element eye[i] is not present and is inferred.
  • eye[i] specifies the eye associated with the i-th rectangular region. When set to 0, the region corresponds to the left eye; when set to 1, the region corresponds to the right eye. When not present eye[i] is inferred to be equal to 0. In a variant example: When eye[i] is not present the content is inferred to be monoscopic (or not stereoscopic).
  • mesh_element_id[i] specifies to which mesh element from the associated MeshBox the i-th rectangular region is mapped to.
  • the mesh_element_id[i] is associated with that MeshBox.
  • the mesh_element_id[i] is associated with the MeshBox in that tile base track.
  • RectRegionStruct specifies a rectangular region. The regions described by any two instances of RectRegionStruct in the same TileMeshGroupEntry shall not overlap.
  • GuardBand(i) specifies the guard bands around the i-th rectangular region.
  • eye[i] may instead be signalled using 2 bits with semantics as follows: eye[i] specifies the eye associated with the i-th rectangular region. When set to 0, the region corresponds to the left eye; when set to 1, the region corresponds to the right eye. When set equal to 3 the content is monoscopic. When set to 2, the region corresponds to both the left and right eye. For non-stereoscopic content, eye[i] shall be set equal to 3.
  • the syntax may be either: Or as:
  • the meaning of the above specified values 0 to 3 for eye[i] may be rearranged such that different semantic meanings are associated with different values.
  • the semantics may be based on the following: When set to 1, the region corresponds to the left eye; when set to 2, the region corresponds to the right eye. When set equal to 0 the content is monoscopic. When set to 3, the region corresponds to both the left and right eye.
  • W18865 further includes an overlay structure for storing and signaling overlays.
  • An overlay may be defined as a rendering of visual media over 360-degree video content.
  • the visual media may include one or more of videos, images, and text. Overlays may be of different types, e.g., a logo, a thumbnail of the recommended viewport, etc.
  • a dynamic overlay timed metadata track may indicate which overlays are active at particular times. Depending upon the application the active overlay(s) may change over time and overlay parameters that may be dynamically changing over time.
  • an overlay may be specified with the following pieces of information:
  • an overlay source may be one of the following:
  • the picture that contains the source region may also contain other overlay source regions or background visual media.
  • the rendering type of an overlay may be one of the following:
  • the following rendering properties may be indicated for an overlay:
  • the following user interaction properties may be indicated for an overlay:
  • the overlay source, rendering type, rendering properties and user interaction properties for an overlay are specified as overlay controls in the SingleOverlayStruct syntax structure.
  • SingleOverlayStruct is extensible, i.e. new overlay controls can be introduced in future documents.
  • overlay controls are indicated to be non-essential or essential. In the latter case, an OMAF player is required to process the overlay control.
  • a content author could specify an overlay control to be non-essential e.g. when applying the overlay control for rendering is preferred but not mandated.
  • SingleOverlayStruct additionally contains an overlay identifier (overlay_id).
  • overlay controls can be static or time-varying. Static overlay controls are contained in a sample entry in a track containing the overlay source or as an item property for an image item containing the overlay source. Time-varying overlay controls are included in a timed metadata track, which also specifies which overlays are active (i.e. turned on, unless otherwise controlled by a user interaction) or inactive (i.e. turned off) at a given time.
  • OverlayStruct specifies the overlay related metadata per each overlay.
  • num_overlays specifies the number of overlays described by this structure.
  • num_overlays equal to 0 is reserved.
  • num_flag_bytes specifies the number of bytes allocated collectively by the overlay_control_flag[i] syntax elements.
  • OMAF players shall parse the syntax of OverlayStruct() regardless of the value of num_flag_bytes. NOTE: In files conforming to this version, num_flag_bytes is expected to be equal to 1 or 2, depending on which overlay control structures are present. overlay_id provides a unique identifier for the overlay. No two overlays shall have the same overlay_id value. overlay_control_flag[i] when set to 1 defines that the structure as defined by the i-th overlay_control_struct[i] is present. overlay_control_flag[i] shall not be equal to 1 when i is greater than LastControlIdx. OMAF players shall allow both values of overlay_control_flag[i] to appear in the syntax for all values of i.
  • overlay_control_essential_flag[i] 0 specifies that OMAF players are not required to process the structure as defined by the i-th overlay_control_struct[i].
  • overlay_control_essential_flag[i] 1 specifies that OMAF players shall process the structure as defined by the i-th overlay_control_struct[i].
  • overlay_control_flag[i] with i in the range of 0 to 6, inclusive, or equal to 11 the value of overlay_control_essential_flag[i] shall be equal to 1.
  • OMAF players shall be able to parse and process the structure corresponding to overlay_control_struct[11] (i.e., the OverlayPriority control structure).
  • overlay_control_essential_flag[i] is equal to 1 and an OMAF player is not capable of parsing or processing the structure as defined by overlay_control_struct[i]
  • byte_count[i] gives the byte count of the structure represented by the i-th overlay_control_struct[i].
  • byte_count[4] when present, shall be equal 0.
  • overlay_control_struct[i][byte_count[i]] defines the i-th structure with a byte count as defined by byte_count[i].
  • Controls with bit indices 0 to 3, inclusive specify the rendering type of the overlay.
  • one and exactly one of overlay_control_flag[i] with i in the range of 0 to 3, inclusive, in each SingleOverlayStruct shall be equal to 1.
  • Controls with bit indices 4 to 6, inclusive specify the source of the overlay content.
  • one and exactly one of overlay_control_flag[i] with i in the range of 4 to 6, inclusive, in each SingleOverlayStruct shall be equal to 1.
  • LastControlIdx is set equal to 13.
  • the syntax and semantics of a 3D mesh overlay may be based on the following: timeline_change_flag equal to 1 specifies that the overlay content playback shall pause if the overlay is not in the user's current viewport, and when the overlay is back in the user's viewport the overlay content playback shall resume with the global presentation timeline of the content. The content in the intermediate interval is skipped.
  • timeline_change_flag 0 specifies that the overlay content playback shall pause if the overlay is not in the user's current viewport, and when the overlay is back in the user's viewport the overlay content playback resumes from the paused sample.
  • timeline_change_flag shall be equal to 1, when the overlay source is specified by OverlaySourceRegion.
  • num_regions specifies the number of packed regions. In another example: num_regions specifies the number of mesh regions or number of packed mesh regions.
  • eye[i] specifies the eye associated with the i-th rectangular region. When set to 0, the region corresponds to the left eye; when set to 1, the region corresponds to the right eye.
  • mesh_element_id[i] specifies to which mesh element from the associated MeshBox the i-th rectangular region is mapped to.
  • a flag may be signaled to indicate if the content of a tile mesh sample grouping is stereoscopic content or non-stereoscopic content and only if the content is stereoscopic content a flag eye[i] is signaled for each of the number of packed regions (for i in the range of 0 to num_regions-1, inclusive) to specify the eye associated with the region. Otherwise, the value of the flag eye[i] is inferred.
  • This signaling provides bit-savings by not signaling a flag eye[i] and 7 reserved bits in cases of non-stereoscopic content.
  • data encapsulator 107 may be configured to signal a flag to indicate if the content of a 3D mesh overlay is stereoscopic content or non-stereoscopic content and only if the content is stereoscopic content a flag eye[i] and 7 reserved bits are signaled for each of the number of regions where eye[i] specifies the eye associated with the region. Otherwise, the value of the flag eye[i] is inferred.
  • data encapsulator 107 may be configured to signal a 3D mesh overlay using and/or based on the following syntax and semantics: timeline_change_flag equal to 1 specifies that the overlay content playback shall pause if the overlay is not in the user's current viewport, and when the overlay is back in the user's viewport the overlay content playback shall resume with the global presentation timeline of the content. The content in the intermediate interval is skipped. timeline_change_flag equal to 0 specifies that the overlay content playback shall pause if the overlay is not in the user's current viewport, and when the overlay is back in the user's viewport the overlay content playback resumes from the paused sample.
  • eye_and_reserved_present_flag (may also be named eye_present_flag) equal to 1 specifies that syntax element eye[i] and 7 reserved bits are present. Eye_and_reserved_present_flag equal to 0 specifies that syntax element eye[i] and 7 reserved bits are not present and eye[i] is inferred.
  • the syntax element may be named eye_present_flag with the semantics as follows: eye_present_flag equal to 1 specifies that syntax element eye[i] is present. Eye_present_flag equal to 0 specifies that syntax element eye[i] is not present and is inferred.
  • num_regions specifies the number of packed regions. In another example, num_regions specifies the number of mesh regions or number of packed mesh regions.
  • eye[i] specifies the eye associated with the i-th rectangular region. When set to 0, the region corresponds to the left eye; when set to 1, the region corresponds to the right eye. When not present eye[i] is inferred to be equal to 0.
  • mesh_element_id[i] specifies to which mesh element from the associated MeshBox the i-th rectangular region is mapped to.
  • data encapsulator 107 represents an example of a device configured to signal a mesh box data structure and signal a value for a flag in the mesh box data structure indicating whether for each element of a mesh an identifier is included in the mesh box or inferred.
  • interface 108 may include any device configured to receive data generated by data encapsulator 107 and transmit and/or store the data to a communications medium.
  • Interface 108 may include a network interface card, such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can send and/or receive information.
  • interface 108 may include a computer system interface that may enable a file to be stored on a storage device.
  • interface 108 may include a chipset supporting Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Express (PCIe) bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, I 2 C, or any other logical and physical structure that may be used to interconnect peer devices.
  • PCI Peripheral Component Interconnect
  • PCIe Peripheral Component Interconnect Express
  • USB Universal Serial Bus
  • destination device 120 includes interface 122, data decapsulator 123, video decoder 124, and display 126.
  • Interface 122 may include any device configured to receive data from a communications medium.
  • Interface 122 may include a network interface card, such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can receive and/or send information.
  • interface 122 may include a computer system interface enabling a compliant video bitstream to be retrieved from a storage device.
  • interface 122 may include a chipset supporting PCI and PCIe bus protocols, proprietary bus protocols, USB protocols, I 2 C, or any other logical and physical structure that may be used to interconnect peer devices.
  • Data decapsulator 123 may be configured to receive a bitstream generated by data encaspulator 107 and perform sub-bitstream extraction according to one or more of the techniques described herein.
  • Video decoder 124 may include any device configured to receive a bitstream and/or acceptable variations thereof and reproduce video data therefrom.
  • Display 126 may include any device configured to display video data.
  • Display 126 may comprise one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display.
  • Display 126 may include a High Definition display or an Ultra High Definition display.
  • Display 126 may include a stereoscopic display. It should be noted that although in the example illustrated in FIG. 1, video decoder 124 is described as outputting data to display 126, video decoder 124 may be configured to output video data to various types of devices and/or sub-components thereof. For example, video decoder 124 may be configured to output video data to any communication medium, as described herein. Destination device 120 may include a receive device.
  • FIG. 11 is a block diagram illustrating an example of a receiver device that may implement one or more techniques of this disclosure. That is, receiver device 600 may be configured to parse a signal based on the semantics described above. Further, receiver device 600 may be configured to operate according to expected player behavior described herein. That is, for example, receiver device 600 may be configured to a use mesh information according to the provided expected OMAF player operations described herein. Further, receiver device 600 may be configured to use mesh information in various ways to provide interactivity to a user. Receiver device 600 is an example of a computing device that may be configured to receive data from a communications network and allow a user to access multimedia content, including a virtual reality application. In the example illustrated in FIG.
  • receiver device 600 is configured to receive data via a television network, such as, for example, television service network 404 described above. Further, in the example illustrated in FIG. 11, receiver device 600 is configured to send and receive data via a wide area network. It should be noted that in other examples, receiver device 600 may be configured to simply receive data through a television service network 404. The techniques described herein may be utilized by devices configured to communicate using any and all combinations of communications networks.
  • receiver device 600 includes central processing unit(s) 602, system memory 604, system interface 610, data extractor 612, audio decoder 614, audio output system 616, video decoder 618, display system 620, I/O device(s) 622, and network interface 624.
  • system memory 604 includes operating system 606 and applications 608.
  • Each of central processing unit(s) 602, system memory 604, system interface 610, data extractor 612, audio decoder 614, audio output system 616, video decoder 618, display system 620, I/O device(s) 622, and network interface 624 may be interconnected (physically, communicatively, and/or operatively) for inter-component communications and may be implemented as any of a variety of suitable circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable gate arrays
  • receiver device 600 is illustrated as having distinct functional blocks, such an illustration is for descriptive purposes and does not limit receiver device 600 to a particular hardware architecture. Functions of receiver device 600 may be realized using any combination of hardware, firmware and/or software implementations.
  • CPU(s) 602 may be configured to implement functionality and/or process instructions for execution in receiver device 600.
  • CPU(s) 602 may include single and/or multi-core central processing units.
  • CPU(s) 602 may be capable of retrieving and processing instructions, code, and/or data structures for implementing one or more of the techniques described herein. Instructions may be stored on a computer readable medium, such as system memory 604.
  • System memory 604 may be described as a non-transitory or tangible computer-readable storage medium. In some examples, system memory 604 may provide temporary and/or long-term storage. In some examples, system memory 604 or portions thereof may be described as non-volatile memory and in other examples portions of system memory 604 may be described as volatile memory. System memory 604 may be configured to store information that may be used by receiver device 600 during operation. System memory 604 may be used to store program instructions for execution by CPU(s) 602 and may be used by programs running on receiver device 600 to temporarily store information during program execution. Further, in the example where receiver device 600 is included as part of a digital video recorder, system memory 604 may be configured to store numerous video files.
  • Applications 608 may include applications implemented within or executed by receiver device 600 and may be implemented or contained within, operable by, executed by, and/or be operatively/communicatively coupled to components of receiver device 600. Applications 608 may include instructions that may cause CPU(s) 602 of receiver device 600 to perform particular functions. Applications 608 may include algorithms which are expressed in computer programming statements, such as, for-loops, while-loops, if-statements, do-loops, etc. Applications 608 may be developed using a specified programming language. Examples of programming languages include, Java TM , Jini TM , C, C++, Objective C, Swift, Perl, Python, PhP, UNIX Shell, Visual Basic, and Visual Basic Script.
  • receiver device 600 includes a smart television
  • applications may be developed by a television manufacturer or a broadcaster.
  • applications 608 may execute in conjunction with operating system 606. That is, operating system 606 may be configured to facilitate the interaction of applications 608 with CPUs(s) 602, and other hardware components of receiver device 600.
  • Operating system 606 may be an operating system designed to be installed on set-top boxes, digital video recorders, televisions, and the like. It should be noted that techniques described herein may be utilized by devices configured to operate using any and all combinations of software architectures.
  • System interface 610 may be configured to enable communications between components of receiver device 600.
  • system interface 610 comprises structures that enable data to be transferred from one peer device to another peer device or to a storage medium.
  • system interface 610 may include a chipset supporting Accelerated Graphics Port (AGP) based protocols, Peripheral Component Interconnect (PCI) bus based protocols, such as, for example, the PCI Express TM (PCIe) bus specification, which is maintained by the Peripheral Component Interconnect Special Interest Group, or any other form of structure that may be used to interconnect peer devices (e.g., proprietary bus protocols).
  • AGP Accelerated Graphics Port
  • PCI Peripheral Component Interconnect
  • PCIe PCI Express TM
  • PCIe Peripheral Component Interconnect Special Interest Group
  • receiver device 600 is configured to receive and, optionally, send data via a television service network.
  • a television service network may operate according to a telecommunications standard.
  • a telecommunications standard may define communication properties (e.g., protocol layers), such as, for example, physical signaling, addressing, channel access control, packet properties, and data processing.
  • data extractor 612 may be configured to extract video, audio, and data from a signal.
  • a signal may be defined according to, for example, aspects DVB standards, ATSC standards, ISDB standards, DTMB standards, DMB standards, and DOCSIS standards.
  • Data extractor 612 may be configured to extract video, audio, and data, from a signal. That is, data extractor 612 may operate in a reciprocal manner to a service distribution engine. Further, data extractor 612 may be configured to parse link layer packets based on any combination of one or more of the structures described above.
  • Audio decoder 614 may be configured to receive and process audio packets.
  • audio decoder 614 may include a combination of hardware and software configured to implement aspects of an audio codec. That is, audio decoder 614 may be configured to receive audio packets and provide audio data to audio output system 616 for rendering.
  • Audio data may be coded using multi-channel formats such as those developed by Dolby and Digital Theater Systems. Audio data may be coded using an audio compression format. Examples of audio compression formats include Motion Picture Experts Group (MPEG) formats, Advanced Audio Coding (AAC) formats, DTS-HD formats, and Dolby Digital (AC-3) formats.
  • MPEG Motion Picture Experts Group
  • AAC Advanced Audio Coding
  • DTS-HD formats DTS-HD formats
  • AC-3 formats Dolby Digital
  • Audio output system 616 may be configured to render audio data.
  • audio output system 616 may include an audio processor, a digital-to-analog converter, an amplifier, and a speaker system.
  • a speaker system may include any of a variety of speaker systems, such as headphones, an integrated stereo speaker system, a multi-speaker system, or a surround sound system.
  • Video decoder 618 may be configured to receive and process video packets.
  • video decoder 618 may include a combination of hardware and software used to implement aspects of a video codec.
  • video decoder 618 may be configured to decode video data encoded according to any number of video compression standards, such as ITU-T H.262 or ISO/IEC MPEG-2 Visual, ISO/IEC MPEG-4 Visual, ITU-T H.264 (also known as ISO/IEC MPEG-4 Advanced video Coding (AVC)), and High-Efficiency Video Coding (HEVC).
  • Display system 620 may be configured to retrieve and process video data for display. For example, display system 620 may receive pixel data from video decoder 618 and output data for visual presentation.
  • display system 620 may be configured to output graphics in conjunction with video data, e.g., graphical user interfaces.
  • Display system 620 may comprise one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device capable of presenting video data to a user.
  • a display device may be configured to display standard definition content, high definition content, or ultra-high definition content.
  • I/O device(s) 622 may be configured to receive input and provide output during operation of receiver device 600. That is, I/O device(s) 622 may enable a user to select multimedia content to be rendered. Input may be generated from an input device, such as, for example, a push-button remote control, a device including a touch-sensitive screen, a motion-based input device, an audio-based input device, or any other type of device configured to receive user input. I/O device(s) 622 may be operatively coupled to receiver device 600 using a standardized communication protocol, such as for example, Universal Serial Bus protocol (USB), Bluetooth, ZigBee or a proprietary communications protocol, such as, for example, a proprietary infrared communications protocol.
  • USB Universal Serial Bus protocol
  • ZigBee ZigBee
  • proprietary communications protocol such as, for example, a proprietary infrared communications protocol.
  • Network interface 624 may be configured to enable receiver device 600 to send and receive data via a local area network and/or a wide area network.
  • Network interface 624 may include a network interface card, such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device configured to send and receive information.
  • Network interface 624 may be configured to perform physical signaling, addressing, and channel access control according to the physical and Media Access Control (MAC) layers utilized in a network.
  • Receiver device 600 may be configured to parse a signal generated according to any of the techniques described above with respect to FIG. 11.
  • receiver device 600 represents an example of a device configured to receive a mesh box data structure, parse value for a flag in the mesh box data structure, and determine for each element of a mesh whether an identifier is included in the mesh box or inferred based on the parsed value of the flag.
  • Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol.
  • Computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave.
  • Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure.
  • a computer program product may include a computer-readable medium.
  • such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.
  • any connection is properly termed a computer-readable medium.
  • a computer-readable medium For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium.
  • DSL digital subscriber line
  • Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
  • processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • processors may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein.
  • the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques could be fully implemented in one or more circuits or logic elements.
  • the techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set).
  • IC integrated circuit
  • a set of ICs e.g., a chip set.
  • Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit or provided by a collection of interoperative hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.
  • each functional block or various features of the base station device and the terminal device used in each of the aforementioned embodiments may be implemented or executed by a circuitry, which is typically an integrated circuit or a plurality of integrated circuits.
  • the circuitry designed to execute the functions described in the present specification may comprise a general-purpose processor, a digital signal processor (DSP), an application specific or general application integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic, or a discrete hardware component, or a combination thereof.
  • the general-purpose processor may be a microprocessor, or alternatively, the processor may be a conventional processor, a controller, a microcontroller or a state machine.
  • the general-purpose processor or each circuit described above may be configured by a digital circuit or may be configured by an analogue circuit. Further, when a technology of making into an integrated circuit superseding integrated circuits at the present time appears due to advancement of a semiconductor technology, the integrated circuit by this technology is also able to be used.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon l'invention, pour transmettre des informations associées à une vidéo omnidirectionnelle, les trois éléments de syntaxe suivants sont utilisés. (Voir paragraphe [0056] pour des détails.) (1) "num_mesh_elements", élément de syntaxe spécifiant un nombre d'éléments mesh dans la meshbox. (2) "explicit_mesh_id", élément de syntaxe indiquant si un élément de syntaxe "mesh_element_id" est présent dans la meshbox. (3) "mesh_element_id", élément de syntaxe spécifiant un identifiant pour identifier de façon unique un élément mesh dans la meshbox.
PCT/JP2020/046443 2019-12-16 2020-12-14 Systèmes et procédés pour signaler des informations pour un mesh dans un support omnidirectionnel WO2021125117A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201962948678P 2019-12-16 2019-12-16
US62/948,678 2019-12-16
US202062956938P 2020-01-03 2020-01-03
US62/956,938 2020-01-03
US202062970064P 2020-02-04 2020-02-04
US62/970,064 2020-02-04

Publications (1)

Publication Number Publication Date
WO2021125117A1 true WO2021125117A1 (fr) 2021-06-24

Family

ID=76476605

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/046443 WO2021125117A1 (fr) 2019-12-16 2020-12-14 Systèmes et procédés pour signaler des informations pour un mesh dans un support omnidirectionnel

Country Status (1)

Country Link
WO (1) WO2021125117A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160352872A1 (en) * 2015-05-26 2016-12-01 Thomson Licensing Method and device for encoding/decoding a packet comprising data representative of a haptic effect

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160352872A1 (en) * 2015-05-26 2016-12-01 Thomson Licensing Method and device for encoding/decoding a packet comprising data representative of a haptic effect

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MOHAMMAD HOSSEINI, VISWANATHAN SWAMINATHAN: "Adaptive 360 VR Video Streaming: Divide and Conquer", 27 September 2016 (2016-09-27), XP055597359, Retrieved from the Internet <URL:https://arxiv.org/pdf/1609.08729v1.pdf> DOI: 10.1109/ISM.2016.0028 *

Similar Documents

Publication Publication Date Title
WO2019189038A1 (fr) Systèmes et procédés de signalisation d&#39;informations de paramètres de caméra
US20200120326A1 (en) Systems and methods for signaling view information for virtual reality applications
WO2020045593A1 (fr) Systèmes et procédés de signalisation d&#39;informations de métadonnées synchronisées à une sous-image
WO2019194241A1 (fr) Systèmes et procédés de signalisation d&#39;informations de composition d&#39;image secondaire pour des applications de réalité virtuelle
WO2019146601A1 (fr) Systèmes et procédés pour la signalisation d&#39;information de position
US10880617B2 (en) Systems and methods for signaling quality information for regions in virtual reality applications
US10848735B2 (en) Systems and methods for signaling information associated with constituent pictures in virtual reality applications
WO2020184645A1 (fr) Systèmes et procédés de signalisation d&#39;informations de point de vue dans du contenu multimédia omnidirectionnel
WO2019235305A1 (fr) Systèmes et procédés de signalisation d&#39;informations de superposition
WO2019139014A1 (fr) Systèmes et procédés de signalisation d&#39;informations de composition d&#39;image secondaire pour des applications de réalité virtuelle
US20200221104A1 (en) Systems and methods for signaling a projected region for virtual reality applications
US20200382809A1 (en) Systems and methods for signaling of information associated with most-interested regions for virtual reality applications
WO2018212009A1 (fr) Systèmes et procédés de mise en correspondance d&#39;emplacements d&#39;échantillons avec des coordonnées angulaires dans des applications de réalité virtuelle
WO2021125117A1 (fr) Systèmes et procédés pour signaler des informations pour un mesh dans un support omnidirectionnel
WO2021137300A1 (fr) Systèmes et procédés de signalisation d&#39;informations de commutation de point de vue dans du contenu multimédia omnidirectionnel
WO2021125185A1 (fr) Systèmes et procédés de signalisation d&#39;informations de bouclage de point de vue dans du contenu multimédia omnidirectionnel
WO2021075407A1 (fr) Systèmes et procédés pour permettre une interactivité pour des emplacements actionnables dans des moyens multimédia omnidirectionnels
US20230421828A1 (en) Systems and methods for signaling content component information in omnidirectional media
WO2019203102A1 (fr) Systèmes et procédés de signalisation de messages spécifiques à une application dans une application de suivi de réalité virtuelle
WO2020141604A1 (fr) Systèmes et procédés de signalisation d&#39;informations de paramètres de caméra
WO2019139052A1 (fr) Systèmes et procédés de signalisation d&#39;informations de source pour des applications de réalité virtuelle
WO2018179843A1 (fr) Systèmes et procédés de signalisation d&#39;informations pour des applications de réalité virtuelle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20903801

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20903801

Country of ref document: EP

Kind code of ref document: A1