WO2018212009A1 - Systèmes et procédés de mise en correspondance d'emplacements d'échantillons avec des coordonnées angulaires dans des applications de réalité virtuelle - Google Patents

Systèmes et procédés de mise en correspondance d'emplacements d'échantillons avec des coordonnées angulaires dans des applications de réalité virtuelle Download PDF

Info

Publication number
WO2018212009A1
WO2018212009A1 PCT/JP2018/017698 JP2018017698W WO2018212009A1 WO 2018212009 A1 WO2018212009 A1 WO 2018212009A1 JP 2018017698 W JP2018017698 W JP 2018017698W WO 2018212009 A1 WO2018212009 A1 WO 2018212009A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
packed
picture
reg
region
Prior art date
Application number
PCT/JP2018/017698
Other languages
English (en)
Inventor
Sachin G. Deshpande
Original Assignee
Sharp Kabushiki Kaisha
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sharp Kabushiki Kaisha filed Critical Sharp Kabushiki Kaisha
Publication of WO2018212009A1 publication Critical patent/WO2018212009A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194—Transmission of image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • This disclosure relates to interactive video distribution and more particularly to techniques for mapping sample locations to angular coordinates.
  • Digital media playback capabilities may be incorporated into a wide range of devices, including digital televisions, including so-called “smart” televisions, set-top boxes, laptop or desktop computers, tablet computers, digital recording devices, digital media players, video gaming devices, cellular phones, including so-called “smart” phones, dedicated video streaming devices, and the like.
  • Digital media content (e.g., video and audio programming) may originate from a plurality of sources including, for example, over-the-air television providers, satellite television providers, cable television providers, online media service providers, including, so-called streaming service providers, and the like.
  • Digital media content may be delivered over packet-switched networks, including bidirectional networks, such as Internet Protocol (IP) networks and unidirectional networks, such as digital broadcast networks.
  • IP Internet Protocol
  • Digital video included in digital media content may be coded according to a video coding standard.
  • Video coding standards may incorporate video compression techniques. Examples of video coding standards include ISO/IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC) and High-Efficiency Video Coding (HEVC).
  • Video compression techniques enable data requirements for storing and transmitting video data to be reduced. Video compression techniques may reduce data requirements by exploiting the inherent redundancies in a video sequence.
  • Video compression techniques may sub-divide a video sequence into successively smaller portions (i.e., groups of frames within a video sequence, a frame within a group of frames, slices within a frame, coding tree units (e.g., macroblocks) within a slice, coding blocks within a coding tree unit, etc.).
  • Prediction coding techniques may be used to generate difference values between a unit of video data to be coded and a reference unit of video data. The difference values may be referred to as residual data.
  • Residual data may be coded as quantized transform coefficients.
  • Syntax elements may relate residual data and a reference coding unit. Residual data and syntax elements may be included in a compliant bitstream. Compliant bitstreams and associated metadata may be formatted according to data structures.
  • Compliant bitstreams and associated metadata may be transmitted from a source to a receiver device (e.g., a digital television or a smart phone) according to a transmission standard.
  • a transmission standard include Digital Video Broadcasting (DVB) standards, Integrated Services Digital Broadcasting Standards (ISDB) standards, and standards developed by the Advanced Television Systems Committee (ATSC), including, for example, the ATSC 2.0 standard.
  • the ATSC is currently developing the so-called ATSC 3.0 suite of standards.
  • this disclosure describes various techniques for coding video data.
  • this disclosure describes techniques for mapping sample locations in a packed frame to angular coordinates of a projection structure.
  • this disclosure describes techniques for mapping sample locations in a packed frame to angular coordinates of a projection structure.
  • the techniques of this disclosure are generally applicable to video coding.
  • the coding techniques described herein may be incorporated into video coding systems, (including video coding systems based on future video coding standards) including block structures, intra prediction techniques, inter prediction techniques, transform techniques, filtering techniques, and/or entropy coding techniques other than those included in ITU-T H.265.
  • ITU-T H.264 and ITU-T H.265 is for descriptive purposes and should not be construed to limit the scope of the techniques described herein. Further, it should be noted that incorporation by reference of documents herein should not be construed to limit or create ambiguity with respect to terms used herein. For example, in the case where an incorporated reference provides a different definition of a term than another incorporated reference and/or as the term is used herein, the term should be interpreted in a manner that broadly includes each respective definition and/or in a manner that includes each of the particular definitions in the alternative.
  • An aspect of the invention is a method of mapping a sample location to angular coordinates, the method comprising: determining whether a sample location belonging to a packed region is located in a guard band; and deriving a sample location on a projected picture based on whether the sample location is located in a guard band.
  • FIG. 1 is a block diagram illustrating an example of a system that may be configured to transmit coded video data according to one or more techniques of this disclosure.
  • FIG. 2A is a conceptual diagram illustrating coded video data and corresponding data structures according to one or more techniques of this disclosure.
  • FIG. 2B is a conceptual diagram illustrating coded video data and corresponding data structures according to one or more techniques of this disclosure.
  • FIG. 3 is a conceptual diagram illustrating coded video data and corresponding data structures according to one or more techniques of this disclosure.
  • FIG. 4 is a conceptual diagram illustrating an example of processing stages that may be used to derive a packed frame from a spherical projection structure according to one or more techniques of this disclosure.
  • FIG. 1 is a block diagram illustrating an example of a system that may be configured to transmit coded video data according to one or more techniques of this disclosure.
  • FIG. 2A is a conceptual diagram illustrating coded video data and corresponding data structures according to one or more techniques of this disclosure.
  • FIG. 5 is a block diagram illustrating an example of components that may be included in an implementation of a system that may be configured to distribute coded video data according to one or more techniques of this disclosure.
  • FIG. 6A is conceptual diagram illustrating examples of a projected picture region and a packed picture according to one or more techniques of this disclosure.
  • FIG. 6B is conceptual diagram illustrating examples of a projected picture region and a packed picture according to one or more techniques of this disclosure.
  • FIG. 6C is conceptual diagram illustrating examples of a projected picture region and a packed picture according to one or more techniques of this disclosure.
  • FIG. 7 is a block diagram illustrating an example of a receiver device that may implement one or more techniques of this disclosure.
  • Video content typically includes video sequences comprised of a series of frames.
  • a series of frames may also be referred to as a group of pictures (GOP).
  • Each video frame or picture may include one or more slices, where a slice includes a plurality of video blocks.
  • a video block may be defined as the largest array of pixel values (also referred to as samples) that may be predictively coded.
  • Video blocks may be ordered according to a scan pattern (e.g., a raster scan).
  • a video encoder performs predictive encoding on video blocks and sub-divisions thereof.
  • ITU-T H.264 specifies a macroblock including 16 x 16 luma samples.
  • ITU-T H.265 specifies an analogous Coding Tree Unit (CTU) structure where a picture may be split into CTUs of equal size and each CTU may include Coding Tree Blocks (CTB) having 16 x 16, 32 x 32, or 64 x 64 luma samples.
  • CTU Coding Tree Block
  • the term video block may generally refer to an area of a picture or may more specifically refer to the largest array of pixel values that may be predictively coded, sub-divisions thereof, and/or corresponding coding parameters and/or structures.
  • each video frame or picture may be partitioned to include one or more tiles, where a tile is a sequence of coding tree units corresponding to a rectangular area of a picture.
  • the CTBs of a CTU may be partitioned into Coding Blocks (CB) according to a corresponding quadtree block structure.
  • CB Coding Blocks
  • one luma CB together with two corresponding chroma CBs and associated syntax elements are referred to as a coding unit (CU).
  • a CU is associated with a prediction unit (PU) structure defining one or more prediction units (PU) for the CU, where a PU is associated with corresponding reference samples.
  • PU prediction unit
  • PU prediction unit
  • a PU may include luma and chroma prediction blocks (PBs), where square PBs are supported for intra prediction and rectangular PBs are supported for inter prediction.
  • Intra prediction data e.g., intra prediction mode syntax elements
  • inter prediction data e.g., motion data syntax elements
  • Residual data may include respective arrays of difference values corresponding to each component of video data (e.g., luma (Y) and chroma (Cb and Cr)). Residual data may be in the pixel domain.
  • a transform such as, a discrete cosine transform (DCT), a discrete sine transform (DST), an integer transform, a wavelet transform, or a conceptually similar transform, may be applied to pixel difference values to generate transform coefficients.
  • DCT discrete cosine transform
  • DST discrete sine transform
  • an integer transform e.g., a wavelet transform, or a conceptually similar transform
  • CUs may be further sub-divided into Transform Units (TUs).
  • an array of pixel difference values may be sub-divided for purposes of generating transform coefficients (e.g., four 8 x 8 transforms may be applied to a 16 x 16 array of residual values corresponding to a 16 x16 luma CB), such sub-divisions may be referred to as Transform Blocks (TBs).
  • Transform coefficients may be quantized according to a quantization parameter (QP).
  • Quantized transform coefficients (which may be referred to as level values) may be entropy coded according to an entropy encoding technique (e.g., content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), etc.).
  • CAVLC content adaptive variable length coding
  • CABAC context adaptive binary arithmetic coding
  • PIPE probability interval partitioning entropy coding
  • syntax elements such as, a syntax element indicating a prediction mode, may also be entropy coded. Entropy encoded quantized transform coefficients and corresponding entropy encoded syntax elements may form a compliant bitstream that can be used to reproduce video data.
  • a binarization process may be performed on syntax elements as part of an entropy coding process. Binarization refers to the process of converting a syntax value into a series of one or more bits. These bits may be referred to as “bins.”
  • VR applications may include video content that may be rendered with a head-mounted display, where only the area of the spherical video that corresponds to the orientation of the user’s head is rendered.
  • VR applications may be enabled by omnidirectional video, which is also referred to as 360 degree spherical video or 360 degree video.
  • Omnidirectional video is typically captured by multiple cameras that cover up to 360 degrees of a scene.
  • a distinct feature of omnidirectional video compared to normal video is that, typically only a subset of the entire captured video region is displayed, i.e., the area corresponding to the current user’s field of view (FOV) is displayed.
  • a FOV is sometimes also referred to as viewport.
  • a viewport may be part of the spherical video that is currently displayed and viewed by the user. It should be noted that the size of the viewport can be smaller than or equal to the field of view.
  • omnidirectional video may be captured using monoscopic or stereoscopic cameras. Monoscopic cameras may include cameras that capture a single view of an object. Stereoscopic cameras may include cameras that capture multiple views of the same object (e.g., views are captured using two lenses at slightly different angles). Further, it should be noted that in some cases, images for use in omnidirectional video applications may be captured using ultra wide-angle lens (i.e., so-called fisheye lens).
  • the process for creating 360 degree spherical video may be generally described as stitching together input images and projecting the stitched together input images onto a three-dimensional structure (e.g., a sphere or cube), which may result in so-called projected frames. Further, in some cases, regions of projected frames may be transformed, resized, and relocated, which may result in a so-called packed frame.
  • a three-dimensional structure e.g., a sphere or cube
  • a region in an omnidirectional video picture may refer to a subset of the entire video region. It should be noted that regions of an omnidirectional video may be determined by the intent of a director or producer, or derived from user statistics by a service or content provider (e.g., through the statistics of which regions have been requested/seen by the most users when the omnidirectional video content was provided through a streaming service). For example, for an omnidirectional video capturing a sporting event, a region may be defined for a view including the center of the playing field and other regions may be defined for views of the stands in a stadium.
  • Regions may be used for data pre-fetching in omnidirectional video adaptive streaming by edge servers or clients, and/or transcoding optimization when an omnidirectional video is transcoded, e.g., to a different codec or projection mapping.
  • signaling regions in an omnidirectional video picture may improve system performance by lowering transmission bandwidth and lowering decoding complexity.
  • Choi et al., ISO/IEC JTC1/SC29/WG11 N16824, “ISO/IEC 23090-2 Omnidirectional Media Format,” April 2017, Hobart, AU, which is incorporated by reference and herein referred to as Choi, defines a media application format that enables omnidirectional media applications.
  • Choi specifies a list of projection techniques that can be used for conversion of a spherical or 360 degree video into a two-dimensional rectangular video; how to store omnidirectional media and the associated metadata using the International Organization for Standardization (ISO) base media file format (ISOBMFF); how to encapsulate, signal, and stream omnidirectional media using dynamic adaptive streaming over Hypertext Transfer Protocol (HTTP) (DASH); and which video and audio coding standards, as well as media coding configurations, may be used for compression and playback of the omnidirectional media signal.
  • ISO International Organization for Standardization
  • each video frame or picture may be partitioned to include one or more slices and further partitioned to include one or more tiles.
  • FIGS. 2A-2B are conceptual diagrams illustrating an example of a group of pictures including slices and further partitioning pictures into tiles.
  • Pic 4 is illustrated as including two slices (i.e., Slice 1 and Slice 2 ) where each slice includes a sequence of CTUs (e.g., in raster scan order).
  • Pic 4 is illustrated as including six tiles (i.e., Tile 1 to Tile 6 ), where each tile is rectangular and includes a sequence of CTUs.
  • a tile may consist of coding tree units contained in more than one slice and a slice may consist of coding tree units contained in more than one tile.
  • ITU-T H.265 provides that one or both of the following conditions shall be fulfilled: (1) All coding tree units in a slice belong to the same tile; and (2) All coding tree units in a tile belong to the same slice.
  • each of the tiles may belong to a respective slice (e.g., Tile 1 to Tile 6 may respectively belong to slices, Slice 1 to Slice 6 ) or multiple tiles may belong to a slice (e.g., Tile 1 to Tile 3 may belong to Slice 1 and Tile 4 to Tile 6 may belong to Slice 2 ).
  • tiles may form tile sets (i.e., Tile 2 and Tile 5 form a tile set).
  • Tile sets may be used to define boundaries for coding dependencies (e.g., intra-prediction dependencies, entropy encoding dependencies, etc.) and as such, may enable parallelism in coding.
  • coding dependencies e.g., intra-prediction dependencies, entropy encoding dependencies, etc.
  • the tile set formed by Tile 2 and Tile 5 may correspond to a visual region including a news anchor reading the news.
  • ITU-T H.265 defines signaling that enables motion-constrained tile sets (MCTS).
  • a motion-constrained tile set may include a tile set for which inter-picture prediction dependencies are limited to the collocated tile sets in reference pictures.
  • motion compensation may be performed on Tile 2 and Tile 5 independent of coding Tile 1 , Tile 3 , Tile 4 , and Tile 6 in Pic 4 and tiles collocated with tiles Tile 1 , Tile 3 , Tile 4 , and Tile 6 in each of Pic 1 to Pic 3 .
  • Coding video data according to MCTS may be useful for video applications including omnidirectional video presentations.
  • tiles may form a region of an omnidirectional video.
  • the tile set formed by Tile 2 and Tile 5 may be a MCTS included within the region.
  • Viewport dependent video coding which may also be referred to as viewport dependent partial video coding, may be used to enable coding of only part of an entire video region. That is, for example, view port dependent video coding may be used to provide sufficient information for rendering of a current FOV.
  • omnidirectional video may be coded using MCTS, such that each potential region covering a viewport can be independently coded from other regions across time.
  • a minimum set of tiles that cover a viewport may be sent to the client, decoded, and/or rendered. This process may be referred to as simple tile based partial decoding (STPD).
  • STPD simple tile based partial decoding
  • the 360 degree video includes Region A, Region B, and Region C.
  • each of the regions are illustrated as including CTUs.
  • CTUs may form slices of coded video data and/or tiles of video data.
  • video coding techniques may code areas of a picture according to video blocks, sub-divisions thereof, and/or corresponding structures and it should be noted that video coding techniques enable video coding parameters to be adjusted at various levels of a video coding structure, e.g., adjusted for slices, tiles, video blocks, and/or at sub-divisions.
  • the 360 degree video illustrated in FIG. 3 may represent a sporting event where Region A and Region C include views of the stands of a stadium and Regions B includes a view of the playing field (e.g., the video is captured by a 360 degree camera placed at the 50-yard line).
  • regions of omnidirectional video may include regions on a sphere.
  • Choi describes where a region on a sphere may be specified by four great circles, where a great circle (also referred to as a Riemannian circle) is an intersection of the sphere and a plane that passes through the center point of the sphere, where the center of the sphere and the center of a great circle are co-located.
  • Choi further describes where a region on a sphere may be specified by two yaw circles and two pitch circles, where a yaw circle is a circle on the sphere connecting all points with the same yaw value, and pitch circle is a circle on the sphere connecting all points with the same pitch value.
  • Choi specifies a list of projection techniques that can be used for conversion of a spherical or 360 degree video into a two-dimensional rectangular video.
  • Choi specifies where a projected frame is a frame that has a representation format by a 360 degree video projection indicator and where a projection is the process by which a set of input images are projected onto a projected frame.
  • Choi specifies where a projection structure includes a three-dimensional structure including one or more surfaces on which the captured image/video content is projected, and from which a respective projected frame can be formed.
  • Choi provides where a region-wise packing includes a region-wise transformation, resizing, and relocating of a projected frame and where a packed frame is a frame that results from region-wise packing of a projected frame.
  • the process for creating 360 degree spherical video may be described as including image stitching, projection, and region-wise packing.
  • Choi specifies a coordinate system, omnidirectional projection formats, including an equirectangular projection, a rectangular region-wise packing format, and an omnidirectional fisheye video format, for the sake of brevity, a complete description of all of these sections of Choi is not provided herein. However, reference is made to the relevant sections of Choi.
  • Choi provides where the projection structure is a unit sphere, the coordinate system can be used for example to indicate the orientation of the projection structure or a spherical location of a point, and the coordinate axes used for defining yaw ( ⁇ ), pitch ( ⁇ ), and roll angles, where yaw rotates around the Y (vertical, up) axis, pitch around the x (lateral, side-to-side) axis, and roll around the Z (back-to-front) axis. Further, Choi provides where rotations are extrinsic, i.e., around the X, Y, and Z fixed reference axes and the angles increase clockwise when looking from the origin towards the positive end of an axis.
  • Choi further provides the following definitions for a projection structure and coordinate system in Clause 5.1: YawAngle indicates the rotation angle around the Y axis, in degrees.
  • RollAngle indicates the rotation angle around the Z axis, in degrees.
  • Choi provides the following in Clause 5.2: Equirectangular projection for one sample Inputs to this clause are: pictureWidth and pictureHeight, which are the width and height, respectively, of the equirectangular panorama picture in samples, and the center point of a sample location (i, j) along horizontal and vertical axes, respectively.
  • Outputs of this clause are: angular coordinates ( ⁇ , ⁇ ) for the sample in degrees relative to the coordinate axes specified in ⁇ Clause 5.1 Projection structure and coordinate system of Choi ⁇ .
  • Choi provides the following in Clause 5.3: Conversion between spherical coordinate systems of different orientations
  • Inputs to this clause are: angular coordinates ( ⁇ , ⁇ ) relative to the coordinate system specified in ⁇ Clause 5.1 Projection structure and coordinate system of Choi ⁇ , and orientation change yaw_center (in the range of -180, inclusive, to 180, exclusive), pitch_center (in the range of -90, inclusive, to 90, inclusive), roll_center (in the range of -180, inclusive, to 180, exclusive), all in units of degrees.
  • Outputs of this clause are: angular coordinates ( ⁇ ', ⁇ ') relative to the coordinate system specified in ⁇ Clause 5.1 Projection structure and coordinate system of Choi ⁇
  • the outputs are derived as follows:
  • FIG. 4 illustrates conversions from a spherical projection structure to a packed picture that can be used in content authoring and the corresponding conversions from a packed picture to a spherical projection structure that can be used in content rendering. It should be note that the example illustrated in FIG. 4 is based on an informative example provided in Choi. However, the example illustrated in FIG. 4 may be generally applicable and should not be construed to limit the scope of techniques for mapping sample locations to angular coordinates described herein.
  • the projection structure is along a global coordinate axes as illustrated in (a), when the equator of the equirectangular panorama picture is aligned with the X axis of the global coordinate axes, the Y axis of the equirectangular panorama picture is aligned with the Y axis of the global coordinate axes, and the Z axis of the global coordinate axes passes through the middle point of the equirectangular panorama picture.
  • content authoring may include one or more the following: rotating a projection structure relative to the global coordinate axes, as illustrated in (b); indicating the coverage as an area enclosed by two yaw circles and two pitch circles, where the yaw and pitch circles may be indicted relative the local coordinate axes; determining a projection picture (or frame); and obtaining a packed picture from a projection picture (e.g., by applying region-wise packing).
  • (c) illustrates an example coverage that is constrained only by two pitch circles while yaw values are not constrained.
  • the coverage corresponds to a rectangle (i.e., (d) in FIG. 4 indicates the 2D correspondence of (c)), where the X and Y axes of the 2D representation may be aligned with the X and Y local coordinate axes of the projection structure.
  • the projected picture may include a portion of the coverage.
  • the projected picture in (e) includes a portion of the coverage illustrated in (d), which may be specified using horizontal and vertical range values.
  • in (f) the side regions are horizontally down sampled, while the middle region is kept at its original resolution.
  • a computing device may perform sequential mappings in reverse order from (f) to (a). That is, a video decoding device may map the luma sample locations within a decoded picture to angular coordinates relative to global coordinate axes.
  • Choi if region-wise packing is not applied, the packed frame is identical to the projected frame. Otherwise, regions of the projected frame are mapped onto a packed frame by indicating the location, shape, and size of each region in the packed frame.
  • the input images of one time instance are stitched to generate a projected frame representing two views, one for each eye. Both views can be mapped onto the same packed frame and encoded by a traditional two-dimensional video encoder.
  • Choi provides, where each view of the projected frame can be mapped to its own packed frame, in which case, the image stitching, projection, and region-wise packing is similar to the monoscopic case described above.
  • a sequence of packed frames of either the left view or the right view can be independently coded or, when using a multiview video encoder, predicted from the other view.
  • the image stitching, projection, and region-wise packing process can be carried out multiple times for the same source images to create different versions of the same content, e.g. for different orientations of the projection structure and similarly, the region-wise packing process can be performed multiple times from the same projected frame to create more than one sequence of packed frames to be encoded.
  • Choi specifies a file format that generally supports the following types of metadata: (1) metadata specifying the projection format of the projected frame; (2) metadata specifying the area of the spherical surface covered by the projected frame; (3) metadata specifying the orientation of the projection structure corresponding to the projected frame in a global coordinate system; (4) metadata specifying region-wise packing information; and (5) metadata specifying optional region-wise quality ranking.
  • unsigned int(n) refers to an unsigned integer having n-bits.
  • bit(n) refers to a bit value having n-bits.
  • Choi specifies how to store omnidirectional media and the associated metadata using the International Organization for Standardization (ISO) base media file format (ISOBMFF). Further, Choi specifies where the file format supports the following types of boxes: a scheme type box (SchemeTypeBox), a scheme information box (SchemeInformationBox), a projected omnidirectional video box (ProjectedOmnidirectionalVideoBox), a stereo video box (StereoVideoBox), a fisheye omnidirectional video box (FisheyeOmnidirectionalVideoBox), a region-wise packing box (RegionWisePackingBox), and a projection orientation box (ProjectionOrientationBox).
  • SchemeTypeBox Scheme type box
  • SchemeInformationBox Scheme information box
  • ProjectedOmnidirectionalVideoBox projected omnidirectional video box
  • StepoVideoBox stereo video box
  • StepoVideoBox fisheye omnidirectional video box
  • RegionWisePackingBox a region-wise packing box
  • ProjectionOrientationBox ProjectionOrientationBox
  • Choi specifies additional types boxes, for the sake of brevity, a complete description of all the type of boxes specified in Choi are not described herein.
  • SchemeTypeBox SchemeInformationBox, ProjectedOmnidirectionalVideoBox, StereoVideoBox, and RegionWisePackingBox
  • Choi provides the following: -
  • the use of the projected omnidirectional video scheme for the restricted video sample entry type 'resv' indicates that the decoded pictures are packed pictures containing either monoscopic or stereoscopic content.
  • the use of the projected omnidirectional video scheme is indicated by scheme_type equal to 'podv' (projected omnidirectional video) within the SchemeTypeBox.
  • the use of the fisheye omnidirectional video scheme for the restricted video sample entry type 'resv' indicates that the decoded pictures are fisheye video pictures.
  • the use of the fisheye omnidirectional video scheme is indicated by scheme_type equal to 'fodv' (fisheye omnidirectional video) within the SchemeTypeBox.
  • -The format of the projected monoscopic pictures is indicated with the ProjectedOmnidirectionalVideoBox contained within the SchemeInformationBox.
  • the format of fisheye video is indicated with the FisheyeOmnidirectionalVideoBox contained within the SchemeInformationBox.
  • One and only one ProjectedOmnidirectionalVideoBox shall be present in the SchemeInformationBox when the scheme type is 'podv'.
  • One and only one FisheyeOmnidirectionalVideoBox shall be present in the SchemeInformationBox when the scheme type is 'fodv'.
  • StereoVideoBox and RegionWisePackingBox may be present in the same SchemeInformationBox.
  • StereoVideoBox and RegionWisePackingBox shall not be present in the same SchemeInformationBox.
  • the frame packing arrangement of the projected left and right pictures is indicated with the StereoVideoBox contained within the SchemeInformationBox.
  • StereoVideoBox indicates that the omnidirectionally projected content of the track is monoscopic.
  • stereo_scheme When StereoVideoBox is present in the SchemeInformationBox for the omnidirectional video scheme, stereo_scheme shall be equal to 4 and stereo_indication_type shall indicate that either the top-bottom frame packing or the side-by-side frame packing is in use and that quincunx sampling is not in use.
  • stereo_indication_type shall indicate that either the top-bottom frame packing or the side-by-side frame packing is in use and that quincunx sampling is not in use.
  • -Optional region-wise packing is indicated with the RegionWisePackingBox contained within the SchemeInformationBox. The absence of RegionWisePackingBox indicates that no region-wise packing is applied.
  • projection_type indicates the particular mapping of the rectangular decoder picture output samples onto the spherical coordinate system specified in ⁇ Clause 5.1 Projection structure and coordinate system of Choi ⁇ .
  • projection_type equal to 0 indicates the equirectangular projection as specified in ⁇ Clause 5.2 Omnidirectional projection formats of Choi ⁇ Other values of projection_type are reserved.
  • RegionWisePackingBox indicates that projected frames are packed region-wise and require unpacking prior to rendering.
  • num_regions specifies the number of packed regions. Value 0 is reserved.
  • proj_picture_width and proj_picture_height specify the width and height, respectively, of the projected picture. proj_picture_width and proj_picture_height shall be greater than 0.
  • guard_band_flag[i] 0 specifies that the i-th region does not have a guard band.
  • guard_band_flag[i] 1 specifies that the i-th region has a guard band.
  • packing_type[i] specifies the type of region-wise packing.
  • packing_type[i] 0 indicates rectangular region-wise packing. Other values are reserved.
  • left_gb_width[i] specifies the width of the guard band on the left side of the i-th region in units of two luma samples.
  • right_gb_width[i] specifies the width of the guard band on the right side of the i-th region in units of two luma samples.
  • top_gb_width[i] specifies the height of the guard band above the i-th region in units of two luma samples.
  • bottom_gb_width[i] specifies the height of the guard band below the i-th region in units of two luma samples.
  • guard_band_flag[i] When guard_band_flag[i] is equal to 1, left_gb_width[i], right_gb_width[i], top_gb_width[i], or bottom_gb_width[i] shall be greater than 0.
  • gb_not_used_for_pred_flag[i] 0 specifies that the guard bands may or may not be used in the inter prediction process.
  • gb_not_used_for_pred_flag[i] 1 specifies that the sample values of the guard bands are not in the inter prediction process.
  • gb_type[i] specifies the type of the guard bands for the i-th region as follows: - gb_type[i] equal to 0 specifies that the content of the guard bands in relation to the content of the regions is unspecified. gb_type shall not be equal to 0, when gb_not_used_for_pred_flag is equal to 0. - gb_type[i] equal to 1 specifies that the content of the guard bands suffices for interpolation of sub-pixel values within the region and less than one pixel outside of the region boundary. NOTE 2: gb_type equal to 1 can be used when the boundary samples of a region have been copied horizontally or vertically to the guard band.
  • - gb_type[i] 2 specifies that the content of the guard bands represents actual image content at quality that gradually changes from the picture quality of the region to that of the spherically adjacent region.
  • - gb_type[i] 3 specifies that the content of the guard bands represents actual image content at the picture quality of the region.
  • - gb_type[i] values greater than 3 are reserved.
  • proj_reg_width[i], proj_reg_height[i], proj_reg_top[i] and proj_reg_left[i] are indicated in units of pixels in a projected picture with width and height equal to proj_picture_width and proj_picture_height, respectively.
  • proj_reg_width[i] specifies the width of the i-th region of the projected picture. proj_reg_width[i] shall be greater than 0. proj_reg_height[i] specifies the height of the i-th region of the projected picture. proj_reg_height[i] shall be greater than 0. proj_reg_top[i] and proj_reg_left[i] specify the top sample row and the left-most sample column in the projected picture. The values shall be in the range from 0, inclusive, indicating the top-left corner of the projected picture, to proj_picture_height and proj_picture_width, exclusive, respectively.
  • proj_reg_width[i] and proj_reg_left[i] shall be constrained such that proj_reg_width[i] + proj_reg_left[i] is less than proj_picture_width.
  • proj_reg_height[i] and proj_reg_top[i] shall be constrained such that proj_reg_height[i] + proj_reg_top[i] is less than proj_picture_height.
  • proj_reg_width[i], proj_reg_height[i], proj_reg_top[i] and proj_reg_left[i] shall be such that the region identified by these fields on the projected picture is within a single constituent picture of the projected picture.
  • transform_type[i] specifies the rotation and mirroring that has been applied to the i-th region of a projected picture to map it to the packed picture before encoding.
  • transform_type[i] specifies both rotation and mirroring, rotation has been applied after mirroring in the region-wise packing from the projected picture to the packed picture before encoding.
  • packed_reg_width[i], packed_reg_height[i], packed_reg_top[i], and packed_reg_left[i] specify the width, height, the top sample row, and the left-most sample column, respectively, of the region in the packed picture.
  • top_gb_width[i] may instead be called top_gb_height[i] and bottom_gb_width[i] may instead be called bottom_gb_height[i].
  • orientation_yaw, orientation_pitch, and orientation_roll specify the yaw, pitch, and roll angles, respectively, of the center point of the projected picture when projected to the spherical surface, in units of 2 -16 degrees relative to the global coordinate axes.
  • orientation_yaw shall be in the range of -180 * 2 16 to 180 *2 16 - 1, inclusive.
  • orientation_pitch shall be in the range of -90 * 2 16 to 90 * 2 16 , inclusive.
  • orientation_roll shall be in the range of -180 * 2 16 to 180 * 2 16 - 1, inclusive.
  • a computing device may map luma sample locations within a picture to angular coordinates relative to global coordinate axes.
  • Choi provides the following in Clause 7.2.2.2: Mapping of luma sample locations within a decoded picture to angular coordinates relative to the global coordinate axes
  • the width and height of a monoscopic projected luma picture (pictureWidth and pictureHeight, respectively) are derived as follows:
  • the variables HorDiv and VerDiv are derived as follows: If StereoVideoBox is absent, HorDiv and VerDiv are set equal to 1.
  • StereoVideoBox is present and indicates side-by-side frame packing
  • HorDiv is set equal to 2 and VerDiv is set equal to 1. Otherwise (StereoVideoBox is present and indicates top-bottom frame packing), HorDiv is set equal to 1 and VerDiv is set equal to 2.
  • RegionWisePackingBox is absent, pictureWidth and pictureHeight are set to be equal to width / HorDiv and height / VerDiv, respectively, where width and height are syntax elements of VisualSampleEntry. Otherwise, pictureWidth and pictureHeight are set equal to proj_picture_width and proj_picture_height, respectively.
  • RegionWisePackingBox is present, the following applies for each region n in the range of 0 to num_regions: For each sample location (xPackedPicture, yPackedPicture) belonging to the n-th region with packing_type[n] equal to 0 (i.e., with rectangular region-wise packing), the following applies: The corresponding sample location (xProjPicture, yProjPicture) of the projected picture is derived as follows: x is set equal to xPackedPicture - packed_reg_left[n]. y is set equal to yPackedPicture - packed_reg_top[n]. offsetX is set equal to 0.5. offsetY is set equal to 0.5.
  • ⁇ Clause 5.4 Conversion of sample locations for rectangular region-wise packing of Choi ⁇ is invoked with x, y, packed_reg_width[n], packed_reg_height[n], proj_reg_width[n], proj_reg_height[n], transform_type[n], offsetX and offsetY as inputs, and the output is assigned to sample location (i, j).
  • xProjPicture is set equal to proj_reg_left[n] + i.
  • xProjPicture is greater than or equal to proj_picture_width
  • xProjPicture is set equal to xProjPicture - proj_picture_width.
  • yProjPicture is set equal to proj_reg_top[n] + j. ⁇ Clause 7.2.2.3 Conversion from a sample location in a projected picture to angular coordinates relative to the global coordinate axes of Choi ⁇ is invoked with xProjPicture, yProjPicture, pictureWidth, and pictureHeight as inputs, and the outputs indicating the angular coordinates and the constituent frame index (for frame-packed stereoscopic video) for the luma sample location (xPackedPicture, yPackedPicture) belonging to the n-th region in the decoded picture.
  • xProjPicture is set equal to x + 0.5.
  • yProjPicture is set equal to y + 0.5.
  • Conversion from a sample location in a projected picture to angular coordinates relative to the global coordinate axes of Choi ⁇ is invoked with xProjPicture, yProjPicture, pictureWidth, and pictureHeight as inputs, and the outputs indicating the angular coordinates and the constituent frame index (for frame-packed stereoscopic video) for the sample location (x, y) within the decoded picture.
  • Outputs of this clause are: angular coordinates (yawGlobal, pitchGlobal), in units of degrees relative to the global coordinate axes, and when StereoVideoBox is present, the index of the constituent picture (constituentPicture) equal to 0 or 1.
  • the outputs are derived with the following ordered steps: If xProjPicture is greater than or equal to pictureWidth or yProjPicture is greater than pictureHeight, the following applies: constituentPicture is set equal to 1. If xProjPicture is greater than or equal to pictureWidth, xProjPicture is set to xProjPicture - pictureWidth.
  • yProjPicture is greater than or equal to pictureHeight, yProjPicture is set to yProjPicture - pictureHeight. Otherwise, constituentPicture is set equal to 0. ⁇ Clause 5.2.1 Equirectangular projection for one sample of Choi ⁇ is invoked with pictureWidth, pictureHeight, xProjPicture, and yProjPicture as inputs, and the output is assigned to yawLocal, pitchLocal.
  • ProjectionOrientationBox is present, clause [5.3 Conversion between spherical coordinate systems of different orientations] is invoked with yawLocal, pitchLocal, orientation_yaw ⁇ 2 16 , orientation_pitch ⁇ 2 16 , and orientation_roll ⁇ 2 16 as inputs, and the output is assigned to yawGlobal and pitchGlobal. Otherwise, yawGlobal is set equal to yawLocal and pitchGlobal is set equal to pitchLocal.
  • the techniques for signaling information associated with region-wise packing and for mapping of luma sample locations within a decoded picture to angular coordinates relative to the global coordinate axes provided in Choi may be less than ideal.
  • FIG. 1 is a block diagram illustrating an example of a system that may be configured to code (i.e., encode and/or decode) video data according to one or more techniques of this disclosure.
  • System 100 represents an example of a system that may encapsulate video data according to one or more techniques of this disclosure.
  • system 100 includes source device 102, communications medium 110, and destination device 120.
  • source device 102 may include any device configured to encode video data and transmit encoded video data to communications medium 110.
  • Destination device 120 may include any device configured to receive encoded video data via communications medium 110 and to decode encoded video data.
  • Source device 102 and/or destination device 120 may include computing devices equipped for wired and/or wireless communications and may include, for example, set top boxes, digital video recorders, televisions, desktop, laptop or tablet computers, gaming consoles, medical imagining devices, and mobile devices, including, for example, smartphones, cellular telephones, personal gaming devices.
  • Communications medium 110 may include any combination of wireless and wired communication media, and/or storage devices.
  • Communications medium 110 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that may be useful to facilitate communications between various devices and sites.
  • Communications medium 110 may include one or more networks.
  • communications medium 110 may include a network configured to enable access to the World Wide Web, for example, the Internet.
  • a network may operate according to a combination of one or more telecommunication protocols. Telecommunications protocols may include proprietary aspects and/or may include standardized telecommunication protocols.
  • Examples of standardized telecommunications protocols include Digital Video Broadcasting (DVB) standards, Advanced Television Systems Committee (ATSC) standards, Integrated Services Digital Broadcasting (ISDB) standards, Data Over Cable Service Interface Specification (DOCSIS) standards, Global System Mobile Communications (GSM) standards, code division multiple access (CDMA) standards, 3rd Generation Partnership Project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards.
  • DVD Digital Video Broadcasting
  • ATSC Advanced Television Systems Committee
  • ISDB Integrated Services Digital Broadcasting
  • DOCSIS Data Over Cable Service Interface Specification
  • GSM Global System Mobile Communications
  • CDMA code division multiple access
  • 3GPP 3rd Generation Partnership Project
  • ETSI European Telecommunications Standards Institute
  • IP Internet Protocol
  • WAP Wireless Application Protocol
  • IEEE Institute of Electrical and Electronics Engineers
  • Storage devices may include any type of device or storage medium capable of storing data.
  • a storage medium may include a tangible or non-transitory computer-readable media.
  • a computer readable medium may include optical discs, flash memory, magnetic memory, or any other suitable digital storage media.
  • a memory device or portions thereof may be described as non-volatile memory and in other examples portions of memory devices may be described as volatile memory.
  • Examples of volatile memories may include random access memories (RAM), dynamic random access memories (DRAM), and static random access memories (SRAM).
  • Examples of non-volatile memories may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories.
  • Storage device(s) may include memory cards (e.g., a Secure Digital (SD) memory card), internal/external hard disk drives, and/or internal/external solid state drives. Data may be stored on a storage device according to a defined file format
  • FIG. 5 is a conceptual drawing illustrating an example of components that may be included in an implementation of system 100.
  • system 100 includes one or more computing devices 402A-402N, television service network 404, television service provider site 406, wide area network 408, local area network 410, and one or more content provider sites 412A-412N.
  • the implementation illustrated in FIG. 5 represents an example of a system that may be configured to allow digital media content, such as, for example, a movie, a live sporting event, etc., and data and applications and media presentations associated therewith to be distributed to and accessed by a plurality of computing devices, such as computing devices 402A-402N.
  • digital media content such as, for example, a movie, a live sporting event, etc.
  • computing devices 402A-402N such as computing devices 402A-402N.
  • computing devices 402A-402N may include any device configured to receive data from one or more of television service network 404, wide area network 408, and/or local area network 410.
  • computing devices 402A-402N may be equipped for wired and/or wireless communications and may be configured to receive services through one or more data channels and may include televisions, including so-called smart televisions, set top boxes, and digital video recorders.
  • computing devices 402A-402N may include desktop, laptop, or tablet computers, gaming consoles, mobile devices, including, for example, “smart” phones, cellular telephones, and personal gaming devices.
  • Television service network 404 is an example of a network configured to enable digital media content, which may include television services, to be distributed.
  • television service network 404 may include public over-the-air television networks, public or subscription-based satellite television service provider networks, and public or subscription-based cable television provider networks and/or over the top or Internet service providers.
  • television service network 404 may primarily be used to enable television services to be provided, television service network 404 may also enable other types of data and services to be provided according to any combination of the telecommunication protocols described herein.
  • television service network 404 may enable two-way communications between television service provider site 406 and one or more of computing devices 402A-402N.
  • Television service network 404 may comprise any combination of wireless and/or wired communication media.
  • Television service network 404 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that may be useful to facilitate communications between various devices and sites.
  • Television service network 404 may operate according to a combination of one or more telecommunication protocols.
  • Telecommunications protocols may include proprietary aspects and/or may include standardized telecommunication protocols. Examples of standardized telecommunications protocols include DVB standards, ATSC standards, ISDB standards, DTMB standards, DMB standards, Data Over Cable Service Interface Specification (DOCSIS) standards, HbbTV standards, W3C standards, and UPnP standards.
  • DOCSIS Data Over Cable Service Interface Specification
  • television service provider site 406 may be configured to distribute television service via television service network 404.
  • television service provider site 406 may include one or more broadcast stations, a cable television provider, or a satellite television provider, or an Internet-based television provider.
  • television service provider site 406 may be configured to receive a transmission including television programming through a satellite uplink/downlink.
  • television service provider site 406 may be in communication with wide area network 408 and may be configured to receive data from content provider sites 412A-412N. It should be noted that in some examples, television service provider site 406 may include a television studio and content may originate therefrom.
  • Wide area network 408 may include a packet based network and operate according to a combination of one or more telecommunication protocols.
  • Telecommunications protocols may include proprietary aspects and/or may include standardized telecommunication protocols. Examples of standardized telecommunications protocols include Global System Mobile Communications (GSM) standards, code division multiple access (CDMA) standards, 3 rd Generation Partnership Project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, European standards (EN), IP standards, Wireless Application Protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards, such as, for example, one or more of the IEEE 802 standards (e.g., Wi-Fi).
  • GSM Global System Mobile Communications
  • CDMA code division multiple access
  • 3GPP 3 rd Generation Partnership Project
  • ETSI European Telecommunications Standards Institute
  • EN European standards
  • IP standards European standards
  • WAP Wireless Application Protocol
  • IEEE Institute of Electrical and Electronics Engineers
  • Wide area network 408 may comprise any combination of wireless and/or wired communication media.
  • Wide area network 480 may include coaxial cables, fiber optic cables, twisted pair cables, Ethernet cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that may be useful to facilitate communications between various devices and sites.
  • wide area network 408 may include the Internet.
  • Local area network 410 may include a packet based network and operate according to a combination of one or more telecommunication protocols. Local area network 410 may be distinguished from wide area network 408 based on levels of access and/or physical infrastructure. For example, local area network 410 may include a secure home network.
  • content provider sites 412A-412N represent examples of sites that may provide multimedia content to television service provider site 406 and/or computing devices 402A-402N.
  • a content provider site may include a studio having one or more studio content servers configured to provide multimedia files and/or streams to television service provider site 406.
  • content provider sites 412A-412N may be configured to provide multimedia content using the IP suite.
  • a content provider site may be configured to provide multimedia content to a receiver device according to Real Time Streaming Protocol (RTSP), HTTP, or the like.
  • RTSP Real Time Streaming Protocol
  • content provider sites 412A-412N may be configured to provide data, including hypertext based content, and the like, to one or more of receiver devices computing devices 402A-402N and/or television service provider site 406 through wide area network 408.
  • Content provider sites 412A-412N may include one or more web servers. Data provided by data provider site 412A-412N may be defined according to data formats.
  • source device 102 includes video source 104, video encoder 106, data encapsulator 107, and interface 108.
  • Video source 104 may include any device configured to capture and/or store video data.
  • video source 104 may include a video camera and a storage device operably coupled thereto.
  • Video encoder 106 may include any device configured to receive video data and generate a compliant bitstream representing the video data.
  • a compliant bitstream may refer to a bitstream that a video decoder can receive and reproduce video data therefrom. Aspects of a compliant bitstream may be defined according to a video coding standard. When generating a compliant bitstream video encoder 106 may compress video data. Compression may be lossy (discernible or indiscernible to a viewer) or lossless.
  • data encapsulator 107 may receive encoded video data and generate a compliant bitstream, e.g., a sequence of NAL units according to a defined data structure.
  • a device receiving a compliant bitstream can reproduce video data therefrom.
  • conforming bitstream may be used in place of the term compliant bitstream.
  • data encapsulator 107 may be configured to signal information associated with region-wise packing according to one or more techniques described herein. It should be noted that data encapsulator 107 need not necessary be located in the same physical device as video encoder 106. For example, functions described as being performed by video encoder 106 and data encapsulator 107 may be distributed among devices illustrated in FIG. 5.
  • data encapsulator 107 may be configured to signal a RegionWisePackingBox based on the syntax provided above and the following example semantics: Semantics num_regions specifies the number of packed regions. Value 0 is reserved. proj_picture_width and proj_picture_height specify the width and height, respectively, of the projected picture. proj_picture_width and proj_picture_height shall be greater than 0. guard_band_flag[i] equal to 0 specifies that the i-th packed region does not have a guard band. guard_band_flag[i] equal to 1 specifies that the i-th packed region has a guard band.
  • packing_type[i] specifies the type of region-wise packing. packing_type[i] equal to 0 indicates rectangular region-wise packing. Other values are reserved.
  • left_gb_width[i] specifies the width of the guard band on the left side of the i-th packed region in units of two luma samples. When left_gb_width[i] is not present it is inferred to be equal to 0.
  • right_gb_width[i] specifies the width of the guard band on the right side of the i-th packed region in units of two luma samples. When right_gb_width[i] is not present it is inferred to be equal to 0.
  • top_gb_width[i] specifies the height of the guard band above the i-th packed region in units of two luma samples. When top_gb_width[i] is not present it is inferred to be equal to 0.
  • bottom_gb_width[i] specifies the height of the guard band below the i-th packed region in units of two luma samples. When bottom_gb_width[i] is not present it is inferred to be equal to 0.
  • guard_band_flag[i] is equal to 1, left_gb_width[i], right_gb_width[i], top_gb_width[i], or bottom_gb_width[i] shall be greater than 0.
  • gb_not_used_for_pred_flag[i] 0 specifies that the guard bands may or may not be used in the inter prediction process.
  • gb_not_used_for_pred_flag[i] 1 specifies that the sample values of the guard bands are not in the inter prediction process.
  • NOTE 1 When gb_not_used_for_pred_flag[i] is equal to 1, the sample values within guard bands in decoded pictures can be rewritten even if the decoded pictures were used as references for inter prediction of subsequent pictures to be decoded.
  • gb_type[i] specifies the type of the guard bands for the i-th packed region as follows: - gb_type[i] equal to 0 specifies that the content of the guard bands in relation to the content of the regions is unspecified. gb_type shall not be equal to 0, when gb_not_used_for_pred_flag is equal to 0. - gb_type[i] equal to 1 specifies that the content of the guard bands suffices for interpolation of sub-pixel values within the region and less than one pixel outside of the region boundary.
  • gb_type 1 can be used when the boundary samples of a region have been copied horizontally or vertically to the guard band.
  • - gb_type[i] 2 specifies that the content of the guard bands represents actual image content at quality that gradually changes from the picture quality of the region to that of the spherically adjacent region.
  • - gb_type[i] 3 specifies that the content of the guard bands represents actual image content at the picture quality of the region.
  • - gb_type[i] values greater than 3 are reserved.
  • proj_reg_width[i], proj_reg_height[i], proj_reg_top[i] and proj_reg_left[i] are indicated in units of pixels in a projected picture with width and height equal to proj_picture_width and proj_picture_height, respectively.
  • proj_reg_width[i] specifies the width of the i-th region of the projected picture. proj_reg_width[i] shall be greater than 0.
  • proj_reg_height[i] specifies the height of the i-th region of the projected picture. proj_reg_height[i] shall be greater than 0.
  • proj_reg_top[i] and proj_reg_left[i] specify the top sample row and the left-most sample column in the projected picture.
  • the values shall be in the range from 0, inclusive, indicating the top-left corner of the projected picture, to proj_picture_height-1 and proj_picture_width-1, exclusive, respectively.
  • proj_reg_width[i] and proj_reg_left[i] shall be constrained such that proj_reg_width[i] + proj_reg_left[i] is less than proj_picture_width.
  • proj_reg_height[i] and proj_reg_top[i] shall be constrained such that proj_reg_height[i] + proj_reg_top[i] is less than proj_picture_height.
  • proj_reg_height[i] proj_reg_top[i]
  • proj_reg_left[i] shall be such that the region identified by these fields on the projected picture is within a single constituent picture of the projected picture.
  • transform_type[i] specifies the rotation and mirroring that has been applied to the i-th region of a projected picture to map it to the packed picture before encoding.
  • transform_type[i] specifies both rotation and mirroring, rotation has been applied after mirroring in the region-wise packing from the projected picture to the packed picture before encoding.
  • the following values are specified and other values are reserved: 0: no transform 1: mirroring horizontally 2: rotation by 180 degrees (counter-clockwise) 3: rotation by 180 degrees (counter-clockwise) after mirroring horizontally 4: rotation by 90 degrees (counter-clockwise) after mirroring horizontally 5: rotation by 90 degrees (counter-clockwise) 6: rotation by 270 degrees (counter-clockwise) after mirroring horizontally 7: rotation by 270 degrees (counter-clockwise)
  • NOTE 3 Clause 5.4 Conversion of sample locations for rectangular region-wise packing of Choi may specify the semantics of transform_type[i] for converting a sample location of a region in a packed picture to a sample location of a region in a projected picture.
  • packed_reg_width[i], packed_reg_height[i], packed_reg_top[i], and packed_reg_left[i] specify the width, height, the top sample row, and the left-most sample column, respectively, of the region in the packed picture.
  • packed_reg_width[i] shall be greater than or equal to 2*(left_gb_width[i] + right_gb_width[i]).
  • packed_reg_height[i] shall be greater than or equal to 2*(top_gb_width[i] + bottom_gb_width[i]).
  • inference rules are defined for left_gb_width[i], right_gb_width[i], top_gb_width[i], and bottom_gb_width[i] when they are not signaled; syntax elements guard_band_flag[i], left_gb_width[i], right_gb_width[i], top_gb_width[i], bottom_gb_width[i], and gb_type[i] are modified with respect to Choi to clarify that they apply to packed region; and constraints are defined for packed_reg_width[i] and packed_reg_height [i] with respect to the guard band signaling, such that invalid values are not allowed to be signaled.
  • a packed region may be a region on a packed picture.
  • FIG. 6A illustrates elements proj_reg_width[i], proj_reg_height[i], proj_reg_top[i], proj_reg_left[i] for one of the regions on the projected picture and syntax elements proj_picture_width and proj_picture_height for the projected picture.
  • 6B illustrates syntax elements packed_reg_width[i], packed_reg_height[i], packed_reg_top[i], packed_reg_left[i], left_gb_width[i], right_gb_width[i], top_gb_width[i], bottom_gb_width[i] for one of the regions on the packed picture for one example.
  • 6C illustrates syntax elements packed_reg_width[i], packed_reg_height[i], packed_reg_top[i], packed_reg_left[i], left_gb_width[i], right_gb_width[i], top_gb_height[i], bottom_gb_height[i] for one of the regions on the packed picture for one example.
  • top_gb_width[i] may instead be called top_gb_height[i]
  • bottom_gb_width[i] may instead be called bottom_gb_height[i].
  • top_gb_width[i] is referred to as top_gb_height[i] and bottom_gb_width[i] is referred to as bottom_gb_height[i].
  • FIG. 6B illustrates an example where guard bands are considered inside of a packed region
  • FIG. 6C illustrates an example where guard bands are considered outside of a packed region.
  • guard bands may be considered on the packed picture. According to the techniques described herein, a sample may be determined to be located in a guard band regardless of whether a guard band is considered inside or outside a packed region.
  • packed_reg_width[i] shall be greater than or equal to 2*(left_gb_width[i] + right_gb_width[i]).
  • packed_reg_height[i] shall be greater than or equal to 2*(top_gb_width[i] + bottom_gb_width[i]).
  • data encapsulator 107 represents an example of a device configured to signal information associated with region-wise packing.
  • interface 108 may include any device configured to receive data generated by data encapsulator 107 and transmit and/or store the data to a communications medium.
  • Interface 108 may include a network interface card, such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can send and/or receive information.
  • interface 108 may include a computer system interface that may enable a file to be stored on a storage device.
  • interface 108 may include a chipset supporting Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Express (PCIe) bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, I 2 C, or any other logical and physical structure that may be used to interconnect peer devices.
  • PCI Peripheral Component Interconnect
  • PCIe Peripheral Component Interconnect Express
  • USB Universal Serial Bus
  • destination device 120 includes interface 122, data decapsulator 123, video decoder 124, and display 126.
  • Interface 122 may include any device configured to receive data from a communications medium.
  • Interface 122 may include a network interface card, such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can receive and/or send information.
  • interface 122 may include a computer system interface enabling a compliant video bitstream to be retrieved from a storage device.
  • interface 122 may include a chipset supporting PCI and PCIe bus protocols, proprietary bus protocols, USB protocols, I 2 C, or any other logical and physical structure that may be used to interconnect peer devices.
  • Data decapsulator 123 may be configured to receive a bitstream and metadata generated by data encaspulator 107 and perform a reciprocal decapsulation process.
  • Video decoder 124 may include any device configured to receive a bitstream and/or acceptable variations thereof and reproduce video data therefrom.
  • Display 126 may include any device configured to display video data.
  • Display 126 may comprise one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display.
  • Display 126 may include a High Definition display or an Ultra High Definition display.
  • Display 126 may include a stereoscopic display. It should be noted that although in the example illustrated in FIG. 1, video decoder 124 is described as outputting data to display 126, video decoder 124 may be configured to output video data to various types of devices and/or sub-components thereof. For example, video decoder 124 may be configured to output video data to any communication medium, as described herein. Destination device 120 may include a receiver device.
  • conversions from a spherical projection structure to a packed picture may be used in content authoring and corresponding conversions from a packed picture to a spherical projection structure may be used in content rendering.
  • the mapping of luma sample locations within a decoded picture to angular coordinates relative to the global coordinate axes provided in Choi may be less than ideal.
  • source device 102 may be configured to perform conversions from a spherical picture to a packed picture and destination device 120 may be configured to perform conversions from packed picture to a spherical picture according to the techniques for mapping luma sample locations to angular coordinates relative to global coordinate axes described below:
  • the width and height of a monoscopic projected luma picture (pictureWidth and pictureHeight, respectively) are derived as follows:
  • the variables HorDiv and VerDiv are derived as follows: If StereoVideoBox is absent, HorDiv and VerDiv are set equal to 1. Otherwise, if StereoVideoBox is present and indicates side-by-side frame packing, HorDiv is set equal to 2 and VerDiv is set equal to 1.
  • StepVideoBox is present and indicates top-bottom frame packing
  • HorDiv is set equal to 1 and VerDiv is set equal to 2.
  • pictureWidth and pictureHeight are set to be equal to width / HorDiv and height / VerDiv, respectively, where width and height are syntax elements of VisualSampleEntry. Otherwise, pictureWidth and pictureHeight are set equal to proj_picture_width/ HorDiv and proj_picture_height/ VerDiv, respectively.
  • RegionWisePackingBox is present, when guard bands are included inside of a packed region as shown in example FIG.
  • y is set equal to yPackedPicture - packed_reg_top[n].
  • offsetX is set equal to 0.5.
  • offsetY is set equal to 0.5.
  • xProjPicture is set equal to proj_reg_left[n] + i.
  • yProjPicture is set equal to proj_reg_top[n] + j.
  • this area of the n-th packed region corresponds to the four guard bands on four sides (left, right, bottom, top).
  • packing_type[n] 0 (i.e., with rectangular region-wise packing)
  • xPackedPicture outside the range of (packed_reg_left[n] + 2*left_gb_width[n]) to (packed_reg_left[n] + packed_reg_width[n] - 2*right_gb_width[n]-1), inclusive and/or with yPackedPicture outside the range of (packed_reg_top[n] + 2*top_gb_width[n]) to (packed_reg_top[n] + packed_reg_height[n] - 2*bottom_gb_width[n]-1), inclusive no sample location is derived on the projected picture.
  • xProjPicture is set equal to x + 0.5.
  • yProjPicture is set equal to y + 0.5.
  • mapping of luma sample locations to angular coordinates relative to global coordinate axes is applied only to part of the packed region outside the guard bands and sample locations in the guard band of the packed region are not used for derivation of sample locations on the projected picture.
  • each sample location the following may be applied in mapping of luma sample locations within a decoded picture region to angular coordinates: o For each sample location (xProjPicture, yProjPicture) belonging to the n-th projected region with packing_type[n] equal to 0 (i.e., with rectangular region-wise packing), with n in the range of 0 to num_regions which is not in the packed picture, Clause 7.2.2.3 Conversion from a sample location in a projected picture to angular coordinates relative to the global coordinate axes of Choi may be invoked with xProjPicture, yProjPicture, pictureWidth, and pictureHeight as inputs, and the outputs indicating the angular coordinates and the constituent frame index (for frame-packed stereoscopic video) for the luma sample location (xProjPicture, yProjPicture) belonging to the n-th region in the projected picture.
  • destination device 120 represents an example of a device configured to determine whether a sample location belonging to a packed region is located in a guard band and deriving a sample location on a projected picture based on whether the sample location is located in a guard band.
  • FIG. 7 is a block diagram illustrating an example of a receiver device that may implement one or more techniques of this disclosure. That is, receiver device 600 may be configured to parse a signal based on the semantics described above and/or perform mapping of mapping sample locations in a packed frame to angular coordinates of a projection structure.
  • Receiver device 600 is an example of a computing device that may be configured to receive data from a communications network and allow a user to access multimedia content, including a virtual reality application.
  • receiver device 600 is configured to receive data via a television network, such as, for example, television service network 404 described above. Further, in the example illustrated in FIG. 7, receiver device 600 is configured to send and receive data via a wide area network. It should be noted that in other examples, receiver device 600 may be configured to simply receive data through a television service network 404.
  • the techniques described herein may be utilized by devices configured to communicate using any and all combinations of communications networks.
  • receiver device 600 includes central processing unit(s) 602, system memory 604, system interface 610, data extractor 612, audio decoder 614, audio output system 616, video decoder 618, display system 620, I/O device(s) 622, and network interface 624.
  • system memory 604 includes operating system 606 and applications 608.
  • Each of central processing unit(s) 602, system memory 604, system interface 610, data extractor 612, audio decoder 614, audio output system 616, video decoder 618, display system 620, I/O device(s) 622, and network interface 624 may be interconnected (physically, communicatively, and/or operatively) for inter-component communications and may be implemented as any of a variety of suitable circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable gate arrays
  • receiver device 600 is illustrated as having distinct functional blocks, such an illustration is for descriptive purposes and does not limit receiver device 600 to a particular hardware architecture. Functions of receiver device 600 may be realized using any combination of hardware, firmware and/or software implementations.
  • CPU(s) 602 may be configured to implement functionality and/or process instructions for execution in receiver device 600.
  • CPU(s) 602 may include single and/or multi-core central processing units.
  • CPU(s) 602 may be capable of retrieving and processing instructions, code, and/or data structures for implementing one or more of the techniques described herein. Instructions may be stored on a computer readable medium, such as system memory 604.
  • System memory 604 may be described as a non-transitory or tangible computer-readable storage medium. In some examples, system memory 604 may provide temporary and/or long-term storage. In some examples, system memory 604 or portions thereof may be described as non-volatile memory and in other examples portions of system memory 604 may be described as volatile memory. System memory 604 may be configured to store information that may be used by receiver device 600 during operation. System memory 604 may be used to store program instructions for execution by CPU(s) 602 and may be used by programs running on receiver device 600 to temporarily store information during program execution. Further, in the example where receiver device 600 is included as part of a digital video recorder, system memory 604 may be configured to store numerous video files.
  • Applications 608 may include applications implemented within or executed by receiver device 600 and may be implemented or contained within, operable by, executed by, and/or be operatively/communicatively coupled to components of receiver device 600. Applications 608 may include instructions that may cause CPU(s) 602 of receiver device 600 to perform particular functions. Applications 608 may include algorithms which are expressed in computer programming statements, such as, for-loops, while-loops, if-statements, do-loops, etc. Applications 608 may be developed using a specified programming language. Examples of programming languages include, Java TM , Jini TM , C, C++, Objective C, Swift, Perl, Python, PhP, UNIX Shell, Visual Basic, and Visual Basic Script.
  • receiver device 600 includes a smart television
  • applications may be developed by a television manufacturer or a broadcaster.
  • applications 608 may execute in conjunction with operating system 606. That is, operating system 606 may be configured to facilitate the interaction of applications 608 with CPUs(s) 602, and other hardware components of receiver device 600.
  • Operating system 606 may be an operating system designed to be installed on set-top boxes, digital video recorders, televisions, and the like. It should be noted that techniques described herein may be utilized by devices configured to operate using any and all combinations of software architectures.
  • System interface 610 may be configured to enable communications between components of receiver device 600.
  • system interface 610 comprises structures that enable data to be transferred from one peer device to another peer device or to a storage medium.
  • system interface 610 may include a chipset supporting Accelerated Graphics Port (AGP) based protocols, Peripheral Component Interconnect (PCI) bus based protocols, such as, for example, the PCI Express TM (PCIe) bus specification, which is maintained by the Peripheral Component Interconnect Special Interest Group, or any other form of structure that may be used to interconnect peer devices (e.g., proprietary bus protocols).
  • AGP Accelerated Graphics Port
  • PCI Peripheral Component Interconnect
  • PCIe PCI Express TM
  • PCIe Peripheral Component Interconnect Special Interest Group
  • receiver device 600 is configured to receive and, optionally, send data via a television service network.
  • a television service network may operate according to a telecommunications standard.
  • a telecommunications standard may define communication properties (e.g., protocol layers), such as, for example, physical signaling, addressing, channel access control, packet properties, and data processing.
  • data extractor 612 may be configured to extract video, audio, and data from a signal.
  • a signal may be defined according to, for example, aspects DVB standards, ATSC standards, ISDB standards, DTMB standards, DMB standards, and DOCSIS standards.
  • Data extractor 612 may be configured to extract video, audio, and data, from a signal. That is, data extractor 612 may operate in a reciprocal manner to a service distribution engine. Further, data extractor 612 may be configured to parse link layer packets based on any combination of one or more of the structures described above.
  • Audio decoder 614 may be configured to receive and process audio packets.
  • audio decoder 614 may include a combination of hardware and software configured to implement aspects of an audio codec. That is, audio decoder 614 may be configured to receive audio packets and provide audio data to audio output system 616 for rendering.
  • Audio data may be coded using multi-channel formats such as those developed by Dolby and Digital Theater Systems. Audio data may be coded using an audio compression format. Examples of audio compression formats include Motion Picture Experts Group (MPEG) formats, Advanced Audio Coding (AAC) formats, DTS-HD formats, and Dolby Digital (AC-3) formats.
  • MPEG Motion Picture Experts Group
  • AAC Advanced Audio Coding
  • DTS-HD formats DTS-HD formats
  • AC-3 formats Dolby Digital
  • Audio output system 616 may be configured to render audio data.
  • audio output system 616 may include an audio processor, a digital-to-analog converter, an amplifier, and a speaker system.
  • a speaker system may include any of a variety of speaker systems, such as headphones, an integrated stereo speaker system, a multi-speaker system, or a surround sound system.
  • Video decoder 618 may be configured to receive and process video packets.
  • video decoder 618 may include a combination of hardware and software used to implement aspects of a video codec.
  • video decoder 618 may be configured to decode video data encoded according to any number of video compression standards, such as ITU-T H.262 or ISO/IEC MPEG-2 Visual, ISO/IEC MPEG-4 Visual, ITU-T H.264 (also known as ISO/IEC MPEG-4 Advanced video Coding (AVC)), and High-Efficiency Video Coding (HEVC).
  • Display system 620 may be configured to retrieve and process video data for display. For example, display system 620 may receive pixel data from video decoder 618 and output data for visual presentation.
  • display system 620 may be configured to output graphics in conjunction with video data, e.g., graphical user interfaces.
  • Display system 620 may comprise one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device capable of presenting video data to a user.
  • a display device may be configured to display standard definition content, high definition content, or ultra-high definition content.
  • I/O device(s) 622 may be configured to receive input and provide output during operation of receiver device 600. That is, I/O device(s) 622 may enable a user to select multimedia content to be rendered. Input may be generated from an input device, such as, for example, a push-button remote control, a device including a touch-sensitive screen, a motion-based input device, an audio-based input device, or any other type of device configured to receive user input. I/O device(s) 622 may be operatively coupled to receiver device 600 using a standardized communication protocol, such as for example, Universal Serial Bus protocol (USB), Bluetooth, ZigBee or a proprietary communications protocol, such as, for example, a proprietary infrared communications protocol.
  • USB Universal Serial Bus protocol
  • ZigBee ZigBee
  • proprietary communications protocol such as, for example, a proprietary infrared communications protocol.
  • Network interface 624 may be configured to enable receiver device 600 to send and receive data via a local area network and/or a wide area network.
  • Network interface 624 may include a network interface card, such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device configured to send and receive information.
  • Network interface 624 may be configured to perform physical signaling, addressing, and channel access control according to the physical and Media Access Control (MAC) layers utilized in a network.
  • Receiver device 600 may be configured to parse a signal generated according to any of the techniques described above.
  • receiver device 600 represents an example of a device configured parse one or more syntax elements including information associated with a virtual reality application
  • the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit.
  • Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol.
  • computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave.
  • Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure.
  • a computer program product may include a computer-readable medium.
  • such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.
  • any connection is properly termed a computer-readable medium.
  • a computer-readable medium For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium.
  • DSL digital subscriber line
  • Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
  • processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • processors may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein.
  • the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques could be fully implemented in one or more circuits or logic elements.
  • the techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set).
  • IC integrated circuit
  • a set of ICs e.g., a chip set.
  • Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit or provided by a collection of interoperative hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.
  • each functional block or various features of the base station device and the terminal device used in each of the aforementioned embodiments may be implemented or executed by a circuitry, which is typically an integrated circuit or a plurality of integrated circuits.
  • the circuitry designed to execute the functions described in the present specification may comprise a general-purpose processor, a digital signal processor (DSP), an application specific or general application integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic, or a discrete hardware component, or a combination thereof.
  • the general-purpose processor may be a microprocessor, or alternatively, the processor may be a conventional processor, a controller, a microcontroller or a state machine.
  • the general-purpose processor or each circuit described above may be configured by a digital circuit or may be configured by an analogue circuit. Further, when a technology of making into an integrated circuit superseding integrated circuits at the present time appears due to advancement of a semiconductor technology, the integrated circuit by this technology is also able to be used.
  • a method for mapping a sample location to angular coordinates comprises determining whether a sample location belonging to a packed region is located in a guard band, and deriving a sample location on a projected picture based on whether the sample location is located in a guard band.
  • a device comprises one or more processors configured to determine whether a sample location belonging to a packed region is located in a guard band, and deriving a sample location on a projected picture based on whether the sample location is located in a guard band.
  • a non-transitory computer-readable storage medium comprises instructions stored thereon that, when executed, cause one or more processors of a device to determine whether a sample location belonging to a packed region is located in a guard band, and deriving a sample location on a projected picture based on whether the sample location is located in a guard band.
  • an apparatus comprises means for determining whether a sample location belonging to a packed region is located in a guard band, and means for deriving a sample location on a projected picture based on whether the sample location is located in a guard band.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un procédé de mise en correspondance d'un emplacement d'échantillon avec des coordonnées angulaires. Le procédé consiste : à déterminer si un emplacement d'échantillon appartenant à une zone conditionnée est situé dans une bande de garde ; et à déduire un emplacement d'échantillon sur une image projetée selon que l'emplacement d'échantillon soit situé ou non dans une bande de garde.
PCT/JP2018/017698 2017-05-15 2018-05-08 Systèmes et procédés de mise en correspondance d'emplacements d'échantillons avec des coordonnées angulaires dans des applications de réalité virtuelle WO2018212009A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201762506536P 2017-05-15 2017-05-15
US62/506,536 2017-05-15
US201762518509P 2017-06-12 2017-06-12
US62/518,509 2017-06-12

Publications (1)

Publication Number Publication Date
WO2018212009A1 true WO2018212009A1 (fr) 2018-11-22

Family

ID=64274287

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/017698 WO2018212009A1 (fr) 2017-05-15 2018-05-08 Systèmes et procédés de mise en correspondance d'emplacements d'échantillons avec des coordonnées angulaires dans des applications de réalité virtuelle

Country Status (1)

Country Link
WO (1) WO2018212009A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113674188A (zh) * 2021-08-04 2021-11-19 深圳中兴网信科技有限公司 视频分析方法、装置、电子设备和可读存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6141034A (en) * 1995-12-15 2000-10-31 Immersive Media Co. Immersive imaging method and apparatus
WO2001095513A1 (fr) * 2000-06-09 2001-12-13 Imove Inc. Enregistrement et lecture en continu d'une video panoramique
JP2012142852A (ja) * 2011-01-05 2012-07-26 Jvc Kenwood Corp 擬似立体画像生成装置、擬似立体画像生成プログラム及び擬似立体画像表示装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6141034A (en) * 1995-12-15 2000-10-31 Immersive Media Co. Immersive imaging method and apparatus
WO2001095513A1 (fr) * 2000-06-09 2001-12-13 Imove Inc. Enregistrement et lecture en continu d'une video panoramique
JP2012142852A (ja) * 2011-01-05 2012-07-26 Jvc Kenwood Corp 擬似立体画像生成装置、擬似立体画像生成プログラム及び擬似立体画像表示装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BYEONGDOO CHOI ET AL.: "WD on ISO/IEC 23000-20 Omnidirectional Media Application Format", ISO/IEC JTC1/SC29/WG11, N16189, 30 June 2016 (2016-06-30), XP055517901 *
QUALCOMM INCORPORATED: "FS_VR: OMAF Progress", 3GPP TSG SA WG4 #93 S4-170324, 28 April 2017 (2017-04-28), XP051259739 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113674188A (zh) * 2021-08-04 2021-11-19 深圳中兴网信科技有限公司 视频分析方法、装置、电子设备和可读存储介质

Similar Documents

Publication Publication Date Title
US20200275129A1 (en) Systems and methods for signaling information associated with a constituent picture
WO2019189038A1 (fr) Systèmes et procédés de signalisation d'informations de paramètres de caméra
US20200120326A1 (en) Systems and methods for signaling view information for virtual reality applications
WO2020045593A1 (fr) Systèmes et procédés de signalisation d'informations de métadonnées synchronisées à une sous-image
US10880617B2 (en) Systems and methods for signaling quality information for regions in virtual reality applications
WO2019139014A1 (fr) Systèmes et procédés de signalisation d'informations de composition d'image secondaire pour des applications de réalité virtuelle
WO2019146601A1 (fr) Systèmes et procédés pour la signalisation d'information de position
WO2019194241A1 (fr) Systèmes et procédés de signalisation d'informations de composition d'image secondaire pour des applications de réalité virtuelle
US10848735B2 (en) Systems and methods for signaling information associated with constituent pictures in virtual reality applications
WO2020184645A1 (fr) Systèmes et procédés de signalisation d'informations de point de vue dans du contenu multimédia omnidirectionnel
WO2019235305A1 (fr) Systèmes et procédés de signalisation d'informations de superposition
WO2018212009A1 (fr) Systèmes et procédés de mise en correspondance d'emplacements d'échantillons avec des coordonnées angulaires dans des applications de réalité virtuelle
US20200221104A1 (en) Systems and methods for signaling a projected region for virtual reality applications
WO2018179903A1 (fr) Systèmes et procédés d'indication d'informations associées aux zones les plus intéressantes pour les applications de réalité virtuelle
WO2021125117A1 (fr) Systèmes et procédés pour signaler des informations pour un mesh dans un support omnidirectionnel
WO2021137300A1 (fr) Systèmes et procédés de signalisation d'informations de commutation de point de vue dans du contenu multimédia omnidirectionnel
WO2021125185A1 (fr) Systèmes et procédés de signalisation d'informations de bouclage de point de vue dans du contenu multimédia omnidirectionnel
WO2021075407A1 (fr) Systèmes et procédés pour permettre une interactivité pour des emplacements actionnables dans des moyens multimédia omnidirectionnels
US20230421828A1 (en) Systems and methods for signaling content component information in omnidirectional media
WO2019203102A1 (fr) Systèmes et procédés de signalisation de messages spécifiques à une application dans une application de suivi de réalité virtuelle
WO2019139052A1 (fr) Systèmes et procédés de signalisation d'informations de source pour des applications de réalité virtuelle
WO2020141604A1 (fr) Systèmes et procédés de signalisation d'informations de paramètres de caméra
WO2018179843A1 (fr) Systèmes et procédés de signalisation d'informations pour des applications de réalité virtuelle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18801566

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18801566

Country of ref document: EP

Kind code of ref document: A1