WO2019083943A1 - Projection de carte cubemap angulaire hybride de codage vidéo à 360 degrés - Google Patents

Projection de carte cubemap angulaire hybride de codage vidéo à 360 degrés

Info

Publication number
WO2019083943A1
WO2019083943A1 PCT/US2018/057015 US2018057015W WO2019083943A1 WO 2019083943 A1 WO2019083943 A1 WO 2019083943A1 US 2018057015 W US2018057015 W US 2018057015W WO 2019083943 A1 WO2019083943 A1 WO 2019083943A1
Authority
WO
WIPO (PCT)
Prior art keywords
transform function
parameter
frame
face
function parameter
Prior art date
Application number
PCT/US2018/057015
Other languages
English (en)
Inventor
Fanyi DUANMU
Yuwen He
Yan Ye
Philippe HANHART
Original Assignee
Vid Scale, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vid Scale, Inc. filed Critical Vid Scale, Inc.
Publication of WO2019083943A1 publication Critical patent/WO2019083943A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process

Definitions

  • VR Virtual reality
  • VR has many application areas, including healthcare, education, social networking, industry design/training, game, movie, shopping, entertainment, etc.
  • VR is gaining attention from industries and consumers because VR is capable of bringing an immersive viewing experience.
  • VR creates a virtual environment surrounding the viewer and generates a true sense of "being there" for the viewer.
  • How to provide the full real feeling in the VR environment is important for a user's experience.
  • the VR system may support interactions through posture, gesture, eye gaze, voice, etc.
  • the VR may provide haptic feedback to the user.
  • a video coding device may be configured to receive a video with frames associated with a parameterized transform function. For a first frame of the video, the video coding device may search through a parameter space for a first transform function parameter associated with the parameterized transform function.
  • the first frame may be or may include a heading frame (e.g., an initial frame of the video, an instantaneous decoder reference (IDR) frame, and/or an initial frame of a video segment).
  • the device may determine a progressive search range relative to the first transform function parameter.
  • the progressive search range may include a portion of the parameter space that comprises the first transform function parameter and extends above and below the first transform function parameter.
  • the device may search for a second transform function parameter from within the progressive search range.
  • the second transform function parameter may be associated with the parametrized transform function for a second frame.
  • the second frame may be a frame subsequent to the first frame.
  • the video coding device may signal the first transform function parameter and the second transform function parameter in a video bitstream.
  • the parameterized transform function may be used to perform projection format conversion.
  • the parameter associated with the parameterized transform function may indicate the projection format for coding.
  • the parameterized transform function having the first transform function parameter may indicate a projection formation for coding of the first frame.
  • the parameterized transform function having the second transform function parameter may indicate another projection formation for coding of the second frame.
  • a frame may include one or more faces, and a parametrized transform function (e.g., a parameterized transform function having a certain transform function parameter) may be associated with a face.
  • a parameterized transform function e.g., a parameterized transform function having a certain transform function parameter
  • the video coding device may search for a transform function parameter associated with the parameterized transform function for a face and/or a direction. For example, the device may search for a transform function parameter associated with the parametrized transform function for the horizontal direction of a face. Also, or alternatively, the device may search for a transform function parameter associated with the parameterized transform function for the vertical direction of a face.
  • the video coding device may be configured to determine the size of a progressive search range.
  • the determined size of the progressive search range may be based on a desired coding efficiency level and/or a desired coding complexity. For example, as the size of the progressive search range increases, the coding efficiency level may decrease, and/or the complexity may increase.
  • FIG. 1 illustrates an example implementation of a 38G-degree video system
  • FIG. 2A illustrates an example three-dimensional (3D) geometric structure associated with cubemap projection (CMP),
  • FIG. 2B illustrates an example two-dimensional (2D) planner for six faces associated with CMP.
  • FIG. 3A illustrates an example uniform sampling of a cube face for CMP.
  • FIG. 3B illustrates an example non-uniform spherical sampling for CMP.
  • FIG. 3C illustrates an example non-uniform sampling of a cube face for unicube map projection
  • FIG. 3D illustrates an example uniform spherical sampling for UNICMP.
  • FIG 4A illustrates an example mapping from the non-uniform partition grid of a cube face to the uniform partition grid of a unicube face.
  • FIG 4B illustrates an example mapping from the uniform partition grid of a unicube face to the nonuniform partition grid of a cube face.
  • FIG. 5A illustrates a transform function of UNICMP example for one non-uniform partition on the cube face.
  • FIG. 5B illustrates a transform function of UNICMP example for the corresponding uniform partition on the sphere.
  • FIG. 5C illustrates a transform function of UNICMP example for a transform function between the coordinate of cube face ⁇ and the coordinate of unicube face ⁇ '.
  • FIG. 8A illustrates an example of mapping a cube face domain to a hybrid angular cube face domain.
  • FIG. 6B illustrates an example of a mapping from a hybrid angular cube face domain to a cube face domain.
  • FIG. 7 illustrates an example comparison of the transform function g(x') for CMP.
  • FIG. 8 illustrates an example 3x2 frame packing where the face boundary (bO, b1 , b2 and b3) connects two 3D neighboring faces.
  • FIG. 9A illustrates a hybrid angular cubemap (HAG) with 3x2 frame packing example without face boundary continuity constraint.
  • FIG. 9B illustrates a HAC with 3x2 frame packing example with a face boundary continuity constraint.
  • FIG. 10 illustrates an example HAC progressive search implementation.
  • FIG. 11 A illustrates a coarse-to-fine search (GFS) with a uniform step-size parameter coarse search.
  • FIG. 11 B illustrates a CFS example with a local parameter search refinement.
  • FIG. 11 G illustrates a CFS example with a recursive local parameter search refinement.
  • FIG. 12A is a system diagram of an example communications system in which one or more disclosed embodiments may be implemented.
  • FIG. 12B is a system diagram of an example wireless transmit/receive unit (WTRU) that may be used within the communications system illustrated in FIG. 12A.
  • WTRU wireless transmit/receive unit
  • FIG. 12C is a system diagram of an example radio access network (RAN) and an example core network (CN) that may be used within the communications system illustrated in FIG. 12A.
  • RAN radio access network
  • CN core network
  • FIG. 12D is a system diagram of further example RAN and a further example CN that may be used within the communications system illustrated in FIG. 12A.
  • VR systems may use 360-degree video to provide the users the capability to view the scene from 360- degree angles in the horizontal direction and 180-degree angles in the vertical direction.
  • VR and 360-degree video may be considered to be the direction for media consumption beyond Ultra High Definition (UHD) service.
  • UHD Ultra High Definition
  • Work on the requirements and potential technologies for omnidirectional media application format may be performed to improve the quality of 360-degree video in VR and/or to standardize the processing chain for client's interoperability.
  • Free view TV FTV may test the performance of one or more of the following: (1 ) 360- degree video (omnidirectional video) based system; (2) multi-view based system.
  • the quality and/or experience of one or more aspects in the VR processing chain may be improved.
  • the quality and/or experience of one or more aspects in capturing, processing, display, etc., VR processing may be improved.
  • VR may use one or more cameras to capture a scene from one or more ⁇ e.g., different) divergent views (e.g., 2-12 views). The views may be stitched together to form a 360-degree video in high resolution ⁇ e.g., 4K or 8K).
  • the virtual reality system may include a computation platform, head mounted display (HMD), and/or head tracking sensors.
  • the computation platform may be in charge of receiving and/or decoding 360-degree video, and/or generating the viewport for display.
  • Two pictures, one for each eye, may be rendered for the viewport.
  • the two pictures may be displayed in a HMD ⁇ e.g., for stereo viewing).
  • the lens may be used to magnify the image displayed in the HMD for better viewing.
  • the head tracking sensor may keep track (e.g., may constantly keep track) of the viewer's head orientation, and/or may feed the orientation information to the system to display the viewport picture for that orientation.
  • VR systems may provide a touch device for a viewer to interact with objects in the virtual world.
  • VR systems may be driven by a powerful workstation with good GPU support.
  • a light VR system e.g., Gear VR
  • the spatial HMD resolution may be 2160x1200
  • refresh rate may be 90Hz
  • the field of view (FOV) may be 110 degrees.
  • the sampling density for head tracking sensor may be 1000Hz, which may capture fast movement.
  • a VR system may include a lens and/or cardboard, and/or may be driven by smartphone.
  • the example workflow for 360-degree video system may include a 360-degree video capturing process which may use one or more cameras to capture videos covering the sphere (e.g., the entire sphere).
  • the videos may be stitched together in a native geometry structure.
  • the videos may be stitched together in an equirectanguiar projection (ERP) format
  • ERP equirectanguiar projection
  • the native geometry structure may be converted to one or more projection formats for encoding, based on the existing video codecs.
  • the video may be decoded, and/or the decompressed video may be converted to the geometry for display.
  • the video may be used for rendering via viewport projection according to users viewing angle.
  • 380-degree video information may be represented using a spherical geometry structure. For example, time-synchronized multiple views captured by multiple cameras may be stitched on the sphere as an integral structure (e.g. one integral structure). The spherical video information may be projected to a 2D planar surface with a given geometry conversion process.
  • Cube map projection of 360-degree video may be performed.
  • a 360-degree video may be compressed and/or delivered.
  • a 360-degree video delivery may include representing the 360-degree information using a sphere geometry structure. For example, synchronized views captured by one or more cameras may be stitched on the sphere as an integral structure.
  • the sphere geometry information may be projected on a 2D planar surface with a given geometry conversion.
  • FIG. 2 shows an example projective geometry of the CMP format.
  • the CMP may include one or more (e.g., 6) square faces, labeled as PX, PY, PZ, NX, NY, NZ, where P may stand for positive, N may stand for negative, and/or X, Y, Z may refer to the axes.
  • the faces may be labeled using numbers. For example, the faces may be labeled as 0-5 according to PX (0), NX (1), PY (2), NY (3), PZ (4), NZ (5). If the radius of the tangent sphere is 1 , the lateral length of each face may be 2.
  • FIG. 2B shows an example packing which may place the 6 faces into a rectangular picture.
  • a face index may be oriented in the direction that is aligned with the corresponding rotation of the face (e.g., to better aid in visualization).
  • face #3 and/or #1 may be rotated counter-clockwise by 270 and 180 degrees, respectively, while one or more (e.g., all) of the other faces may not rotated.
  • An example picture with CMP may be shown in FIG. 2C.
  • the resulting motion field (e.g., which may describe the temporal correlation between neighboring 2D projective pictures) generated by CMP may be represented (e.g., efficiently represented) by the translationai motion model of video codecs, for example, due to its rectilinear structure.
  • CMP may be supported on graphics hardware.
  • Unicube map projection for 38Q-degree video coding may be performed.
  • One or more samples on a sphere may be unevenly sampled by a CMP format with a higher sampling density near face boundaries and/or a lower sampling density near face centers.
  • Non-uniform spherical sampling may affect the efficiency of 360-degree video representation and/or may reduce the efficiency of 360- degree video coding, for example.
  • the non-uniform sampling of the CMP may result in the quality of the regions around face boundaries being higher than that of the regions around face centers.
  • One or more samples on the sphere may not have the same importance with respect to a viewer's visual experience. For example, viewers may be more likely to view the content near face centers than to view the content near face boundaries. Having different sampling densities may cause warping and/or deformation of an object as it moves from the center of the face to the face boundary ⁇ e.g., or vice versa) in the temporal domain.
  • a unicube map projection (UNICMP) format may be used.
  • the UNICMP may convert the sampling grid of the CMP into a uniform sampling grid on the sphere.
  • the UNICMP may use a transform function to modify the coordinates of the samples on a 2D planar face ⁇ e.g., before the actual UNICMP faces are generated).
  • UNICMP may improve the representation of spherical data, for example, due to its uniform spherical sampling.
  • FIG. 3 shows an example comparison of the planar and spherical sampling patterns between CMP and UNICMP.
  • the sampling grid of a CMP face may include one or more (e.g., two) sets of parallel lines.
  • One set of parallel lines may be in the horizontal direction and/or another set of parallel lines may be in the vertical direction.
  • a set of parallel partitioning lines may be separated with a uniform interval. For example, as illustrated in FIGs. 3A-3B, when a CMP face is projected onto a sphere, the sampling grid may be distorted where the straight lines in the planar face ⁇ e.g., in FIG. 3A) become curved ⁇ e.g., in FIG. 3B).
  • the sampling density at the boundary of a face may be higher than the sampling density at the center of a face ⁇ e.g., each face).
  • Rectilinear projection may not be a distance- preserving projection, and/or a corresponding sampling grid on the sphere may become non-uniform, as shown in FIG. 3B.
  • a face in UNICMP format may be sampled based on one or more ⁇ e.g., two) sets of parallel lines.
  • the parallel lines in a set may be distributed in a nonuniform way e.g., as illustrated in FIG. 3C), such that the corresponding sampling grid on the sphere may be uniform (e.g., as illustrated in FIG. 3D).
  • a transform function may be used to transform a non-uniform planar sampling grid into a uniform planar sampling grid.
  • the transform function of x and the transform function of y may be identical. Derivation of the transform function of y are discussed herein. Similar techniques may be used to derive the transform function of x.
  • FIG. 5 illustrates an example of how to calculate the transform functions between the coordinate of cube face and the coordinate of unicube face.
  • ⁇ e [-1, 1] be the coordinate of the pattern area on the cube.
  • ( ?) may be made proportional to the area of the spherical region corresponding to ⁇ .
  • the value of / ' ( ?) may be equal to the ratio between the area of the pattern spherical region and that of the quarter of the sphere corresponding to one cubemap face.
  • the transform function f >) may be calculated as: where ⁇ e [-1, 1].
  • the corresponding inverse transform function $(/?') ⁇ e.g., the mapping from the unicube face to cube face), may be calculated as: where ⁇ ' e [-1, 1].
  • FIG. 5C illustrates an example corresponding mapping relationship between ⁇ and ⁇ '.
  • CMP-like projections for 380-degree video coding may be performed.
  • CMP-like projection formats may achieve different spherical sampling features. This may be done by adjusting the coordinates of the cube face using one or more (e.g., one or more different) transform function(s)
  • ACP adjusted cubemap projection
  • sgn( ) may be a function which returns the sign of the input value.
  • EAC projection may convert the coordinates between the cube domain and the EAC domain based on the tangent of the angle of a (e.g., each) spherical sample on an EAC sampling grid.
  • the transform functions for an EAC projection may be:
  • a parameter which may be used for a transform function, may be searched for (e.g., searched for from a parameter search space).
  • a hybrid cubemap projection (HCP) format may be used.
  • the HCP format may be based on the ACP format.
  • the HCP format may use parameters (e.g., fixed parameters as seen in (3) and/or (4)).
  • the HCP format may use an encoder to choose an optimal sampling distribution based on an input video. For example, the encoder may choose an optimal sampling distribution via the selected HCP parameter(s).
  • the selected HCP parameters may be coded and may be sent in a bitstream (e.g., a video bitstream).
  • the HCP format may adjust the sampling distribution in the horizontal and/or vertical direction(s) (e.g., by adjusting the HCP parameters in the horizontal and/or vertical direction(s)).
  • the sampling distribution may be adjusted and applied for one or more faces (e.g., by adjusting the HCP parameters for each face).
  • a vertical HCP parameter(s) may be the same for one or more faces (e.g., all the faces) in a face row (e.g., each face row).
  • a horizontal HCP parameter(s) may be adjusted (e.g., adjusted separately) for a face (e.g., each face).
  • HCP parameters may be selected, for example, such that the end-to-end weighted conversion sum of square error (SSE) are minimized.
  • SSE square error
  • An example HCP parameter selection may be seen in (7).
  • F, (ERP) may be a portion of a source ERP, which may correspond to a face Fi in HCP
  • F, (ERP') may be the portion of reconstructed ERP corresponding to the face Fi in HGP
  • the parameters for HCP may be derived, for example, by an iterative search, which may be performed in one or more directions (e.g., two directions). For example, a search for parameters in first direction may be performed while fixing parameters for a second direction. The parameters for the first direction may be updated with the optimal parameters found by the search in the first direction, if there is no update, searching may stop. If there is an update, a search for parameters in the second direction may be performed while fixing parameters for the first direction. The parameters for the second direction may be updated with the optimal parameters found in searching, if there is no update, searching may stop.
  • an iterative search which may be performed in one or more directions (e.g., two directions). For example, a search for parameters in first direction may be performed while fixing parameters for a second direction. The parameters for the first direction may be updated with the optimal parameters found by the search in the first direction, if there is no update, searching may stop.
  • a parameter search in a direction may search for a parameter (e.g., an optimal parameter value) from a step value within a search range in the parameter domain.
  • a parameter e.g., an optimal parameter value
  • forward and/or backward projection format conversion [e.g., from input projection format to the hybrid cubemap projection back) may be performed to calculate Fj (ERP) and Fi (ERP') in (7).
  • a HCP format may update the HCP parameters for an [e.g., each) intra random access point (I RAP).
  • I RAP intra random access point
  • the HCP parameters between neighboring IRAPs may vary.
  • devices may support cubemap projection. This may be due to the computational simplicity and/or rectilinear spherical structure of CMP.
  • CMP may not evenly sample a spherical surface.
  • CMP may include a higher sampling density at face boundaries and/or a lower sampling density at face center.
  • the EAC, UNICMP and/or ACP formats may provide a sampling grid on a spherical surface, which may be uniform.
  • EAC, UNICMP, and/or ACP may maintain a face structure that may be similar to the CMP format. Further adjustments to the coordinates of the samples within the face(s) ⁇ e.g., within each face) may be made.
  • the frame-packing schemes and/or hardware implementations that are used in CMP may be used in EAC, UNICMP, and/or ACP.
  • a HCP projection format may be used.
  • ACP projection may include a fixed transform function for one or more directions in the faces (e.g., all faces).
  • a transform function ( f(x) ) of a HCP which may convert coordinates from a cubemap domain to a hybrid cubemap domain, may include a quadratic function.
  • the transform function may have limits for an input.
  • a transform function of an EAC format may be based on trigonometry functions, and/or may be a monotonic function.
  • the EAC format may be performed for ⁇ e.g., some) 360 ⁇ degree video coding tools, for example, geometric padding.
  • a hybrid projection format based on the EAC format may allow for the determination of (e.g., the dynamic determination of) an (e.g., optimal) EAC transform function(s). This determination may be based on an input video.
  • Optimization of a transform function may be based on the content of an input video. This optimization may be temporally updated, as content may change in the temporal domain. Transform functions may change a sampling distribution (e.g., spatial sampling distribution). Temporal correlation may be reduced if, for example, two pictures at two different time instances ⁇ e.g., the current picture and its reference picture) have different transform functions in the same face, A parameter search having a lower complexity may be used.
  • an EAC projection format may increase the sampling uniformity on a sphere.
  • a projection format of a 380-degree video may dynamically adjust the projection format (e.g., transform function) for a face (e.g., each individual face).
  • the characteristics of the spherical data for an (e.g., each) individual 360-degree video may be considered.
  • a (e.g., optimal) parametrized transform function may be adaptively selected for a direction and/or a face.
  • the transform function parameters of a direction and/or face may be adaptively selected.
  • HAC may use different parametrized transform functions (e.g., may use different transform function parameters for the parameterized transform function) for the x- and y- directions on a face.
  • HAC may use different transform functions for each face.
  • the parametrized transform function may be a tangent based parameterized transform function.
  • HAC may match the spherical sampling density of a face to the characteristics of its corresponding 3D content (e.g., match the transform function parameters of a face to the characteristics of its corresponding 3D content).
  • Syntax elements may be used to signal the geometric information for HAC in a bit-stream.
  • the geometric information may be used by coding tools (e.g., low-level coding tools).
  • the geometric information may be used by a post-processing module, which may convert a 360-degree video from 2D to 3D for a display.
  • a progressive HAC parameter search may be performed to reduce the number of search candidates.
  • a progressive HAC parameter search may limit the temporal variation of transform functions.
  • a constrained HAC parameter search may be performed such that neighboring faces (e.g., any two neighboring faces in a 3D geometry) may use similar sampling functions (e.g., the same sampling functions). For example, sampling functions may be similar (e.g., may be the same) in the directions parallel to the shared boundaries.
  • the vertical sampling functions for neighboring faces e.g., any three continuous neighboring faces within a row of a 3x2 frame packed picture
  • neighboring faces e.g., any two neighboring faces in a 3D geometry
  • similar sampling functions may be similar (e.g., may be the same) in the directions parallel to the shared boundaries.
  • the vertical sampling functions for neighboring faces e.g., any three continuous neighboring faces within a row of a 3x2 frame packed picture
  • HAC may transform a cubemap into another spherical mapping. This may be performed by using a function (e.g., a transform function) to modify the coordinates, which may occur, for example, before a CMP face is generated.
  • a function e.g., a transform function
  • CMP-like projection formats may perform similar transform functions (e.g., the same transform function) in the x- and y-directions for a face ⁇ e.g., all faces).
  • the transform functions in HAC may be created dynamically for a face and/or a direction, which may be based upon the video content.
  • a transform function may be dynamically created for the x direction (e.g., horizontal direction) of a face.
  • a transform function may be dynamically created for the y direction (e.g., vertical direction) of a face.
  • a transform function may be applied for a 2D-to-3D mapping process. Coordinates (x ! , y') may be the sampling coordinates in a hybrid angular cubemap domain, and (x, y) may be the sampling coordinates in a regular cubemap domain. As seen in FIG. 8, a mapping from ( ⁇ ', y ! ) to (x, y) may be performed to transform the sampling coordinates from the HAC domain to the CMP domain.
  • p x and p y may include transform function parameters with positive values. These parameters may determine the behavior of their respective transform functions, which may be seen in FIG. 7. As illustrated in FIG. 7, if, for example, the parameter is 1.0, then HAC may be equivalent to EAC. As illustrated in FIG. 7, if, for example, the parameter is greater than 1.0, the transform function may be more curved than EAC, which may indicate that the sampling at the face center may be dense.
  • HAC may approximate to CMP.
  • HAC may be used to represent different sampling distribution in the cube face (e.g., by adjusting the transform function parameter).
  • the transform function for an inverse mapping from (x, y) to ( ⁇ ', y') ⁇ e.g., transforming the sampling coordinates from cube domain to hybrid angular cube domain) may be shown as: ⁇
  • the relationship between a coordinate ( ⁇ ', y') in the hybrid angular cubemap domain and the coordinate (x, y) in the cubemap domain may be determined.
  • the geometric relationship between the coordinate in the cubemap domain and the corresponding 3D point P s on the sphere may be identified.
  • Projection format conversion between the HAC format and another projection format may be achieved. For example, this may be achieved by using the CMP as an intermediate format
  • a coordinate in the HAC format may be mapped to another projection format (e.g., by mapping the coordinate into an intermediate coordinate). This may be defined in a CMP format and/or based on (8) and (9).
  • the intermediate coordinate may be projected onto a target projection format.
  • a mapped coordinate of its corresponding coordinate in the ERP may be calculated by one or more of the following.
  • Coordinate conversion from HAC to CMP may be performed by using input coordinates (x c yc) in the HAC to calculate an intermediate coordinate (x c , y c ) in CMP according to (8) and/or (9),
  • a 2D-to-3D mapping from CMP to sphere may be performed using the intermediate coordinates (x c , yc), to calculate a coordinate of the corresponding 3D point P s on a sphere.
  • 3D-to- 2D mapping from sphere to ERP may be performed using the coordinates of the 3D point P s to calculate the coordinates (x e , y e ) of its projection point in an ERP domain.
  • a conversion from an ERP to a HAC may be performed using the mapping functions (10) and (11).
  • One or more of the following may apply.
  • a 2D-to-3D mapping from ERP to sphere may be performed. For example, using an (e.g., one) input coordinate (x e , y e ) in an ERP, a corresponding 3D point P s on a sphere may be calculated.
  • a 3D-to-2D mapping from sphere to CMP may be performed. For example, using the coordinate of the 3D point P s , the coordinates (x c , y c ) of its projection point in a CMP may be calculated.
  • Coordinate conversion from CMP to HAC may be performed. For example, given the coordinate (x c , y c ) in a CMP, an output coordinate (3 ⁇ 4', y c ') in HAC according to (10) and (11) may be calculated.
  • a one or more transform function(s) between the HAC and the CMP may be determined by adjusting the values of p x and p y for a direction (e.g., each direction) on a face (e.g., each face).
  • the configurations of the parameters may implicate the transform functions of other projection formats.
  • the setting of p x ⁇ p y ⁇ 0 may correspond to the transform function of CMP
  • FIG. 7 illustrates an example of different HAC transform functions that may be generated to approximate a projection format(s). As seen in FIG, 7, this may be performed by changing the values of the parameters p x and/or p y (e.g., in (8) and (9)). if the parameters are adjusted, the HAC format may determine different transform function(s), which may be used in the x- and/or y-directsons of a face (e.g., each face).
  • the spherical sampling features of a face may be dynamically adapted to the characteristics of its corresponding content on a sphere (e.g., significance of the content, the complexity of texture, whether the content is flat or smooth, and/or whether the content has sharp edges). For example, if HAC is applied, the transform function of a HAG with a larger parameter value (e.g., greater than 1.0) may be applied to a face which contains content at the center of the face. A transform function of a HAC with a larger parameter value (e.g., greater than 1.0) may be applied to a face with detailed content at the center of its face.
  • a larger parameter value e.g., greater than 1.0
  • the transform function of a HAC with a smaller parameter value may be applied to a face which may be located at the face boundaries.
  • a transform function of a HAC with a smaller parameter value may be applied to a face if the content at the face's boundaries includes complex regions with complex textures.
  • transform functions defined for a face may be multiple (e.g., two) transform functions defined for a face ⁇ e.g., each face) in a HAC.
  • a transform function may be used for the horizontal direction(s) and another transform function may be used for the vertical direction(s).
  • the transform functions defined for the faces in HAC may include (8)-(11). If the HAC is frame packed with a 3x2 layout, as seen in FIG. 8, face 0 may be connected with face 4 and/or face 5 in 3D space.
  • the content may be continuous across boundary bO and/or may be continuous across boundary bl if the vertical transform function ef face 4 is different from the vertical transform function of face 0, the sampled content across the sides (e.g., the two sides) of boundary bO may be misaligned in the vertical direction. For example, if there is a horizontal line across the boundary bO, the line may be broken at the boundary bO. if the vertical transform functions ef face 0 is different from vertical transform function ef face 5, the sampled content across the sides (e.g., the two sides) of boundary b1 may be misaligned.
  • the boundaries b2 and b3 in the 3x2 layout may have similar (e.g., the same) continuity characteristics as that of bO and b1. Misalignment may occur at boundaries b2 and/or b3.
  • FIG. 9A illustrates an example of a vertical transform function of face 0 that may be different from that of face 4 and/or face 5, and a vertical transform function of face 1 that may be different from that of face 2 and/or face 3.
  • the face numbering scheme of FIG. 9A may be the same as the face numbering scheme of FIG. 8.
  • misalignment may occur at faces bO, b1 , b2 and/or b3.
  • This discontinuity may cause a coding loss for intra prediction and/or inter prediction.
  • the reference samples may be different from the current block if the current block is located at a misaligned face boundary (e.g., bO, b1 , b2 and b3).
  • the prediction unit may be split to keep the prediction units (e.g., ail the prediction units) located within a face (e.g., one face).
  • the neighboring faces at boundary bO, b1 , b2, and/o b3 may be aligned if a continuity constraint is applied.
  • the face numbering scheme of FIG. 9B may be the same as the face numbering scheme of FIG. 8.
  • the constraint, which may maintain continuity at the face boundaries may be performed in a frame packing based encoding in the HAC.
  • the continuity constraint may be defined as one or more of the following. If two faces in a frame packed layout are connected (e.g., if two faces are neighboring faces) in 3D space, their vertical transform function may be the same, which may maintain continuity.
  • the transform function in the horizontal direction of a face may not have a constraint.
  • the face e.g., each face
  • the face may have a set (e.g., one set) of parameters for the horizontal and/or vertical transform functions.
  • 12 sets of transform function parameters may be used to signal the transform functions for 8 faces (e.g., a transform function parameter for the horizontal and vertical directions of each face), which may be performed without a constraint (e.g., any constraint), if the constraint is applied in the vertical direction for a 3x2 cubemap frame packing (e.g., as seen in FIG.
  • face 4, face 0 and/or face 5 may share a vertical transform function; face 3, face 1 and/or face 2 may share a vertical transform function.
  • 8 sets of transform function parameters e.g., 6 sets of parameters for a horizontal transform function and 2 sets of parameters for a vertical transform function
  • a transform function may be determined (e.g., optimized) for a sampling according to the content's characteristics. For example, if the area contains high frequency components (e.g., strong edges and/or textures) the sampling density may be higher, if the area is smooth, then the sampling density may be lower.
  • the content within one face may change (e.g., drastically change) over time. If, for example, a camera moves quickly in the span of one second, the appearance of a (e.g., each) face may change (e.g., drastically change).
  • a transform function (e.g., the parameters of a transform function) may be updated (e.g., may be updated periodically), if a transform function is updated from picture P, the inter prediction of the pictures using picture P as a reference picture may have different transform functions from picture P. The appearance of an object in the reference pictures may be different than the object in picture P (e.g., the same object in picture P).
  • a conversion step may align the transform functions used in a reference picture with the current picture, which may improve temporal correlation and/or motion compensated prediction. The conversion may be applied such that the transform function of a reference picture may be aligned with the transform function of the current picture. The conversion may be applied such that the current picture transform function may be aligned with that of the reference picture.
  • mapping functions for HAC may be simplified if, for example, their relationship with CMP is considered.
  • first set of transform functions is defined as (8) (9) (10) (11)
  • y g' y' " ) (13)
  • x' f' x) 14)
  • the projection format of a second set of transform functions of HAC defined by (12) (13) (14) and (15) may be indicated as HAC-2.
  • the projection format of a first set of transform functions of HAC defined by (8) (9) (10) and (11) may be indicated HAC-1.
  • the projected position ( ⁇ 1', y1') in HAC-1 may be calculated directly from HAC-2 with the following transform functions. yi' - /v(y) - fy(g ! x (y ')) (17)
  • [0074] (18) and (17) may not require a 2D to 3D and/or 3D to 2D conversion, and may be performed (e.g., directly performed) within the 2D domain.
  • f x ⁇ g' x ⁇ )) and f y (g' y ( )) may include referencing a lookup table (LUT) given the transform functions defined by (8) (9) (10) (11) and (12) (13) (14) (15), which may reduce complexity.
  • mapping and inverse mapping functions may be implemented by a LUT given the transform functions defined by (8), (9) and (10), (11), respectively.
  • Function symmetries e.g., even and/or odd symmetry
  • trigonometric identities may be implemented in the LUT. The function symmetries and/or trigonometric identities may reduce the number of entries in the LUT.
  • a LUT is used to implement tan ""1 x
  • Linear interpolation may be implemented, for example, to approximate values that may not correspond to a LUT entry (e.g., to exactly one LUT entry).
  • the precision may be determined based on an application requirement and/or a memory size (e.g., a memory size restriction).
  • a fixed point representation may represent floating point values and/or may use a fixed precision.
  • the information associated with the HAC may be signaled.
  • a decompressed 360-degree video may be converted to display geometry.
  • the display geometry may be used for dynamically rendering viewports according to users viewing angle (e.g., via an HMD and/or other display device(s)).
  • the parameters of the transform functions may be transmitted and/or signaled to the decoder.
  • the decoder may process the 360-degree video for display (e.g., on as HMD and/or other display devices).
  • the transform function parameter may be used by coding tools (e.g., low-level coding tools) for improving the efficiency of 360-degree video coding.
  • geometric padding may improve motion- compensated prediction for 360-degree video. This may be performed by padding reference samples while considering the 3D geometric structure represented in the coding projection format.
  • High-level syntax elements may be used to signal the geometric information of the HAC format in a bit-stream (e.g., a video bitstream).
  • transform function parameters may be real values, which may be quantized (e.g., before transmission).
  • the techniques used for quantization of the transform parameters in the x-direction may be extended to the techniques used for quantization of the transform function parameters in the y- direction.
  • the real-value parameter p may be (e.g., uniformly) quantized using a quantization step size g S fep. and may be approximated by an integer value (e.g., denoted as p Int ).
  • This quantization technique may be described as:
  • p min may be the minimum value of the parameter.
  • g S f ep may be approximated by a multiplication of an integer factor M, which may be followed by a right shift of N bits, e.g.,
  • the multiplication factor M, the right shit value N, p Int , and/or p min may be determined (e.g., determined by an encoder) and/or signaled (e.g., signaled to a decoder). As illustrated by (19), the multiplication factor M and the right shit value N, may correspond to the quantization step q step .
  • the transform coefficients p may be transmitted and/or signaled. Transmission of the transform coefficients may be performed by specifying the quantized parameters p lm and/or factors M and N in a bit- stream.
  • An appropriate quantization step size(s) (e.g., q step ) may be determined when HAC is applied for 360- degree video coding.
  • the quantization step sizes may be different [e.g., which may result in different M values and N values) for the x- and y-direction of a face.
  • the value of M and the value of N may be signaled for a direction (e.g., each direction) and/or a face (e.g., each face).
  • a value of M and a value of N may be signaled for a first face in the y-direction [e.g., vertical direction), and another value of M and another value of N may be signaled for a second face in y- direction (e.g., vertical direction).
  • a value of M and a value of N may be signaled for a face in the y-direction [e.g., vertical direction), and another value of M and another value of N may be signaled for the face in the x-direction (e.g., horizontal direction).
  • the quantization step size may be the same [e.g., the M value and N value may be the same) for both the x- and y-directions of a face.
  • a value of M and a value of N may be signaled, and those same values of M and N may be used for the x- and y-directions of all the faces.
  • the values of M and N may be signaled to a decoder [e.g., signaled in a video bit-stream).
  • the values of M and N may be fixed at an encoder and/or a decoder. If the values of M and N are fixed, signaling of the values of M and N may not be performed.
  • a syntax element set, hac_parameter_set() may be used to indicate the transform function parameters (e.g., for the HAC format) and/or signaled in the bit-stream.
  • the syntax element set may deliver the transform function parameters from an encoder to a decoder, if, for example, the quantization step size is the same (e.g., the M and N values are the same) for the x- and y-directions in ail the faces, Table 1 may illustrate an example structure of the syntax elements. The information in Table 1 may be used for signaling the mapping coefficients for the HAC format.
  • an indication such as, num_face_rows_minus1 plus one, may indicate the number of face rows in the frame packed picture.
  • An indication such as, num_face_columns_minus1 plus one, may specify the number ef face columns in the frame packed picture.
  • An indication such as, usejdenticaljrans Junes _for_all_faces, may indicate whether the same transform functions [e.g., the same transform function parameters) are used for all the faces in the frame packed picture, if usejdenticaljrans Junes Jor_aliJaces flag is equal to 1 , the transform function (e.g., the transform function parameter) indicated for the face located in the first row and the first column in the frame- packed picture may be reused as the transform function (e.g., the transform function parameter) of the other faces. If usejdenticaljrans Junes Jor_all Jaces flag is equal to 0, the faces in the frame-packed picture may use different transform functions (e.g., may use different transform function parameters).
  • the same transform functions e.g., the same transform function parameters
  • An indication such as, face_boundary_continuity_constraint, may indicate whether the same vertical transform functions are used for the faces within a face row. If face_boundary_continuity_constraint is equal to 1 , the vertical transform function ⁇ e.g., the transform function parameter in the vertical direction) for the face located at the first column may be reused for all other faces in the same row. if face_boundary_continuity_constraint is equal to 1 , the vertical transform function ⁇ e.g., the transform function parameter in the vertical direction) for the face located at the first column may be reused for all other faces in the same row. if
  • the faces may use different vertical transform functions (e.g., different transform function parameters in the vertical direction).
  • An indication such as, horizontal Jrans_coeff[i][j] may indicate the quantized coefficient value for the horizontal transform function (e.g., may indicate the quantized transform function parameter in the horizontal direction) of the face located at the i-th row and j-th column in a frame packed picture.
  • An indication such as, HorTransCoeff[i][j] may indicate the real coefficient value for the horizontal transform function (e.g., may indicate the real transform function parameter in the horizontal direction) of the face located at the i-th row and j-th column in the frame packed picture.
  • An indication such as, vertical Jrans_coeff[i][j] may indicate the quantized coefficient value for the vertical transform function (e.g., may indicate the quantized transform function parameter in the vertical direction) of the face located at the i-th row and j-th column in the frame packed picture.
  • An indication, such as, VerTransCoeff p][j] may indicate the real coefficient value for the vertical transform function (e.g., may indicate the real transform function parameter in the vertical direction) of the face located at the i-th row and j-th column in the frame packed picture.
  • An indication such as, quantjnfojag
  • quantjnfojag may be an indication of whether the scaling factor and/or bit shift information is transmitted/received. For example, if quantjnfojag is equal to 1 , the scaling factor and/or bit shift information may be transmitted/received, if quantjnfojag is equal to 0, a default scaling factor and bit shift number may be used (e.g., may be used by an encoder and/or decoder).
  • An indication such as, coeff_scalingJactor_minus1 plus one, may indicate the value of the scaling factor (e.g., the multiplication factor Mas described herein with respect to (19)) used to calculate the coefficients of the transform function (e.g., calculate the transform function parameter).
  • coefLscaling Jactor_minus1 may be set to 0 by default
  • the indication of the scaling factor value may be separately signaled for each direction (e.g., horizontal and/or vertical) of a face. Also, or alternatively, the indication of the scaling factor value may be separately signaled for each direction (e.g., horizontal and/or vertical) of each face.
  • An indication such as, coeff_bit_shift, may indicate the number of right shifts (e.g., the right shit value N as described herein with respect to (19)) used to calculate the real coefficients of the transform function (e.g., calculate the transform function parameter).
  • coeff_bit_shift may be set to 6 by default.
  • the indication of the number of right shifts may be separately signaled for each direction (e.g., horizontal and/or vertical) of a face. Also, or alternatively, the indication of the number of right shifts may be separately signaled for each direction (e.g., horizontal and/or vertical) of each face.
  • hac_parameter_set() may be signaled at a sequence-level parameter set, e.g., video parameter set (VPS) and/or sequence parameter set (SPS).
  • the selection of the transform functions may be allowed at the sequence-level such that a set of transform functions (e.g. , the same transform function) may be used for the picture(s) (e.g., all the pictures) in the same video sequence.
  • the hac _parameter_set() may be signaled at the picture-level parameter set, e.g., PPS or slice header, if the hac_parameter_set() is signaled at the picture-level parameter set, the transform function may be adapted at the picture-level by allowing a picture (e.g., each picture) to select its own transform function.
  • Face based signaling and/or region based signaling may be performed.
  • a face e.g., each face
  • the regions may be equal or un-equai.
  • a transform function may be signaled for a region ⁇ e.g., each region).
  • the transform function between two neighboring regions may have an equal value at a region boundary, which may allow a face (e.g., the whole face) to be sampled without overlapping.
  • the transform functions (e.g., in horizontal and/or vertical directions) of a face may be signaled with or without prediction.
  • the temporal correlation of pictures in a video sequence may be strong.
  • the transform function(s) of a face may be similar to the transform function(s) of a collocated face in the face's temporal neighboring pictures.
  • a prediction implementation may be used for coding the transform functions of a face.
  • a flag, trans_coeff_pred_enable_flag may be signaled, for example, to indicate whether a face is predictively coded, if trans_coeff_pred_enable_flag is equal to zero, the parameters of the transform functions in the face may be independently coded without prediction.
  • the parameters of the transform functions in the face may be predicted from the transform function parameters of the same face in a previously coded picture (e.g., the temporal reference picture that precedes the current picture in the decoding order). If, for example, the temporal scalability is enabled, a picture may not be predicted from a reference picture that is located at higher temporal layers, if a prediction method is enabled, the transform function parameters of the faces in the current picture maybe predicted from the preceding picture with an equal or lower temporal layer index. A reference picture that precedes (e.g., directly precedes) the current picture may be used to predict the transform function parameters in the current picture.
  • a candidate list of transform function parameters of a temporal reference pictures may be created and maintained at an encoder and/or a decoder.
  • the list may be used for predicting the transform functions of the current picture.
  • an indication such as, trans_coeff_predictor_idx, may be signaled.
  • the indication (e.g., trans_coeff_predictor_idx) may indicate the transform functions of the reference pictures within the candidate list that are used to predict the transform functions of the current picture.
  • the HAC parameter may be predicted from a neighboring coded face.
  • the transform function parameter for a current face may be predicted from the transform function parameter of another face (e.g., the transform function parameter in the same direction of a neighboring coded face in the same face row).
  • a face index may be signaled to indicate the face used the prediction of transform function parameter(s) in a direction(s) (e.g., both directions).
  • Transform function parameters for a HAC projection format may be determined and/or signaled.
  • the transform function parameters for a HAC projection format may be determined by an encoder.
  • the transform function parameters may be signaled (e.g., signaled to a decoder) using one or more syntax elements (e.g., using hac_parameter_set() and/or the techniques described herein in association with Table 1).
  • HAC parameters may be determined to minimize the conversion error. For example, an iterative search in one or more directions may be performed to determine the HAC parameters.
  • a progressive fast HAC parameter search PROG-HAC
  • transform function parameter determination e.g., parameter estimation
  • HAC e.g., a transform function associated with HAC
  • the search candidates for a transform function parameter may be reduced with an initial fast search (e.g., on the first frame of the video sequence), the transform function parameters for subsequent pictures may be refined within a window (e.g., a progressive search range) based on a first transform function parameter(s) (e.g., a first set of transform function parameters), and/or the transform function parameters for an intra random access point (IRAP) may be refined within a window (e.g., a progressive search range) when HAC parameters are updated periodically.
  • an initial fast search e.g., on the first frame of the video sequence
  • the transform function parameters for subsequent pictures may be refined within a window (e.g., a progressive search range) based on a first transform function parameter(s) (e.g., a first set of transform function parameters)
  • IRAP intra random access point
  • a 360-degree video clip may include a plurality of frames.
  • a "heading-frame” ( ⁇ H) may be or may include an initial frame of the 360-degree video, an IRAP frame, an instantaneous decoder reference (IDR) frame, and/or an initial frame of a video segment.
  • the HAC parameters derived for ⁇ H may be referred to as P , where "h” may be the heading-frame and / may be the cube-face index. As described herein, / may range from 0 to 5.
  • the parameters F ⁇ may be determined during 380-degree video encoding or conversion.
  • the HAC transform function parameters for subsequent IRAPs and/or pictures may be searched within a neighborhood (e.g., - ⁇ , - J) with respect to P .
  • ⁇ may indicate a progressive search range (PSR) (e.g., half of the PSR) and may be determined empirically and/or based on video statistics, for example, through pre-processing.
  • PSR progressive search range
  • An example HAC fast progressive search is illustrated in FIG. 10.
  • the HAC parameters may be updated for an IRAP frame (e.g., each IRAP frame).
  • the HAC parameters may be updated for a subsequent picture(s), e.g., including non-IRAP pictures.
  • the HAC transform function parameters for the heading-fame may be searched from a range of HAC transform function parameters (e.g., the HAC parameter space described in FIG. 10).
  • the range of HAC transform function parameters search for the heading frame may be referred to as a parameter space.
  • a parameter space may be quantized, for example, using (19). If, for example, p m / n is set to 0.3, M is set to 5, and N is set to 8, the integer parameter search range may include [0, 63].
  • the quantized integer parameter search range may correspond to a real parameter range [0.3, 1.53], which may be associated with transform functions that correspond to CMP and/or EAC (e.g., as seen in FIG. 7).
  • ⁇ may be pre-determined as an integer parameter ⁇ e.g., 4) and/or may indicate a search range (e.g., may indicate half of the search range) for subsequent transform function parameters, which may be referred to as the PSR,
  • the PSR may be a range relative to a selected transform function parameter.
  • the PSR may be determined relative to a first transform function parameter (e.g., the selected transform function parameter for IRAP:0).
  • the PSR may include ⁇ parameters above and below the first transform function parameter.
  • the temporal variation of HAC transform function parameters for subsequent IRAPs may be limited.
  • Parameter refinement may be (e.g., may only be) performed in a neighborhood based on the HAC parameters of the heading-frame ⁇ e.g., the PSR).
  • the HAC transform function parameters for a subsequent !RAP frames may be searched for from within the PSR, which may include a range of transform function parameter relative to the transform function parameter selected for the heading frame ⁇ e.g., IRAP:0).
  • the HAC transform function parameters for a subsequent picture which may or may not be an I RAP frame, may be searched for from within the PSR.
  • the HAC parameters for the heading-frame and the subsequent IRAPs may be derived using one or more searches, e.g., coarse-to-fine search (CFS) or binary search (BS).
  • CFS coarse-to-fine search
  • BS binary search
  • a CFS may search the entire parameter space with a uniform coarse search step-size ⁇ . If the optimal parameter is pinpointed, a local search refinement may be triggered in a smaller neighborhood of the previously chosen optimal parameter and the search step-size may be reduced.
  • the local search refinement may ⁇ e.g., may only) search within two neighbors around the optimal parameter in the previous search step. This refinement may continue until a convergence criterion is met (e.g., the step-size reduces to 1).
  • An example CFS search pattern is illustrated in FIG. 11. For example, the ⁇ may be set to 4 at the beginning. In the refinement stage, the step size may be reduced by half at each step.
  • a BS may decrease the complexity of searching for a transform function parameter.
  • the cost of a HAC parameter in linear space e.g., one direction HAC parameter search space of one face
  • BS may return a local minimum.
  • PL may be the left end parameter
  • PR may be the right end parameter.
  • a fast binary search may be performed using one or more of the following.
  • the cost of the left and right end point may be calculated.
  • the search window may be updated based on the costs at two ends. If the cost of the left end is smaller than the cost of right end, the right end may be updated by the middle point, and the search window may be updated as [PL, (PL+ PR)/2].
  • the left end may be updated by the middle point, and the search window may be updated as [(PL+ PR)/2, PR]. if the distance between the two end points is not greater than 1 , the optimal point may be the smaller cost two ends. If the distance between the two end points is greater than 1 , the cost of left and right end points may be calculated.
  • a 360-degree video encoder or converter may select different configurations for search a range ⁇ , a PSR, and/or search step-size ⁇ , etc, based on a desired complexity and/or qualify.
  • a BS and/or CFS may be chosen and/or combined among the heading-frame and the subsequent IRAP frames.
  • a parameter progressive search range may be adapted based on the distance between the current picture and the heading-frame. For example, as the distance between the heading frame and the current picture increases, the size of the progressive search range may increase.
  • the heading-frame may be generaiizabie (e.g., a heading frame may include an initial frame of the 380-degree video, and IRAP frame, an instantaneous decoder reference (IDR) frame, and/or an initial frame of a video segment).
  • a current IRAP may use the HAC parameters from a previous IRAP picture as the heading-frame to start local progressive search,
  • a parameter search may be applicable to other projection formats with spatially dynamic transform function parameters (e.g., transform function parameters associated with a direction(s) and/or a face(s)) and/or temporally dynamic transform function parameters (e.g., periodically updating the transform function parameter).
  • spatially dynamic transform function parameters e.g., transform function parameters associated with a direction(s) and/or a face(s)
  • temporally dynamic transform function parameters e.g., periodically updating the transform function parameter.
  • One or more HAC parameters may be set to a default value(s). For example, if HAC does not yield coding benefits in one or more segments of a video (e.g., a portion of or the entire video), the HAC parameters may be set to default values, which may correspond to EAC. One or more of the following may apply. Multi-pass encoding may be used to determine whether HAC yields increased coding benefits.
  • the HAC parameters may be set to a default value
  • the characteristics of a video varies over time (e.g., varies drastically over time)
  • HAC parameters may be set to a default value
  • the difference between two temporally separated frames in a projection format e.g., ERP or EAC
  • the HAC parameters may be set to a default value.
  • the difference between two frames may be measured using, for example, one or more of the following: the sum of absolute differences (SAD) computed between the two frames, and/or the sum of square differences (SSD) computed between the two frames.
  • SAD sum of absolute differences
  • SSD sum of square differences
  • a HAC parameter search (e.g., a constrained HAC parameter search) may be performed with, e.g., continuous sampling between neighboring faces.
  • the vertical HAC parameters for the faces in a face row may be similar (e.g., may be the same), if, for example, the vertical HAC parameters for the faces in each face row are the same, content continuity at the borders between continuous faces may be preserved, if adaptive frame packing is used, the location and/or orientation of the faces within a frame packed picture (e.g., each face in a frame packed picture) may depend on the frame packing arrangement (e.g., the frame packing arrangement used during video encoding).
  • Frame packing arrangements may be dynamically updated at regular and/or irregular intervals, e.g., at every intra random access point picture. If frame packing arrangements are dynamically updated, temporal prediction may be performed near face discontinuities and/or the presence of seam artifacts may be reduced. If the frame packing configuration of a HAC are dynamically updated, the HAC parameters for the updated HAC frame packing configuration may be searched using one or more constraints. For example, the constraints may include similar vertical sampling (e.g., the same vertical sampling) for the faces within a face row (e.g., all the faces within each face row). If the vertical sampling for the faces within each face row is the same, content continuity may be preserved at the borders between continuous faces.
  • the constraints may include similar vertical sampling (e.g., the same vertical sampling) for the faces within a face row (e.g., all the faces within each face row). If the vertical sampling for the faces within each face row is the same, content continuity may be preserved at the borders between continuous faces.
  • a current picture may use different HAC parameters than a reference picture of the current picture. If a current picture uses a reference picture associated with a different set of HAC parameters, a conversion step may align the transform functions used in a reference picture to the current picture, if the frame packing configuration is changed during video encoding, e.g., when using adaptive frame packing, the initial HAG parameter search may be changed (e.g., to avoid HAC parameter conversion).
  • a HAC parameter search may be performed (e.g., may only be performed) for a frame (e.g., the first frame of a video) using one or more constraints.
  • the constraints may include using similar HAC parameters (e.g., using the same HAC parameters) in directions parallel to a shared boundary between neighboring faces (e.g., any two neighboring faces in a 3D geometry).
  • HAC parameters in directions parallel to the X axis may be similar (e.g., may be the same) for faces (e.g., all faces) that are perpendicular to the YZ plane.
  • the HAC parameters in directions parallel to the Y axis may be similar (e.g., may be the same) for faces (e.g., all faces) that are perpendicular to the XZ plane.
  • the HAG parameters in directions parallel to the Z axis may be similar (e.g., may be the same) for faces (e.g., for all faces) that are perpendicular to the XY plane.
  • HAC parameters may be searched and/or may be signaled in the bit stream.
  • HAC parameters may be derived for a first frame (e.g., may be derived once for a first frame).
  • HAC parameters may not be derived for every frame packing configuration updated picture. Conversion of a reference picture (e.g., to align with the current picture) may be skipped.
  • HAC parameters may be derived for every frame packing configuration updated picture.
  • a conversion step may align the transform functions used in a reference picture to the current picture.
  • a parameter space may be iteratively searched and/or, during the parameter space search, HAC parameters may be fixed at an iteration (e.g., fixed at each iteration).
  • HAC parameters may be fixed in a first set of one or more directions (e.g., for the X and Z direction).
  • a HAC parameter search may be performed (e.g., may only be performed) in the remaining direction(s) (e.g., the Y direction).
  • the search may be sequentially refined (e.g., may then be sequentially refined) for the first set of directions in a similar way (e.g., fixing the search parameters in the Z and Y directions and performing a HAC parameter search in the X direction).
  • the iterative search process may be repeated (e.g., may be repeated in ail directions) until a criterion is met. For example, the iterative search may repeat until the distortion difference between two updates falls below a given threshold.
  • 360-degree videos may include omni-directional videos, spherical videos, six degree of freedom (6DoF) media, monoscopic and stereoscopic (3D) virtual reality videos, and/or the like.
  • 6DoF six degree of freedom
  • 3D monoscopic and stereoscopic
  • FIG. 12A is a diagram illustrating an example communications system 100 in which one or more disclosed embodiments may be implemented.
  • the communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users.
  • the communications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth.
  • the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TD A), frequency division multiple access (FDMA), orthogonal FDMA (OFD A), single-carrier FDMA (SC-FDMA), zero-tail unique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC), and the like.
  • CDMA code division multiple access
  • TD A time division multiple access
  • FDMA frequency division multiple access
  • OFD A orthogonal FDMA
  • SC-FDMA single-carrier FDMA
  • ZT UW DTS-s OFDM zero-tail unique-word DFT-Spread OFDM
  • UW-OFDM unique word OFDM
  • FBMC filter bank multicarrier
  • the communications system 100 may include wireless transmit/receive units (WTRUs) 102a, 102b, 102c, 102d, a RAN 104/113, a CN 106/115, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements.
  • WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and/or communicate in a wireless environment.
  • the WTRUs 102a, 102b, 102c, 102d may be configured to transmit and/or receive wireless signals and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications ⁇ e.g., a robot and/or other wireless devices operating in an industrial and/or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and/or industrial wireless networks, and the like.
  • UE user equipment
  • PDA personal digital assistant
  • smartphone a laptop
  • a netbook a personal computer
  • the communications systems 100 may also include a base station 114a and/or a base station 114b.
  • Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the VVTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106/115, the Internet 110, and/or the other networks 112.
  • the base stations 114a, 114b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a gNB, a NR NodeB, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and/or network elements.
  • the base station 114a may be part of the RAN 104/113, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc.
  • BSC base station controller
  • RNC radio network controller
  • the base station 114a and/or the base station 114b may be configured to transmit and/or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum.
  • a ceil may provide coverage for a wireless service to a specific geographical area that may be relatively fixed or that may change over time. The ceil may further be divided i to cell sectors.
  • the cell associated with the base station 114a may be divided into three sectors.
  • the base station 114a may include three transceivers, i.e., one for each sector of the cell, in an embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the cell.
  • MIMO multiple-input multiple output
  • beamforming may be used to transmit and/or receive signals in desired spatial directions.
  • the base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link ⁇ e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.).
  • the air interface 116 may be established using any suitable radio access technology (RAT).
  • RAT radio access technology
  • the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like.
  • the base station 114a in the RAN 104/113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 115/116/117 using wideband CDMA (WCDMA).
  • WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+).
  • HSPA may include High-Speed Downlink (DL) Packet Access (HSDPA) and/or High-Speed UL Packet Access (HSUPA).
  • the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A) and/or LTE-Advanced Pro (LTE-A Pro).
  • E-UTRA Evolved UMTS Terrestrial Radio Access
  • LTE Long Term Evolution
  • LTE-A LTE-Advanced
  • LTE-A Pro LTE-Advanced Pro
  • the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR Radio Access, which may establish the air interface 116 using New Radio (NR),
  • NR New Radio
  • the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies.
  • the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for instance using dual connectivity (DC) principles.
  • DC dual connectivity
  • the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and/or transmissions sent to/from multiple types of base stations ⁇ e.g., an eNB and a gNB).
  • the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, interim Standard 2000 (IS- 2000), Interim Standard 95 (lS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.
  • IEEE 802.11 i.e., Wireless Fidelity (WiFi)
  • IEEE 802.16 i.e., Worldwide interoperability for Microwave Access (WiMAX)
  • CDMA2000, CDMA2000 1X, CDMA2000 EV-DO interim Standard 2000 (IS- 2000), Interim Standard 95 (lS-95), Interim Standard 856 (IS-856), Global System for
  • the base station 114b in FIG. 12A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a roadway, and the like, in one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN).
  • WLAN wireless local area network
  • the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN).
  • the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT ⁇ e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR etc.) to establish a picoceil or femtoceil.
  • the base station 1 14b may have a direct connection to the Internet 110.
  • the base station 114b may not be required to access the Internet 1 10 via the CN 106/1 15.
  • the RAN 104/113 may be in communication with the CN 106/1 15, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d.
  • the data may have varying quality of service (QoS)
  • the CN 106/1 15 may provide call control, billing services, mobile location-based services, pre-paid calling, internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication. Although not shown in FIG. 12A, it will be appreciated that the RAN 104/1 13 and/or the CN 106/1 15 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104/1 13 or a different RAT.
  • the CN 106/115 may also be in communication with another RAN (not shown) employing a GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.
  • the CN 106/115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 1 10, and/or the other networks 112.
  • the PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS).
  • POTS plain old telephone service
  • the internet 1 10 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and/or the internet protocol (IP) in the TCP/IP internet protocol suite.
  • the networks 1 12 may include wired and/or wireless communications networks owned and/or operated by other service providers.
  • the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104/1 13 or a different RAT.
  • Some or ail of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities ⁇ e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links).
  • the WTRU 102c shown in FIG. 12A may be configured to communicate with the base station 1 14a, which may employ a cellular-based radio technology, and with the base station 1 14b, which may employ an IEEE 802 radio technology.
  • FIG. 12B is a system diagram illustrating an example WTRU 102.
  • the WTRU 102 may include a processor 1 18, a transceiver 120, a transmit/receive element 122, a speaker/microphone 124, a keypad 126, a display/touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and/or other peripherals 138, among others.
  • GPS global positioning system
  • the VVTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment
  • the processor 118 may be a general-purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like.
  • the processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate in a wireless environment.
  • the processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While FIG. 12B depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.
  • the transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station ⁇ e.g., the base station 114a) over the air interface 116.
  • the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals.
  • the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example, in yet another embodiment, the transmit/receive element 122 may be configured to transmit and/or receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.
  • the WTRU 102 may include any number of transmit/receive elements 122. More specifically, the WTRU 102 may employ MiMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.
  • the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.
  • the transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122.
  • the WTRU 102 may have multi-mode capabilities.
  • the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11 , for example.
  • the processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128 ⁇ e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit).
  • the processor 118 may also output user data to the speaker/microphone 124, the keypad 128, and/or the dispiay/touchpad 128.
  • the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132.
  • the non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device.
  • the removable memory 132 may include a subscriber identify module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like, in other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).
  • SIM subscriber identify module
  • SD secure digital
  • the processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the WTRU 102.
  • the power source 134 may be any suitable device for powering the WTRU 102.
  • the power source 134 may include one or more dry ceil batteries (e.g., n/cte/-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.
  • the processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. in addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 116 from a base station (e.g., base stations 114a, 114b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment
  • the processor 118 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity.
  • the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs and/or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, a Virtual Reality and/or Augmented Reality (VR/AR) device, an activity tracker, and the like.
  • FM frequency modulated
  • the peripherals 138 may include one or more sensors, the sensors may be one or more of a gyroscope, an accelerometer, a hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and/or a humidity sensor.
  • the WTRU 102 may include a full duplex radio for which transmission and reception of some or ail of the signals ⁇ e.g., associated with particular subframes for both the UL (e.g., for transmission) and downlink (e.g., for reception) may be concurrent and/or simultaneous.
  • the full duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference via either hardware ⁇ e.g., a choke) or signal processing via a processor ⁇ e.g., a separate processor (not shown) or via processor 118).
  • the WRTU 102 may include a half-duplex radio for which transmission and reception of some or all of the signals ⁇ e.g., associated with particular subframes for either the UL (e.g., for transmission) or the downlink (e.g., for reception).
  • FIG. 12C is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment.
  • the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116.
  • the RAN 104 may also be in communication with the CN 106.
  • the RAN 104 may include eNode ⁇ Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment.
  • the eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116.
  • the eNode-Bs 160a, 160b, 160c may implement MIMO technology.
  • the eNode-B 160a for example, may use multiple antennas to transmit wireless signals to, and/or receive wireless signals from, the WTRU 102a.
  • Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and/or DL, and the like. As shown in FIG. 12C, the eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 interface.
  • the CN 106 shown in FIG. 12C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. While each of the foregoing elements are depicted as part of the CN 106, it will be appreciated that any of these elements may be owned and/or operated by an entity other than the CN operator.
  • MME mobility management entity
  • SGW serving gateway
  • PGW packet data network gateway
  • the MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node.
  • the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like.
  • the MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and/or WCDMA.
  • the SGW 164 may be connected to each of the eNode Bs 160a, 160b, 160c in the RAN 104 via the S1 interface.
  • the SGW 164 may generally route and toward user data packets to/from the WTRUs 102a, 102b, 102c.
  • the SGW 164 may perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like,
  • the SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.
  • packet-switched networks such as the Internet 110
  • the CN 106 may facilitate communications with other networks.
  • the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices.
  • the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108.
  • IMS IP multimedia subsystem
  • the CN 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 112, which may include other wired and/or wireless networks that are owned and/or operated by other service providers.
  • the WTRU is described in FIGS. 14A-14D as a wireless terminal, it is contemplated that in certain representative embodiments that such a terminal may use ⁇ e.g., temporarily or permanently) wired communication interfaces with the communication network.
  • the other network 112 may be a WLAN.
  • a WLAN in infrastructure Basic Serace Set (BSS) mode may have an Access Point (AP) for the BSS and one or more stations (STAs) associated with the AP.
  • the AP may have an access or an interface to a Distribution System (DS) or another type of wired/wireless network that carries traffic in to and/or out of the BSS.
  • Traffic to STAs that originates from outside the BSS may arrive through the AP and may be delivered to the STAs.
  • Traffic originating from STAs to destinations outside the BSS may be sent to the AP to be delivered to respective destinations.
  • Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP and the AP may deliver the traffic to the destination STA.
  • the traffic between STAs within a BSS may be considered and/or referred to as peer-to-peer traffic.
  • the peer-to- peer traffic may be sent between (e.g., directly between) the source and destination STAs with a direct link setup (DLS).
  • the DLS may use an 802.11e DLS or an 802.11 z tunneled DLS (TDLS).
  • a WLAN using an independent BSS (IBSS) mode may not have an AP, and the STAs ⁇ e.g., all of the STAs) within or using the IBSS may communicate directly with each other.
  • IBSS independent BSS
  • the AP may transmit a beacon on a fixed channel, such as a primary channel.
  • the primary channel may be a fixed width (e.g., 20 MHz wide bandwidth) or a dynamically set width via signaling.
  • the primary channel may be the operating channel of the BSS and may be used by the STAs to establish a connection with the AP.
  • Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) may be implemented, for example in in 802.11 systems.
  • the STAs may sense the primary channel. If the primary channel is sensed/detected and/or determined to be busy by a particular STA, the particular STA may back off.
  • One STA e.g. , only one station may transmit at any given time in a given BSS.
  • High Throughput (HT) STAs may use a 40 MHz wide channel for communication, for example, via a combination of the primary 20 MHz channel with an adjacent or nonadjacent 20 MHz channel to form a 40 MHz wide channel.
  • VHT STAs may support 20MHz, 40 MHz, 80 MHz, and/or 160 MHz wide channels.
  • the 40 MHz, and/or 80 MHz, channels may be formed by combining contiguous 20 MHz channels.
  • a 160 MHz channel may be formed by combining 8 contiguous 20 MHz channels, or by combining two noncontiguous 80 MHz channels, which may be referred to as an 80+80 configuration.
  • 80+80 For the 80+80
  • the data after channel encoding, may be passed through a segment parser that may divide the data into two streams.
  • the streams may be mapped on to the two 80 MHz channels, and the data may be transmitted by a transmitting STA, At the receiver of the receiving STA, the above described operation for the 80+80 configuration may be reversed, and the combined data may be sent to the Medium Access Control (MAC).
  • MAC Medium Access Control
  • Sub 1 GHz modes of operation are supported by 802.11 af and 802.11 ah.
  • the channel operating bandwidths, and carriers, are reduced in 802.11 af and 802.11 ah relative to those used in 802,11 ⁇ , and 802.11ac.
  • 802.11 af supports 5 MHz, 10 MHz and 20 MHz bandwidths in the TV White Space (TVWS) spectrum
  • 802,11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum.
  • 802.11 ah may support Meter Type Control/Machine- Type Communications, such as MTC devices in a macro coverage area.
  • MTC devices may have certain capabilities, for example, limited capabilities including support for (e.g., only support for) certain and/or limited bandwidths.
  • the MTC devices may include a battery with a battery life above a threshold (e.g., to maintain a very long battery life).
  • WLAN systems which may support multiple channels, and channel bandwidths, such as 802.1 1 n, 802.1 1 ac, 802.11 af, and 802.1 1 ah, include a channel which may be designated as the primary channel.
  • the primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all ST As in the BSS.
  • the bandwidth of the primary channel may be set and/or limited by a STA, from among all ST As in operating in a BSS, which supports the smallest bandwidth operating mode, in the example of 802.11 ah, the primary channel may be 1 MHz wide for STAs ⁇ e.g., MTC type devices) that support (e.g., only support) a 1 MHz mode, even if the AP, and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 18 MHz, and/or other channel bandwidth operating modes.
  • Carrier sensing and/or Network Allocation Vector (NAV) settings may depend on the status of the primary channel, if the primary channel is busy, for example, due to a STA (which supports only a 1 MHz operating mode), transmitting to the AP, the entire available frequency bands may be considered busy even though a majority of the frequency bands remains idle and may be available.
  • NAV Network Allocation Vector
  • the available frequency bands which may be used by 802.1 1 ah, are from 902 MHz to 928 MHz. In Korea, the available frequency bands are from 917.5 MHz to 923.5 MHz. in Japan, the available frequency bands are from 918.5 MHz to 927.5 MHz.
  • the total bandwidth available for 802.11 ah is 6 MHz to 28 MHz depending on the country code.
  • FIG. 12D is a system diagram illustrating the RAN 1 13 and the CN 115 according to an embodiment.
  • the RAN 1 13 may employ an NR radio technology to communicate with the VVTRUs 102a, 102b, 102c over the air interface 1 16.
  • the RAN 1 13 may also be in communication with the CN 1 1 5.
  • the RAN 113 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment.
  • the gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 1 16.
  • the gNBs 180a, 180b, 180c may implement MIMO technology.
  • gNBs 180a, 108b may utilize beamforming to transmit signals to and/or receive signals from the gNBs 1 80a, 180b, 180c.
  • the gNB 180a may use multiple antennas to transmit wireless signals to, and/or receive wireless signals from, the VVTRU 102a.
  • the gNBs 180a, 180b, 180c may implement carrier aggregation technology.
  • the gNB 180a may transmit multiple component carriers to the VVTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum while the remaining component carriers may be on licensed spectrum, in an embodiment, the gNBs 1 80a, 180b, 180c may implement Coordinated Multi-Point (CoMP) technology.
  • VVTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and/or gNB 180c).
  • CoMP Coordinated Multi-Point
  • the WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using transmissions associated with a scalable numerology. For example, the OFDM symbol spacing and/or OFDM subcarrier spacing may vary for different transmissions, different cells, and/or different portions of the wireless transmission spectrum.
  • the VVTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using subframe or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing varying number of OFDM symbols and/or lasting varying lengths of absolute time).
  • TTIs subframe or transmission time intervals
  • the gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and/or a non-standalone configuration. In the standalone configuration, VVTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c without also accessing other RANs ⁇ e.g., such as eNode-Bs 160a, 160b, 160c).
  • VVTRUs 102a, 102b, 102c may utilize one or more of gNBs 180a, 180b, 180c as a mobility anchor point, in the standalone configuration, VVTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using signals in an unlicensed band.
  • WTRUs 102a, 102b, 102c may communicate with/connect to gNBs 180a, 180b, 180c while also communicating with/connecting to another RAN such as eNode-Bs 160a, 160b, 160c.
  • WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously.
  • eNode-Bs 160a, 160b, 160c may serve as a mobility anchor for WTRUs 102a, 102b, 102c and gNBs 180a, 180b, 180c may provide additional coverage and/or throughput for servicing VVTRUs 102a, 102b, 102c.
  • Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and/or DL, support of network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data towards User Plane Function (UPF) 184a, 184b, routing of control plane information towards
  • UPF User Plane Function
  • Access and Mobility Management Function 182a, 182b and the like.
  • AMF Access and Mobility Management Function
  • 180a, 180b, 180c may communicate with one another over an Xn interface.
  • the CN 115 shown in FIG. 12D may include at least one AMF 182a, 182b, at least one UPF
  • SMS Session Management Function
  • DN Data Network
  • the AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N2 interface and may serve as a control node.
  • the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, support for network slicing ⁇ e.g., handling of different PDU sessions with different requirements), selecting a particular S F 183a, 183b, management of the registration area, termination of NAS signaling, mobility management, and the like.
  • Network slicing may be used by the AMF 182a, 182b in order to customize CN support for VVTRUs 102a, 102b, 102c based on the types of services being utilized VVTRUs 102a, 102b, 102c.
  • different network slices may be established for different use cases such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for machine type communication (MTC) access, and/or the like.
  • URLLC ultra-reliable low latency
  • eMBB enhanced massive mobile broadband
  • MTC machine type communication
  • the AMF 182 may provide a control plane function for switching between the RAN 1 13 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and/or non-3GPP access technologies such as WiFi.
  • radio technologies such as LTE, LTE-A, LTE-A Pro, and/or non-3GPP access technologies such as WiFi.
  • the SMF 183a, 183b may be connected to an AMF 182a, 182b in the CN 1 15 via an N1 1 interface.
  • the SMF 183a, 183b may also be connected to a UPF 184a, 184b in the CN 1 15 via an N4 interface.
  • the SMF 183a, 183b may select and control the UPF 184a, 184b and configure the routing of traffic through the UPF 184a, 184b.
  • the SMF 183a, 183b may perform other functions, such as managing and allocating UE IP address, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notifications, and the like.
  • a PDU session type may be IP-based, non-IP based, Ethernet-based, and the like.
  • the UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 1 80c in the RAN 1 13 via an N3 interface, which may provide the VVTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 1 10, to facilitate communications between the VVTRUs 102a, 102b, 102c and IP-enabled devices.
  • the UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and the like.
  • the CN 1 15 may facilitate communications with other networks.
  • the CN 1 15 may include, or may communicate with, an IP gateway ⁇ e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 1 15 and the PSTN 108.
  • the CN 1 15 may provide the VVTRUs 102a, 102b, 102c with access to the other networks 1 12, which may include other wired and/or wireless networks that are owned and/or operated by other service providers.
  • IMS IP multimedia subsystem
  • the VVTRUs 102a, 102b, 102c may be connected to a local Data Network (DN) 185a, 185b through the UPF 184a, 184b via the N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b,
  • DN local Data Network
  • one or more, or ail, of the functions described herein with regard to one or more of: VVTRU 102a-d, Base Station 1 14a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and/or any other device(s) described herein, may be performed by one or more emulation devices (not shown).
  • the emulation devices may be one or more devices configured to emulate one or more, or all, of the functions described herein.
  • the emulation devices may be used to test other devices and/or to simulate network and/or VVTRU functions.
  • the emulation devices may be designed to implement one or more tests of other devices in a lab environment and/or in an operator network environment.
  • the one or more emulation devices may perform the one or more, or ail, functions while being fully or partially implemented and/or deployed as part of a wired and/or wireless communication network in order to test other devices within the communication network.
  • the one or more emulation devices may perform the one or more, or all, functions while being temporarily implemented/deployed as part of a wired and/or wireless communication network.
  • the emulation device may be directly coupled to another device for purposes of testing and/or may performing testing using over-the-air wireless communications,
  • the one or more emulation devices may perform the one or more, including all, functions while not being implemented/deployed as part of a wired and/or wireless communication network.
  • the emulation devices may be utilized in a testing scenario in a testing laboratory and/or a non-deployed ⁇ e.g., testing) wired and/or wireless communication network in order to implement testing of one or more components.
  • the one or more emulation devices may be test equipment. Direct RF coupling and/or wireless communications via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and/or receive data.
  • RF circuitry e.g., which may include one or more antennas
  • ROM read only memory
  • RAM random access memory
  • register cache memory
  • semiconductor memory devices magnetic media such as, but not limited to, internal hard disks and removable disks, magneto-optical media, and/or optical media such as CD- ROM disks, and/or digital versatile disks (DVDs)
  • a processor in association with software may be used to implement a radio frequency transceiver for use in a VVTRU, terminal, base station, Rokay.NG, and/or any host computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un dispositif qui peut recevoir une vidéo à 360 degrés comprenant une ou plusieurs trames. Les trames peuvent comprendre des faces multiples et/ou peuvent être associées à une ou à plusieurs fonctions de transformée paramétrées. Lesdites fonctions de transformée paramétrées peuvent être associées à un paramètre de fonction de transformée. Par exemple, un paramètre de fonction de transformée et/ou une fonction de transformée paramétrée peuvent être définis pour chaque visage et/ou dans chaque direction. Le dispositif peut rechercher dans un espace de paramètre un premier paramètre de fonction de transformée pour une première trame. Le dispositif peut déterminer une plage de recherche progressive (PSR) qui peut être relative au premier paramètre de fonction de transformée. Par exemple, la PSR peut comprendre une plage qui entoure le premier paramètre de fonction de transformée. Le dispositif peut rechercher dans la PSR un second paramètre de fonction de transformée d'une seconde trame. Le dispositif peut signaler le premier et le second paramètre de fonction de transformée dans un flux binaire vidéo.
PCT/US2018/057015 2017-10-24 2018-10-23 Projection de carte cubemap angulaire hybride de codage vidéo à 360 degrés WO2019083943A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201762576413P 2017-10-24 2017-10-24
US62/576,413 2017-10-24
US201862621780P 2018-01-25 2018-01-25
US62/621,780 2018-01-25
US201862678804P 2018-05-31 2018-05-31
US62/678,804 2018-05-31

Publications (1)

Publication Number Publication Date
WO2019083943A1 true WO2019083943A1 (fr) 2019-05-02

Family

ID=64308808

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/057015 WO2019083943A1 (fr) 2017-10-24 2018-10-23 Projection de carte cubemap angulaire hybride de codage vidéo à 360 degrés

Country Status (1)

Country Link
WO (1) WO2019083943A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020253816A1 (fr) * 2019-06-21 2020-12-24 Huawei Technologies Co., Ltd. Codeur, décodeur et procédés correspondants pour un mode de partitionnement de sous-bloc
WO2021000904A1 (fr) * 2019-07-02 2021-01-07 Mediatek Inc. Procédé de codage vidéo avec signalisation d'élément de syntaxe de fonction de mappage utilisée par projection cubique et procédé de décodage vidéo associé
US11190768B2 (en) 2019-07-02 2021-11-30 Mediatek Inc. Video encoding method with syntax element signaling of packing of projection faces derived from cube-based projection and associated video decoding method and apparatus
US11659206B2 (en) 2019-07-02 2023-05-23 Mediatek Inc. Video encoding method with syntax element signaling of guard band configuration of projection-based frame and associated video decoding method and apparatus

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
"Algorithm descriptions of projection format conversion and video quality metrics in 360Lib Version 4", 119. MPEG MEETING;17-7-2017 - 21-7-2017; TORINO; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), no. N17056, 1 October 2017 (2017-10-01), XP030023717 *
COBAN M ET AL: "AHG8: Adjusted cubemap projection for 360-degree video", 6. JVET MEETING; 31-3-2017 - 7-4-2017; HOBART; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://PHENIX.INT-EVRY.FR/JVET/, no. JVET-F0025-v3, 2 April 2017 (2017-04-02), XP030150675 *
HANHART P ET AL: "Description of Core Experiment 13 (CE13): Projection formats", 10. JVET MEETING; 10-4-2018 - 20-4-2018; SAN DIEGO; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://PHENIX.INT-EVRY.FR/JVET/, no. JVET-J1033-v2, 12 May 2018 (2018-05-12), XP030151337 *
XIU X ET AL: "Description of SDR, HDR and 360° video coding technology proposal by InterDigital Communications and Dolby Laboratories", 10. JVET MEETING; 10-4-2018 - 20-4-2018; SAN DIEGO; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://PHENIX.INT-EVRY.FR/JVET/, no. JVET-J0015, 3 April 2018 (2018-04-03), XP030151174 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020253816A1 (fr) * 2019-06-21 2020-12-24 Huawei Technologies Co., Ltd. Codeur, décodeur et procédés correspondants pour un mode de partitionnement de sous-bloc
US11539948B2 (en) 2019-06-21 2022-12-27 Huawei Technologies Co., Ltd. Encoder, a decoder and corresponding methods for sub-block partitioning mode
US11962773B2 (en) 2019-06-21 2024-04-16 Huawei Technologies Co., Ltd. Encoder, decoder and corresponding methods for sub-block partitioning mode
WO2021000904A1 (fr) * 2019-07-02 2021-01-07 Mediatek Inc. Procédé de codage vidéo avec signalisation d'élément de syntaxe de fonction de mappage utilisée par projection cubique et procédé de décodage vidéo associé
US11190801B2 (en) 2019-07-02 2021-11-30 Mediatek Inc. Video encoding method with syntax element signaling of mapping function employed by cube-based projection and associated video decoding method
US11190768B2 (en) 2019-07-02 2021-11-30 Mediatek Inc. Video encoding method with syntax element signaling of packing of projection faces derived from cube-based projection and associated video decoding method and apparatus
US11659206B2 (en) 2019-07-02 2023-05-23 Mediatek Inc. Video encoding method with syntax element signaling of guard band configuration of projection-based frame and associated video decoding method and apparatus

Similar Documents

Publication Publication Date Title
CN110915217B (zh) 360度视频译码方法、设备和可读介质
US20230188752A1 (en) Sample Derivation For 360-degree Video Coding
CN111713111B (zh) 用于360度视频译码的面不连续性滤波
CN111183646B (zh) 用于编码的方法和装置、用于解码的方法和装置以及存储介质
CN110999296B (zh) 解码360度视频的方法、设备及计算机可读介质
WO2018191224A1 (fr) Codage de vidéo à 360 degrés utilisant des continuités de face
WO2018067952A1 (fr) Conversion de géométrie et compression de trame associées à des vidéos à 360 degrés
WO2019089382A1 (fr) Codage vidéo à 360 degrés à l'aide d'un remplissage géométrique basé sur un visage
WO2019083943A1 (fr) Projection de carte cubemap angulaire hybride de codage vidéo à 360 degrés
TWI797234B (zh) 360度視訊編碼適應性訊框填裝
US11223848B2 (en) Weighted to spherically uniform PSNR for 360-degree video quality evaluation using cubemap-based projections
US12003770B2 (en) Face discontinuity filtering for 360-degree video coding
RU2786015C2 (ru) Получение сэмпла для кодирования 360-градусного видео

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18803504

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18803504

Country of ref document: EP

Kind code of ref document: A1