WO2019037656A1 - Method and apparatus of signalling syntax for immersive video coding - Google Patents

Method and apparatus of signalling syntax for immersive video coding Download PDF

Info

Publication number
WO2019037656A1
WO2019037656A1 PCT/CN2018/100953 CN2018100953W WO2019037656A1 WO 2019037656 A1 WO2019037656 A1 WO 2019037656A1 CN 2018100953 W CN2018100953 W CN 2018100953W WO 2019037656 A1 WO2019037656 A1 WO 2019037656A1
Authority
WO
WIPO (PCT)
Prior art keywords
viewport
virtual reality
displacement
reality images
degree virtual
Prior art date
Application number
PCT/CN2018/100953
Other languages
French (fr)
Inventor
Peng Wang
Hung-Chih Lin
Jian-Liang Lin
Shen-Kai Chang
Original Assignee
Mediatek Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mediatek Inc. filed Critical Mediatek Inc.
Priority to EP18847331.8A priority Critical patent/EP3656128A4/en
Priority to CN201880050495.5A priority patent/CN111418213B/en
Priority to US16/105,147 priority patent/US10827159B2/en
Priority to TW107129203A priority patent/TWI684359B/en
Publication of WO2019037656A1 publication Critical patent/WO2019037656A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression

Definitions

  • the present invention relates to image/video processing or coding for 360-degree virtual reality (VR) images/sequences.
  • the present invention relates to syntax signalling for immersive video coding in a pyramid projection format.
  • the 360-degree video also known as immersive video is an emerging technology, which can provide “feeling as sensation of present” .
  • the sense of immersion is achieved by surrounding a user with wrap-around scene covering a panoramic view, in particular, 360-degree field of view.
  • the “feeling as sensation of present” can be further improved by stereographic rendering. Accordingly, the panoramic video is being widely used in Virtual Reality (VR) applications.
  • VR Virtual Reality
  • Immersive video involves the capturing a scene using multiple cameras to cover a panoramic view, such as 360-degree field of view.
  • the immersive camera usually uses a panoramic camera or a set of cameras arranged to capture 360-degree field of view. Typically, two or more cameras are used for the immersive camera. All videos must be taken simultaneously and separate fragments (also called separate perspectives) of the scene are recorded. Furthermore, the set of cameras are often arranged to capture views horizontally, while other arrangements of the cameras are possible.
  • the 360-degree virtual reality (VR) images may be captured using a 360-degree spherical panoramic camera or multiple images arranged to cover all filed of views around 360 degrees.
  • the three-dimensional (3D) spherical image is difficult to process or store using the conventional image/video processing devices. Therefore, the 360-degree VR images are often converted to a two-dimensional (2D) format using a 3D-to-2D projection method.
  • 2D two-dimensional
  • equirectangular projection (ERP) and cubemap projection (CMP) have been commonly used projection methods. Accordingly, a 360-degree image can be stored in an equirectangular projected format.
  • the equirectangular projection maps the entire surface of a sphere onto a flat image.
  • the vertical axis is latitude and the horizontal axis is longitude.
  • the areas in the north and south poles of the sphere are stretched more severely (i.e., from a single point to a line) than areas near the equator.
  • predictive coding tools often fail to make good prediction, causing reduction in coding efficiency.
  • syntax signalling related to new projection formats is disclosed.
  • Methods and apparatus of processing 360-degree virtual reality images are disclosed. According to one method, at a source side or an encoder side, input data for 360-degree virtual reality images are received. A selected viewport associated with the 360-degree virtual reality images is determined. One or more parameters related to the selected pyramid projection format corresponding to the selected viewport are then determined. The pyramid projection format includes a primary viewport face and four secondary viewport faces. According to the present invention, one or more syntax elements for said one or more parameters are included in coded data of the 360-degree virtual reality images. The coded data of the 360-degree virtual reality images are provided as output data.
  • coded data for 360-degree virtual reality images are received.
  • One or more syntax elements for one or more parameters are parsed from the coded data of the 360-degree virtual reality images.
  • a selected pyramid projection format associated with the 360-degree virtual reality images is determined based on information including said one or more parameters.
  • the pyramid projection format includes a primary viewport face and four secondary viewport faces.
  • the 360-degree virtual reality images are then recovered according to a selected viewport corresponding to the selected pyramid projection format.
  • said one or more parameters comprise yaw of main viewport centre, pitch of main viewport centre, width of main viewport face, FOV (field of view) angle of main viewport face, non-uniform factor for the four secondary viewport faces, or a combination thereof.
  • said one or more parameters comprise a packing type, a displacement indicator regarding whether secondary viewport face centre displacement is allowed, horizontal centre displacement, vertical centre displacement, or a combination thereof. When the displacement indicator indicates the secondary viewport centre displacement is allowed, one or more syntax element for the horizontal centre displacement, the vertical centre displacement, or both are included in the coded data of the 360-degree virtual reality images.
  • Fig. 1A illustrates an example of viewport that is represented as a pyramid.
  • the 360VR video contents on the sphere are projected on an inscribed pyramid, which comprises an apex, a rectangular base and four triangle lateral faces.
  • Fig. 1B illustrates an example of the pyramid consisting of five faces referred as Front or Main and four lateral faces labelled as R (right) , L (left) , T (Top) and B (Bottom) .
  • Fig. 1C illustrates an example of compact pyramid projection layout, where the triangle faces with shrunk height are rearranged with the Front face to form a compact layout.
  • Fig. 1D illustrates an example of exponential non-uniform mapping function.
  • Fig. 2 illustrates an example of viewpoint based pyramid projection, where the viewpoint centre, the value of yaw (i.e., longitude) ⁇ and the value of pitch (i.e., latitude) ⁇ are indicated.
  • Fig. 4 depicts an example of the viewport represented as a pyramid enclosed in a sphere, where related parameters are indicated.
  • Fig. 5 illustrates examples of asymmetric pyramid projection layout in two packing types (first type, second type) , where the primary view and the secondary view are stacked horizontally.
  • Fig. 6 illustrates examples of asymmetric pyramid projection layout in two packing types (third type, fourth type) , where the primary view and the secondary view are stacked vertically.
  • Fig. 7 illustrates an example of asymmetric pyramid projection layout, where the apex location is offset from the default centre in both horizontal and vertical directions.
  • Fig. 8 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a source side or encoder side according to an embodiment of the present invention.
  • Fig. 9 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a receiver side or decoder side according to an embodiment of the present invention.
  • a viewport-based pyramid projection format for VR360 video streaming has been disclosed in JVET-E0058 (Peng Wang, et al., “AHG 8: A viewport-based pyramid projection for VR360 video streaming” , Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 5th Meeting: Geneva, CH, 12–20 January 2017, Document: JVET-E0058) .
  • the viewport-based pyramid projection format is composed of two regions: primary region and secondary region.
  • the primary area preserves high-fidelity viewing region specified by a pre-defined FOV while the secondary area represents a down-sampled version of the region outside the primary area.
  • the bandwidth of the viewport-based projection format can be significantly reduced for streaming application.
  • the viewport switching scheme allows viewing the VR videos in different viewports. These primary areas of these viewport videos comprise the entire 360°x180° region. Usually, the centre of the primary area of each viewport video is indicated by the yaw and pitch angles. Thus, according to the user’s viewing angle, the user can receive the corresponding viewport video that encompasses the viewing area.
  • the viewport is represented as a pyramid and the 360VR video contents on the sphere are projected on an inscribed pyramid, which comprises an apex, a rectangular base and four triangle lateral faces.
  • the base of the pyramid corresponds to a square.
  • the pyramid consists of five faces (i.e., the square base 110 and four triangle lateral faces) as shown in Fig. 1B, where the base face 110 is referred as Front or Main and the four lateral faces are labelled as R (right) , L (left) , T (Top) and B (Bottom) .
  • the height of each unfolded lateral faces is referred as H as shown in Fig. 1B.
  • the main face is the primary viewport face, which contains a 90ox90o region.
  • the other four isosceles triangle faces are referred as secondary faces.
  • these triangle faces with shrunk height (labelled as h) are rearranged with the Front face 120 to form a very compact layout.
  • the four secondary faces with shrunk height in a compact format is referred as a compact secondary view 130.
  • the whole image (i.e., the primary view 120 and the compact secondary view 130) in the compact format is referred as a pyramid compact layout.
  • the height of the four triangle faces is shrunk from H to h.
  • the triangle faces can be down-sampled vertically to reduce the number of samples.
  • a linear mapping function can be used to shrink from H to h in the vertical direction.
  • a re-sampling process may use a non-uniform mapping function to result in denser samples of the corresponding lateral projection face U/B/L/R near the main projection face.
  • the non-uniform mapping function is an exponential function as shown in Fig. 1D
  • the non-uniform mapping function may be expressed by using the following equation.
  • n is positive and n ⁇ 1
  • Y is the vertical coordinate of an original lateral projection face
  • y’ is the vertical coordinate of the vertically shrunk lateral projection face.
  • the parameter n is referred as a non-uniform factor in this disclosure.
  • the non-uniformly down-sampling process can achieve that the degradation of the image quality of the projection area close to the primary region is lower than that of the area far from the primary region.
  • syntax elements for the parameters associated with a selected pyramid project format are disclosed in the present invention.
  • the syntax elements may include one of more the following syntax elements:
  • main_viewpoint_center_yaw specifies the value of yaw (i.e., longitude) ⁇ .
  • the rotation direction is clockwise and the value range is [-180°, 180°].
  • main_viewpoint_center_pitch specifies the value of pitch (i.e., latitude) ⁇ .
  • the rotation direction is clockwise and the value range is [-90°, 90°].
  • Fig. 2 illustrates an example of view-point based pyramid projection, where the view-point centre is indicated by a black dot; the value of yaw (i.e., longitude) ⁇ and the value of pitch (i.e., latitude) ⁇ are indicated.
  • yaw i.e., longitude
  • pitch i.e., latitude
  • the centre of the main face is indicated by a white dot.
  • the boundaries of the four triangle faces are indicated by white lines in Fig. 3.
  • the image contents are continuous across the boundaries.
  • the main_viewpoint_face_width specifies the pixels of the width and height of the main viewport face.
  • the main_viewpoint_face_width for the exemplary picture in a pyramid projection format is indicated in Fig. 3.
  • the viewport is represented as a pyramid enclosed in a sphere as shown in illustration 410.
  • the main_viewpoint_fov_angle specifies the degree of angle, which defines the field-of-view (FOV) size of the squared primary viewing region.
  • d is the distance between the centre of sphere and main viewport face.
  • a cross section of the pyramid passing the apex and a line splitting the primary view into two rectangles of equal size is represented in Fig. 4.
  • the width (w) of main viewport face and the height (h’) of pyramid are calculated as:
  • Fig. 4 also illustrates the related parameters for deriving the width (w) of main viewport face and the height (h’) of pyramid.
  • illustration 410 shows a perspective view of a pyramid enclosed in a sphere with the main face in the front, where the width (w) of the main viewport face is indicated.
  • Illustration 420 represents a side view of the pyramid enclosed in the sphere, where the radius (R) of the sphere, the height (h’) of pyramid, the centre of the sphere 422, the distance (d) between the centre of sphere and the value of field-of-view (FOV) angle ( ⁇ ) are indicated.
  • FOV field-of-view
  • Illustration 430 represents a three-dimensional view of the pyramid enclosed in the sphere, where the radius (R) of the sphere, the height (h’) of pyramid, the centre of the sphere 422 and the distance (d) between the centre of sphere and main viewport face are indicated.
  • Figs. 1A-C the four triangle lateral faces are symmetrical around the main face and have the same shape and size.
  • the four triangle lateral faces are folded into a square in the right side of the layout in Fig. 1C, the four triangle lateral faces are symmetric with the apex joined in the centre of the square on the right side of the layout in Fig. 1C.
  • pyramid projection layout with asymmetric lateral faces is also disclosed.
  • Fig. 5 illustrates examples of asymmetric pyramid projection layout in two packing types (first type 510, second type 520) , where the primary view and the secondary view are stacked horizontally.
  • the picture width and height of the asymmetric pyramid projection layout are referred as Wa and Ha respectively.
  • the frame resolution of the asymmetric pyramid projection layout is Wa ⁇ Ha.
  • the resolution of the main view is Ha ⁇ Ha.
  • the resolution of the secondary view is (Wa -Ha) ⁇ Ha.
  • the apex location (512, 522) is offset from the default centre (514, 524) of the secondary view and the horizontal offset distance is indicated (516 and 526) in Fig. 5 respectively.
  • Fig. 6 illustrates examples of asymmetric pyramid projection layout in two packing types (third type 610, fourth type 620) , where the primary view and the secondary view are stacked vertically.
  • the picture width and height of the asymmetric pyramid projection layout are referred as Wa and Ha respectively.
  • the frame resolution of the asymmetric pyramid projection layout is Wa ⁇ Ha.
  • the resolution of the main view is Wa ⁇ Wa.
  • the resolution of the secondary view is Wa ⁇ (Ha -Wa) .
  • the apex location (612, 622) is offset from the default centre (614, 624) of the secondary view and the vertical offset distance is indicated (616 and 626) in Fig. 6 respectively.
  • Fig. 7 illustrates an example of asymmetric pyramid projection layout 710, where the primary view and the secondary view are stacked horizontally (i.e., first type) .
  • the apex location (712) is offset from the default centre (714) in both horizontal and vertical directions.
  • the horizontal offset (716) and vertical offset (718) from the default centre of the secondary view are indicated in Fig. 7.
  • the present invention also discloses additional syntax elements to be signalled in the video bitstream so that a decoder can recover the selected asymmetric pyramid projection layout accordingly.
  • the additional syntax elements comprise:
  • Syntax element packing_type identifies which of the four types is selected.
  • Syntax element disable_center_displacement identifies whether centre displacement is disabled. If disable_center_displacement is equal to 1, center_displacement_x and center_displacement_y are inferred to 0; otherwise, the centre displacements are signalled using syntax elements center_displacement_x and center_displacement_y (in unit of pixel) .
  • the apex centre of the asymmetric pyramid projection layouts can be determined from the default centre of the secondary view and the offset values.
  • the x coordinate x_c (measured from left edge of the frame) and the y coordinate y_c (measured from top edge of the frame) of the default centre of the secondary view can be determined for the four types according to:
  • the center_displacement_x When the apex is on the right of the default centre, the center_displacement_x >0. When the apex is on the left of the default centre, the center_displacement_x ⁇ 0. When the apex is above of the default centre, the center_displacement_y>0. When the apex is below the default centre, the center_displacement_y ⁇ 0.
  • the centre coordinate of the apex is calculated as follows for four types when disable_center_displacement is equal to 0 (i.e., asymmetric projection format allowed) :
  • Types 1 and 2 (x_c + center_displacement_x, y_c) ;
  • Types 3 and 4 (x_c, y_c + center_displacement_y) .
  • the centre coordinate of the apex is calculated as follows when disable_center_displacement is equal to 0 (i.e., asymmetric projection format allowed) :
  • Fig. 8 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a source side or encoder side according to an embodiment of the present invention.
  • the steps shown in the flowchart may be implemented as program codes executable on one or more processors (e.g., one or more CPUs) at the encoder side.
  • the steps shown in the flowchart may also be implemented based hardware such as one or more electronic devices or processors arranged to perform the steps in the flowchart.
  • input data for 360-degree virtual reality images are received in step 810.
  • a selected viewport associated with the 360-degree virtual reality images is determined in step 820.
  • One or more parameters related to the selected pyramid projection format corresponding to the selected viewport, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces are determined in step 830.
  • One or more syntax elements for said one or more parameters are included in coded data of the 360-degree virtual reality images in step 840.
  • the coded data of the 360-degree virtual reality images are provided as output data in step 850.
  • Fig. 9 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a receiver side or decoder side according to an embodiment of the present invention.
  • Coded data for 360-degree virtual reality images are received in step 910.
  • One or more syntax elements for one or more parameters are parsed from the coded data of the 360-degree virtual reality images in step 920.
  • a selected pyramid projection format associated with the 360-degree virtual reality images is determined based on information including said one or more parameters in step 930, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces.
  • the 360-degree virtual reality images are recovered according to a selected viewport corresponding to the selected pyramid projection format in step 940.
  • Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both.
  • an embodiment of the present invention can be one or more electronic circuits integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein.
  • An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein.
  • DSP Digital Signal Processor
  • the invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA) .
  • These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention.
  • the software code or firmware code may be developed in different programming languages and different formats or styles.
  • the software code may also be compiled for different target platforms.
  • different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

According to one method, at a source side or an encoder side, a selected viewport associated with the 360-degree virtual reality images is determined. One or more parameters related to the selected pyramid projection format are then determined. According to the present invention, one or more syntax elements for said one or more parameters are included in coded data of the 360-degree virtual reality images. The coded data of the 360-degree virtual reality images are provided as output data. At a receiver side or a decoder side, one or more syntax elements for one or more parameters are parsed from the coded data of the 360-degree virtual reality images. A selected pyramid projection format associated with the 360-degree virtual reality images is determined based on information including said one or more parameters. The 360-degree virtual reality images are then recovered according to the selected viewport.

Description

METHOD AND APPARATUS OF SIGNALLING SYNTAX FOR IMMERSIVE VIDEO CODING
The present invention claims priority to U.S. Provisional Patent Application, Serial No. 62/549,016, filed on August 23, 2017. The U.S. Provisional Patent Application is hereby incorporated by reference in its entirety.
FIELD OF THE INVENTION
The present invention relates to image/video processing or coding for 360-degree virtual reality (VR) images/sequences. In particular, the present invention relates to syntax signalling for immersive video coding in a pyramid projection format.
BACKGROUND AND RELATED ART
The 360-degree video, also known as immersive video is an emerging technology, which can provide “feeling as sensation of present” . The sense of immersion is achieved by surrounding a user with wrap-around scene covering a panoramic view, in particular, 360-degree field of view. The “feeling as sensation of present” can be further improved by stereographic rendering. Accordingly, the panoramic video is being widely used in Virtual Reality (VR) applications.
Immersive video involves the capturing a scene using multiple cameras to cover a panoramic view, such as 360-degree field of view. The immersive camera usually uses a panoramic camera or a set of cameras arranged to capture 360-degree field of view. Typically, two or more cameras are used for the immersive camera. All videos must be taken simultaneously and separate fragments (also called separate perspectives) of the scene are recorded. Furthermore, the set of cameras are often arranged to capture views horizontally, while other arrangements of the cameras are possible.
The 360-degree virtual reality (VR) images may be captured using a 360-degree spherical panoramic camera or multiple images arranged to cover all filed of views around 360 degrees. The three-dimensional (3D) spherical image is difficult to process or store using the conventional image/video processing devices. Therefore, the 360-degree VR images are often converted to a two-dimensional (2D) format using a 3D-to-2D projection method. For example, equirectangular projection (ERP) and cubemap projection (CMP) have been commonly used projection methods. Accordingly, a 360-degree image can be stored in an equirectangular projected format. The equirectangular projection maps the entire surface of a sphere onto a flat image. The vertical axis is latitude and the horizontal axis is longitude. For the ERP projection, the areas in the north and south poles of the sphere are stretched more severely (i.e., from a single point to a line) than areas near the equator. Furthermore, due to distortions introduced by the stretching, especially near the two poles, predictive coding tools often fail to make good prediction, causing reduction in coding efficiency.
In the present invention, syntax signalling related to new projection formats is disclosed.
BRIEF SUMMARY OF THE INVENTION
Methods and apparatus of processing 360-degree virtual reality images are disclosed. According to one  method, at a source side or an encoder side, input data for 360-degree virtual reality images are received. A selected viewport associated with the 360-degree virtual reality images is determined. One or more parameters related to the selected pyramid projection format corresponding to the selected viewport are then determined. The pyramid projection format includes a primary viewport face and four secondary viewport faces. According to the present invention, one or more syntax elements for said one or more parameters are included in coded data of the 360-degree virtual reality images. The coded data of the 360-degree virtual reality images are provided as output data.
At a receiver side or a decoder side, coded data for 360-degree virtual reality images are received. One or more syntax elements for one or more parameters are parsed from the coded data of the 360-degree virtual reality images. A selected pyramid projection format associated with the 360-degree virtual reality images is determined based on information including said one or more parameters. The pyramid projection format includes a primary viewport face and four secondary viewport faces. The 360-degree virtual reality images are then recovered according to a selected viewport corresponding to the selected pyramid projection format.
In one embodiment, said one or more parameters comprise yaw of main viewport centre, pitch of main viewport centre, width of main viewport face, FOV (field of view) angle of main viewport face, non-uniform factor for the four secondary viewport faces, or a combination thereof. In another embodiment, said one or more parameters comprise a packing type, a displacement indicator regarding whether secondary viewport face centre displacement is allowed, horizontal centre displacement, vertical centre displacement, or a combination thereof. When the displacement indicator indicates the secondary viewport centre displacement is allowed, one or more syntax element for the horizontal centre displacement, the vertical centre displacement, or both are included in the coded data of the 360-degree virtual reality images.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1A illustrates an example of viewport that is represented as a pyramid. The 360VR video contents on the sphere are projected on an inscribed pyramid, which comprises an apex, a rectangular base and four triangle lateral faces.
Fig. 1B illustrates an example of the pyramid consisting of five faces referred as Front or Main and four lateral faces labelled as R (right) , L (left) , T (Top) and B (Bottom) .
Fig. 1C illustrates an example of compact pyramid projection layout, where the triangle faces with shrunk height are rearranged with the Front face to form a compact layout.
Fig. 1D illustrates an example of exponential non-uniform mapping function.
Fig. 2 illustrates an example of viewpoint based pyramid projection, where the viewpoint centre, the value of yaw (i.e., longitude) θ and the value of pitch (i.e., latitude) φ are indicated.
Fig. 3 depicts an example of the viewport image of the proposed layout with (yaw, pitch) = (0, 0) and non-uniform factor n=2.5.
Fig. 4 depicts an example of the viewport represented as a pyramid enclosed in a sphere, where related parameters are indicated.
Fig. 5 illustrates examples of asymmetric pyramid projection layout in two packing types (first type, second type) , where the primary view and the secondary view are stacked horizontally.
Fig. 6 illustrates examples of asymmetric pyramid projection layout in two packing types (third type,  fourth type) , where the primary view and the secondary view are stacked vertically.
Fig. 7 illustrates an example of asymmetric pyramid projection layout, where the apex location is offset from the default centre in both horizontal and vertical directions.
Fig. 8 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a source side or encoder side according to an embodiment of the present invention.
Fig. 9 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a receiver side or decoder side according to an embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
A viewport-based pyramid projection format for VR360 video streaming has been disclosed in JVET-E0058 (Peng Wang, et al., “AHG 8: A viewport-based pyramid projection for VR360 video streaming” , Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 5th Meeting: Geneva, CH, 12–20 January 2017, Document: JVET-E0058) . The viewport-based pyramid projection format is composed of two regions: primary region and secondary region. The primary area preserves high-fidelity viewing region specified by a pre-defined FOV while the secondary area represents a down-sampled version of the region outside the primary area. In other words, the bandwidth of the viewport-based projection format can be significantly reduced for streaming application. The viewport switching scheme allows viewing the VR videos in different viewports. These primary areas of these viewport videos comprise the entire 360°x180° region. Usually, the centre of the primary area of each viewport video is indicated by the yaw and pitch angles. Thus, according to the user’s viewing angle, the user can receive the corresponding viewport video that encompasses the viewing area.
In Fig. 1A, the viewport is represented as a pyramid and the 360VR video contents on the sphere are projected on an inscribed pyramid, which comprises an apex, a rectangular base and four triangle lateral faces. In the particular example shown in Fig. 1A, the base of the pyramid corresponds to a square. The pyramid consists of five faces (i.e., the square base 110 and four triangle lateral faces) as shown in Fig. 1B, where the base face 110 is referred as Front or Main and the four lateral faces are labelled as R (right) , L (left) , T (Top) and B (Bottom) . The height of each unfolded lateral faces is referred as H as shown in Fig. 1B. The main face is the primary viewport face, which contains a 90ox90o region. The other four isosceles triangle faces are referred as secondary faces. As illustrated in Fig. 1C, these triangle faces with shrunk height (labelled as h) are rearranged with the Front face 120 to form a very compact layout. In addition, there is no discontinuous edge between any two connected faces in the compact layout. In Fig. 1C, the four secondary faces with shrunk height in a compact format is referred as a compact secondary view 130. The whole image (i.e.., the primary view 120 and the compact secondary view 130) in the compact format is referred as a pyramid compact layout.
When packing the five faces into the compact layout, the height of the four triangle faces is shrunk from H to h. The triangle faces can be down-sampled vertically to reduce the number of samples. A linear mapping function can be used to shrink from H to h in the vertical direction. However, to preserve more details near the main projection face, a re-sampling process may use a non-uniform mapping function to result in denser samples of the  corresponding lateral projection face U/B/L/R near the main projection face. In a case where the non-uniform mapping function is an exponential function as shown in Fig. 1D, the non-uniform mapping function may be expressed by using the following equation.
Figure PCTCN2018100953-appb-000001
In the above equation, n is positive and n≠1, Y is the vertical coordinate of an original lateral projection face, and y’is the vertical coordinate of the vertically shrunk lateral projection face. The parameter n is referred as a non-uniform factor in this disclosure. The non-uniformly down-sampling process can achieve that the degradation of the image quality of the projection area close to the primary region is lower than that of the area far from the primary region.
In order to accommodate various view-point based pyramid projection formats, the parameters associated with a selected pyramid project format need to be signalled so that a decoder can properly reconstruct the VR video. Accordingly, syntax elements for the parameters associated with a selected pyramid project format are disclosed in the present invention. For example, the syntax elements may include one of more the following syntax elements:
signed int (16) main_viewpoint_center_yaw;
signed int (16) main_viewpoint_center_pitch;
unsigned int (16) main_viewpoint_face_width;
unsigned int (8) main_viewpoint_fov_angle;
unsigned int (32) non_uniform_factor;
In the above list, main_viewpoint_center_yaw specifies the value of yaw (i.e., longitude) θ. The rotation direction is clockwise and the value range is [-180°, 180°]. In the above list, main_viewpoint_center_pitch specifies the value of pitch (i.e., latitude) φ. The rotation direction is clockwise and the value range is [-90°, 90°].
Fig. 2 illustrates an example of view-point based pyramid projection, where the view-point centre is indicated by a black dot; the value of yaw (i.e., longitude) θ and the value of pitch (i.e., latitude) φ are indicated.
Fig. 3 depicts an example of the viewport image of the proposed layout with (yaw, pitch) = (0, 0) and non-uniform factor n = 2.5. In Fig. 3, the centre of the main face is indicated by a white dot. The boundaries of the four triangle faces are indicated by white lines in Fig. 3. As shown in Fig. 3, the image contents are continuous across the boundaries. The main_viewpoint_face_width specifies the pixels of the width and height of the main viewport face. The main_viewpoint_face_width for the exemplary picture in a pyramid projection format is indicated in Fig. 3.
In Fig. 4, the viewport is represented as a pyramid enclosed in a sphere as shown in illustration 410. The main_viewpoint_fov_angle specifies the degree of angle, which defines the field-of-view (FOV) size of the squared primary viewing region. Let d is the distance between the centre of sphere and main viewport face. A cross section of the pyramid passing the apex and a line splitting the primary view into two rectangles of equal size is represented in Fig. 4. The width (w) of main viewport face and the height (h’) of pyramid are calculated as:
Figure PCTCN2018100953-appb-000002
Figure PCTCN2018100953-appb-000003
Fig. 4 also illustrates the related parameters for deriving the width (w) of main viewport face and the height (h’) of pyramid. In Fig. 4, illustration 410 shows a perspective view of a pyramid enclosed in a sphere with the main face in the front, where the width (w) of the main viewport face is indicated. Illustration 420 represents a side view of the pyramid enclosed in the sphere, where the radius (R) of the sphere, the height (h’) of pyramid, the centre of the sphere 422, the distance (d) between the centre of sphere and the value of field-of-view (FOV) angle (θ) are indicated. Illustration 430 represents a three-dimensional view of the pyramid enclosed in the sphere, where the radius (R) of the sphere, the height (h’) of pyramid, the centre of the sphere 422 and the distance (d) between the centre of sphere and main viewport face are indicated.
In Figs. 1A-C, the four triangle lateral faces are symmetrical around the main face and have the same shape and size. When the four triangle lateral faces are folded into a square in the right side of the layout in Fig. 1C, the four triangle lateral faces are symmetric with the apex joined in the centre of the square on the right side of the layout in Fig. 1C. In the present invention, pyramid projection layout with asymmetric lateral faces is also disclosed. Fig. 5 illustrates examples of asymmetric pyramid projection layout in two packing types (first type 510, second type 520) , where the primary view and the secondary view are stacked horizontally. The picture width and height of the asymmetric pyramid projection layout are referred as Wa and Ha respectively. In other words, the frame resolution of the asymmetric pyramid projection layout is Wa × Ha. The resolution of the main view is Ha × Ha. The resolution of the secondary view is (Wa -Ha) × Ha. The apex location (512, 522) is offset from the default centre (514, 524) of the secondary view and the horizontal offset distance is indicated (516 and 526) in Fig. 5 respectively. In addition, there is no discontinuous edge between any two connected faces in the compact layout.
Fig. 6 illustrates examples of asymmetric pyramid projection layout in two packing types (third type 610, fourth type 620) , where the primary view and the secondary view are stacked vertically. The picture width and height of the asymmetric pyramid projection layout are referred as Wa and Ha respectively. In other words, the frame resolution of the asymmetric pyramid projection layout is Wa × Ha. The resolution of the main view is Wa ×Wa.The resolution of the secondary view is Wa × (Ha -Wa) . The apex location (612, 622) is offset from the default centre (614, 624) of the secondary view and the vertical offset distance is indicated (616 and 626) in Fig. 6 respectively. In addition, there is no discontinuous edge between any two connected faces in the compact layout.
Fig. 7 illustrates an example of asymmetric pyramid projection layout 710, where the primary view and the secondary view are stacked horizontally (i.e., first type) . The apex location (712) is offset from the default centre (714) in both horizontal and vertical directions. The horizontal offset (716) and vertical offset (718) from the default centre of the secondary view are indicated in Fig. 7. In addition, there is no discontinuous edge between any two connected faces in the compact layout.
In order to support the asymmetric pyramid projection layout, the present invention also discloses additional syntax elements to be signalled in the video bitstream so that a decoder can recover the selected asymmetric pyramid projection layout accordingly. The additional syntax elements comprise:
packing_type;
disable_center_displacement;
center_displacement_x;
center_displacement_y;
As mentioned earlier, there are four types of asymmetric pyramid projection layouts as shown in Fig. 5 and Fig. 6. Syntax element packing_type identifies which of the four types is selected. Syntax element disable_center_displacement identifies whether centre displacement is disabled. If disable_center_displacement is equal to 1, center_displacement_x and center_displacement_y are inferred to 0; otherwise, the centre displacements are signalled using syntax elements center_displacement_x and center_displacement_y (in unit of pixel) . The apex centre of the asymmetric pyramid projection layouts can be determined from the default centre of the secondary view and the offset values. The x coordinate x_c (measured from left edge of the frame) and the y coordinate y_c (measured from top edge of the frame) of the default centre of the secondary view can be determined for the four types according to:
Type 1: x_c = (Wa + Ha) /2, y_c = Ha /2;
Type 2: x_c = (Wa -Ha) /2, y_c = Ha /2;
Type 3: x_c = Wa /2, y_c = (Ha -Wa) /2;
Type 4: x_c = Wa /2, y_c = (Ha + Wa) /2.
When the apex is on the right of the default centre, the center_displacement_x >0. When the apex is on the left of the default centre, the center_displacement_x <0. When the apex is above of the default centre, the center_displacement_y>0. When the apex is below the default centre, the center_displacement_y <0. The centre coordinate of the apex is calculated as follows for four types when disable_center_displacement is equal to 0 (i.e., asymmetric projection format allowed) :
Types 1 and 2: (x_c + center_displacement_x, y_c) ;
Types 3 and 4: (x_c, y_c + center_displacement_y) .
For the asymmetric pyramid projection layout in Fig. 7, where the apex location (712) is offset from the default centre (714) in both horizontal and vertical directions, the centre coordinate of the apex is calculated as follows when disable_center_displacement is equal to 0 (i.e., asymmetric projection format allowed) :
(x_c + center_displacement_x, y_c + center_displacement_y) .
Fig. 8 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a source side or encoder side according to an embodiment of the present invention. The steps shown in the flowchart may be implemented as program codes executable on one or more processors (e.g., one or more CPUs) at the encoder side. The steps shown in the flowchart may also be implemented based hardware such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to this method, input data for 360-degree virtual reality images are received in step 810. A selected viewport associated with the 360-degree virtual reality images is determined in step 820. One or more parameters related to the selected pyramid projection format corresponding to the selected viewport, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces are determined in step 830. One or more syntax elements for said one or more parameters are included in coded data of the 360-degree virtual reality images in step 840. The coded data of the 360-degree virtual reality images are provided as output data in step 850.
Fig. 9 illustrates an exemplary flowchart of a system for processing 360-degree virtual reality images at a receiver side or decoder side according to an embodiment of the present invention. Coded data for 360-degree virtual reality images are received in step 910. One or more syntax elements for one or more parameters are parsed  from the coded data of the 360-degree virtual reality images in step 920. A selected pyramid projection format associated with the 360-degree virtual reality images is determined based on information including said one or more parameters in step 930, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces. The 360-degree virtual reality images are recovered according to a selected viewport corresponding to the selected pyramid projection format in step 940.
The flowcharts shown above are intended for serving as examples to illustrate embodiments of the present invention. A person skilled in the art may practice the present invention by modifying individual steps, splitting or combining steps with departing from the spirit of the present invention.
The above description is presented to enable a person of ordinary skill in the art to practice the present invention as provided in the context of a particular application and its requirement. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present invention. Nevertheless, it will be understood by those skilled in the art that the present invention may be practiced.
Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be one or more electronic circuits integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA) . These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.
The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims (16)

  1. A method of processing 360-degree virtual reality images, the method comprising:
    receiving input data for 360-degree virtual reality images;
    determining a selected viewport associated with the 360-degree virtual reality images;
    determining one or more parameters related to a selected pyramid projection format corresponding to the selected viewport, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces;
    including one or more syntax elements for said one or more parameters in coded data of the 360-degree virtual reality images; and
    providing the coded data of the 360-degree virtual reality images as output data.
  2. The method of Claim 1, wherein said one or more parameters comprise yaw of main viewport centre, pitch of main viewport centre, width of main viewport face, FOV (field of view) angle of main viewport face, non-uniform factor for the four secondary viewport faces, or a combination thereof.
  3. The method of Claim 1, wherein said one or more parameters comprise a packing type, a displacement indicator regarding whether secondary viewport face centre displacement is allowed, horizontal face centre displacement, vertical face centre displacement, or a combination thereof.
  4. The method of Claim 3, wherein when the displacement indicator indicates the secondary viewport face centre displacement is allowed, one or more syntax element for the horizontal face centre displacement, the vertical face centre displacement, or both are included in the coded data of the 360-degree virtual reality images.
  5. An apparatus for processing 360-degree virtual reality images, the apparatus comprising one or more electronic devices or processors configured to:
    receive input data for 360-degree virtual reality images;
    determine a selected viewport associated with the 360-degree virtual reality images;
    determine one or more parameters related to a selected pyramid projection format corresponding to the selected viewport, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces;
    include one or more syntax elements for said one or more parameters in coded data of the 360-degree virtual reality images; and
    provide the coded data of the 360-degree virtual reality images as output data.
  6. The apparatus of Claim 5, wherein said one or more parameters comprise yaw of main viewport centre, pitch of main viewport centre, width of main viewport face, FOV (field of view) angle of main viewport face, non-uniform factor for the four secondary viewport faces, or a combination thereof.
  7. The apparatus of Claim 5, wherein said one or more parameters comprise a packing type, a displacement indicator regarding whether secondary viewport face centre displacement is allowed, horizontal face centre displacement, vertical face centre displacement, or a combination thereof.
  8. The apparatus of Claim 7, wherein when the displacement indicator indicates the secondary viewport face centre displacement is allowed, one or more syntax element for the horizontal face centre displacement, the vertical face centre displacement, or both are included in the coded data of the 360-degree virtual reality images.
  9. A method of processing 360-degree virtual reality images, the method comprising:
    receiving coded data for 360-degree virtual reality images;
    parsing one or more syntax elements for one or more parameters from the coded data of the 360-degree virtual reality images;
    determining a selected pyramid projection format associated with the 360-degree virtual reality images based on information including said one or more parameters, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces; and
    recovering the 360-degree virtual reality images according to a selected viewport corresponding to the selected pyramid projection format.
  10. The method of Claim 9, wherein said one or more parameters comprise yaw of main viewport centre, pitch of main viewport centre, width of main viewport face, FOV (field of view) angle of main viewport face, non-uniform factor for the four secondary viewport faces, or a combination thereof.
  11. The method of Claim 9, wherein said one or more parameters comprise a packing type, a displacement indicator regarding whether secondary viewport face centre displacement is allowed, horizontal face centre displacement, vertical face centre displacement, or a combination thereof.
  12. The method of Claim 11, wherein when the displacement indicator indicates the secondary viewport face centre displacement is allowed, one or more syntax element for the horizontal face centre displacement, the vertical face centre displacement, or both are included in the coded data of the 360-degree virtual reality images.
  13. An apparatus for processing 360-degree virtual reality images, the apparatus comprising one or more electronic devices or processors configured to:
    receive coded data for 360-degree virtual reality images;
    parse one or more syntax elements for one or more parameters from the coded data of the 360-degree virtual reality images;
    determine a selected pyramid projection format associated with the 360-degree virtual reality images based on information including said one or more parameters, wherein the pyramid projection format includes a primary viewport face and four secondary viewport faces; and
    recover the 360-degree virtual reality images according to a selected viewport corresponding to the selected pyramid projection format.
  14. The apparatus of Claim 13 wherein said one or more parameters comprise yaw of main viewport centre, pitch of main viewport centre, width of main viewport face, FOV (field of view) angle of main viewport face, non-uniform factor for the four secondary viewport faces, or a combination thereof.
  15. The apparatus of Claim 13, wherein said one or more parameters comprise a packing type, a displacement indicator regarding whether secondary viewport face centre displacement is allowed, horizontal face centre displacement, vertical face centre displacement, or a combination thereof.
  16. The apparatus of Claim 15, wherein when the displacement indicator indicates the secondary viewport face centre displacement is allowed, one or more syntax element for the horizontal face centre displacement, the vertical face centre displacement, or both are included in the coded data of the 360-degree virtual reality images.
PCT/CN2018/100953 2017-08-23 2018-08-17 Method and apparatus of signalling syntax for immersive video coding WO2019037656A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP18847331.8A EP3656128A4 (en) 2017-08-23 2018-08-17 Method and apparatus of signalling syntax for immersive video coding
CN201880050495.5A CN111418213B (en) 2017-08-23 2018-08-17 Method and apparatus for signaling syntax for immersive video coding
US16/105,147 US10827159B2 (en) 2017-08-23 2018-08-20 Method and apparatus of signalling syntax for immersive video coding
TW107129203A TWI684359B (en) 2017-08-23 2018-08-22 Method and apparatus of signalling syntax for immersive video coding

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762549016P 2017-08-23 2017-08-23
US62/549,016 2017-08-23

Publications (1)

Publication Number Publication Date
WO2019037656A1 true WO2019037656A1 (en) 2019-02-28

Family

ID=65438393

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/100953 WO2019037656A1 (en) 2017-08-23 2018-08-17 Method and apparatus of signalling syntax for immersive video coding

Country Status (4)

Country Link
EP (1) EP3656128A4 (en)
CN (1) CN111418213B (en)
TW (1) TWI684359B (en)
WO (1) WO2019037656A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11190801B2 (en) 2019-07-02 2021-11-30 Mediatek Inc. Video encoding method with syntax element signaling of mapping function employed by cube-based projection and associated video decoding method
US11190768B2 (en) * 2019-07-02 2021-11-30 Mediatek Inc. Video encoding method with syntax element signaling of packing of projection faces derived from cube-based projection and associated video decoding method and apparatus
US11659206B2 (en) 2019-07-02 2023-05-23 Mediatek Inc. Video encoding method with syntax element signaling of guard band configuration of projection-based frame and associated video decoding method and apparatus

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105898254A (en) * 2016-05-17 2016-08-24 亿唐都科技(北京)有限公司 VR (Virtual Reality) panoramic video layout method and device and VR panoramic video presentation method and system capable of saving bandwidth
CN106358033A (en) * 2016-08-25 2017-01-25 北京字节跳动科技有限公司 A Panoramic Video Key Frame Coding Method and Device
CN106507094A (en) * 2016-10-31 2017-03-15 北京疯景科技有限公司 The method and device of correction panoramic video display view angle
CN106780317A (en) * 2017-01-25 2017-05-31 北京数码视讯科技股份有限公司 panoramic projection method and device
CN106803884A (en) * 2012-12-26 2017-06-06 株式会社理光 Image processing system and image processing method
US20170200255A1 (en) * 2016-01-07 2017-07-13 Mediatek Inc. Method and Apparatus of Image Formation and Compression of Cubic Images for 360 Degree Panorama Display

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110273451A1 (en) * 2010-05-10 2011-11-10 Salemann Leo J Computer simulation of visual images using 2d spherical images extracted from 3d data
CN101931772B (en) * 2010-08-19 2012-02-29 深圳大学 Panoramic video fusion method, system and video processing device
CN105245841B (en) * 2015-10-08 2018-10-09 北京工业大学 A kind of panoramic video monitoring system based on CUDA
CN106803994B (en) * 2017-03-29 2019-07-26 暴风集团股份有限公司 Identify the method and system of rectangular pyramid panoramic video

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106803884A (en) * 2012-12-26 2017-06-06 株式会社理光 Image processing system and image processing method
US20170200255A1 (en) * 2016-01-07 2017-07-13 Mediatek Inc. Method and Apparatus of Image Formation and Compression of Cubic Images for 360 Degree Panorama Display
CN105898254A (en) * 2016-05-17 2016-08-24 亿唐都科技(北京)有限公司 VR (Virtual Reality) panoramic video layout method and device and VR panoramic video presentation method and system capable of saving bandwidth
CN106358033A (en) * 2016-08-25 2017-01-25 北京字节跳动科技有限公司 A Panoramic Video Key Frame Coding Method and Device
CN106507094A (en) * 2016-10-31 2017-03-15 北京疯景科技有限公司 The method and device of correction panoramic video display view angle
CN106780317A (en) * 2017-01-25 2017-05-31 北京数码视讯科技股份有限公司 panoramic projection method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3656128A4 *

Also Published As

Publication number Publication date
EP3656128A4 (en) 2020-09-16
TW201921950A (en) 2019-06-01
EP3656128A1 (en) 2020-05-27
TWI684359B (en) 2020-02-01
CN111418213A (en) 2020-07-14
CN111418213B (en) 2022-07-01

Similar Documents

Publication Publication Date Title
EP3669333B1 (en) Sequential encoding and decoding of volymetric video
US20200296282A1 (en) Imaging system, imaging apparatus, and system
US10264282B2 (en) Method and apparatus of inter coding for VR video using virtual reference frames
EP3198866B1 (en) Reconstruction of three-dimensional video
WO2017190710A1 (en) Method and apparatus for mapping omnidirectional image to a layout output format
US20180098090A1 (en) Method and Apparatus for Rearranging VR Video Format and Constrained Encoding Parameters
WO2018196682A1 (en) Method and apparatus for mapping virtual-reality image to a segmented sphere projection format
US9436973B2 (en) Coordinate computation device and method, and an image processing device and method
KR20170017700A (en) Electronic Apparatus generating 360 Degrees 3D Stereoscopic Panorama Images and Method thereof
WO2013108339A1 (en) Stereo imaging device
US10827159B2 (en) Method and apparatus of signalling syntax for immersive video coding
CN110730296A (en) Image processing apparatus, image processing method, and computer readable medium
CN109691104B (en) Method and device for processing 360-degree virtual reality image
WO2019037656A1 (en) Method and apparatus of signalling syntax for immersive video coding
WO2017168347A1 (en) Multi-camera image coding
US11270413B2 (en) Playback apparatus and method, and generation apparatus and method
US20180338160A1 (en) Method and Apparatus for Reduction of Artifacts in Coded Virtual-Reality Images
US20200068205A1 (en) Geodesic intra-prediction for panoramic video coding
JP2013223008A (en) Image processing device and method
US9338429B2 (en) Video processing apparatus capable of reproducing video content including a plurality of videos and control method therefor
WO2019008222A1 (en) A method and apparatus for encoding media content
KR20190074490A (en) Image processing apparatus and method for image processing thereof
CN114257733A (en) Method and system for image processing of omni-directional image with viewpoint offset
WO2013156250A1 (en) View synthesis using low resolution depth maps
KR101947799B1 (en) 360 degrees Fisheye Rendering Method for Virtual Reality Contents Service

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18847331

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018847331

Country of ref document: EP

Effective date: 20200219