WO2019037365A1 - 虚拟现实全景视频流投影方法和设备 - Google Patents

虚拟现实全景视频流投影方法和设备 Download PDF

Info

Publication number
WO2019037365A1
WO2019037365A1 PCT/CN2017/118537 CN2017118537W WO2019037365A1 WO 2019037365 A1 WO2019037365 A1 WO 2019037365A1 CN 2017118537 W CN2017118537 W CN 2017118537W WO 2019037365 A1 WO2019037365 A1 WO 2019037365A1
Authority
WO
WIPO (PCT)
Prior art keywords
regions
spherical
user
region
sub
Prior art date
Application number
PCT/CN2017/118537
Other languages
English (en)
French (fr)
Inventor
马睿
马志友
Original Assignee
深圳看到科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳看到科技有限公司 filed Critical 深圳看到科技有限公司
Priority to US16/640,796 priority Critical patent/US20210368148A1/en
Publication of WO2019037365A1 publication Critical patent/WO2019037365A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/16—Spatio-temporal transformations, e.g. video cubism
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/756—Media network packet handling adapting media to device capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/282—Image signal generators for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H04N9/3185—Geometric adjustment, e.g. keystone or convergence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H04N9/3188—Scale or resolution adjustment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3191—Testing thereof
    • H04N9/3194—Testing thereof including sensor feedback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/764—Media network packet handling at the destination 

Definitions

  • the present invention relates to the field of virtual reality technologies, and more particularly to a virtual reality panoramic video stream projection method and apparatus.
  • Virtual Reality is a technology that has been applied in video, photography, cinema and game scenarios. It includes cross-combination of multimedia, human-computer interaction, sensors, and network technologies. Virtual reality can create a subjective and freely viewable virtual world based on the user's vision, hearing and even touch. It brings a high degree of immersion and participation to users, and is an important development direction in the future of multimedia and online entertainment. Virtual reality technology generally includes two parts: related hardware and software. Virtual reality hardware includes, for example, human body trackers and sensors, user input devices, 3D displays, projection systems, head mounted displays, stereoscopic systems, motion capture devices, eye tracking devices, and other interactive devices. Virtual reality software includes display drivers, data transmission and codec algorithms required in virtual reality video and games.
  • a virtual reality device such as a host establishes data communication, so that the user can experience the application of the virtual reality video through the network in real time. Since the virtual reality video must include visual information around all angles within the user's spherical range so that the user can view any angle, this requires real-time streaming to propagate high-definition video data that consumes a large amount of bandwidth and other network resources.
  • the universal resolution of the virtual reality video is high, and the code rate is high, the network condition of the general user is difficult to meet the needs of streaming virtual reality video.
  • the viewing angle of the user at any time must be limited, and it is impossible to view all the images in the sphere.
  • the image that is seen is changed only when the user turns to look at other angles, and the images in the remaining range are equal to not functioning and waste network resources. Therefore, it is necessary to save network resources while ensuring the video quality within the user's viewing angle.
  • the object of the present invention is to solve the above problems in the prior art, and to provide a method and apparatus for ensuring video quality of a user's viewing angle and reducing transmission bandwidth.
  • the invention discloses a method for projecting a virtual reality (VR) panoramic video stream, which is characterized in that a panoramic video stream is divided into a plurality of spherical sub-regions in a spherical surface with a user as a center of the sphere, and the viewing angle information of the user is continuously detected.
  • VR virtual reality
  • the first video quality is higher than the second video quality in at least one of resolution and frame rate.
  • the plurality of spherical sub-regions includes 18 spherical sub-regions.
  • the 18 spherical sub-regions include eight spherical sub-regions located in the spherical equatorial region, four spherical sub-regions in the north-south latitude 45-degree region, and one spherical sub-region in each of the two poles.
  • projecting the primary and non-primary viewing regions includes performing one of a cube projection, an isometric cube projection, an equidistant projection, and an equilateral projection.
  • the predetermined offset is one-half of the spherical radius.
  • the panoramic video stream is received over a wired or wireless network.
  • the invention also discloses an apparatus for projecting a virtual reality (VR) panoramic video stream, characterized by comprising a sensor, a display, a memory and a processor, wherein the memory stores instructions executable by the processor, and the transceiver is used for wired or wireless
  • the network receives the virtual reality panoramic video stream, and the processor is configured to perform the following actions when the instruction is executed: dividing the panoramic video stream into a plurality of spherical sub-regions in a sphere centered on the user, and reading is continuously detected by the sensor
  • the viewing angle information of the user determines at least one spherical sub-region corresponding to the user perspective information in the plurality of spherical sub-regions as the main viewing region, and defines other spherical sub-regions other than the main viewing region as the non-primary viewing region, and determines the ball
  • the heart is at a predetermined offset of the user's point of view, and the indicator display is based on the user's point of view, project
  • the first video quality is higher than the second video quality in at least one of resolution and frame rate.
  • the plurality of spherical sub-regions includes 18 spherical sub-regions.
  • the 18 spherical sub-regions include eight spherical sub-regions located in the spherical equatorial region, four spherical sub-regions in the north-south latitude 45-degree region, and one spherical sub-region in each of the two poles.
  • the display of the primary view region and the non-primary view region includes one of cube projection, isometric cube projection, isometric projection, and equilateral projection.
  • the predetermined offset is one-half of the spherical radius.
  • the embodiment of the present invention provides an optimized video reconstruction effect by means of dynamic cut flow, and can ensure the video quality of the user mainly playing the viewing angle in various application scenarios such as VR live broadcast, VR on-demand, streaming server, and APP play end. Reduce the network resources needed to transfer video.
  • FIG. 1 is a block diagram of an apparatus for projecting a virtual reality panoramic video stream, in accordance with some embodiments of the present invention.
  • FIG. 2 is a flow chart of a method of projecting a virtual reality panoramic video stream, in accordance with some embodiments of the present invention.
  • FIG. 3 is a schematic diagram of spherical sub-area partitioning of a virtual reality panoramic video, in accordance with some embodiments of the present invention.
  • FIG. 4 is a schematic diagram of user watchpoint selection in accordance with some embodiments of the present invention.
  • first, second, etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish various elements from each other. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element without departing from the scope of the invention. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
  • an apparatus 100 for projecting a virtual reality panoramic video stream in accordance with some embodiments includes a processor 101, a sensor 103, a memory 105, a display 107, a transceiver 109, and an optional audio unit 111 and user interaction unit 113.
  • Processor 101 may be any general purpose or special purpose processing device for executing instructions, such as a CISC or RISC instruction set processor, an x86 instruction set processor, a multi-core processor, a microcontroller, a controller, a logic control unit, or any other micro Processor or central processing unit (CPU).
  • CISC or RISC instruction set processor such as a CISC or RISC instruction set processor, an x86 instruction set processor, a multi-core processor, a microcontroller, a controller, a logic control unit, or any other micro Processor or central processing unit (CPU).
  • CPU central processing unit
  • the sensor 103 is for detecting the posture of the user who is watching the virtual reality panoramic video, and continuously transmits the detected user gesture to the processor 101 to determine the viewing angle information of the user.
  • processor 101 and sensor 103 can continue to communicate in real time or near real time and determine the user's perspective information to reduce the sense of delay and improve the user experience.
  • the sensor 103 may include an eyeball tracking sensor, a head posture sensor, a multi-axis attitude sensor, a somatosensory gamepad, and the like. Methods for detecting eyeball position and head posture have been disclosed in the prior art such as CN102946791B and CN102156537B.
  • the sensor 103 can track the user's eye or face direction based on similar techniques to determine changes in the angle of view.
  • the memory 105 is used to store machine executable instructions that are executable by the processor 101 to implement the virtual reality panoramic video playback described in the embodiments, and may also store the received virtual reality panoramic video data for buffering or local playback in some cases.
  • Memory 105 includes volatile memory such as, but not limited to, random access memory (RAM), dynamic DRAM, static RAM.
  • RAM random access memory
  • DRAM dynamic DRAM
  • static RAM static RAM
  • the memory 105 also includes non-volatile memory such as, but not limited to, a CDROM disc, a compact disc, a DVD, a Blu-ray disc, a floppy disk, a magnetic disk, a solid state disk, a read only memory (ROM), an EPROM, an EEPROM, a flash memory, and/or a network storage device.
  • the memory 105 can also be provided through a remote memory or a cloud or the like.
  • the memory 105 can make a dedicated optimization in response time, read/write speed, and the like according to the application scenario of the virtual reality.
  • Display 107 includes a corresponding dedicated graphics processing unit (GPU) for displaying virtual reality images and/or interactive objects to a user.
  • the GPU can communicate with display 107 via an analog or digital interface.
  • the display 107 includes, for example, various existing imaging means such as a television, a flat panel display, a liquid crystal display, a head mounted display, a projection screen, a media player, and the like.
  • display 107 can be combined with audio unit 111.
  • the display 107 will include a display interface compatible with the virtual reality panoramic video stream, including but not limited to a high fidelity multimedia interface HDMI, wireless HDMI, MHL, VGA interface, DVI interface, mini display port MDP, and the like.
  • Display 107 also includes a corresponding codec to encode and decode the virtual reality panoramic video to be played.
  • the transceiver 109 can access a wireless or wired network to provide connectivity to receive a panoramic video stream to be played or vice versa.
  • the transceiver 109 can also be used to receive control commands and communicate with the processor 101 for remote power up, power off, play, fast forward or stop operations.
  • the transceiver 109 may include a wired network card, a modem, an optical modem, etc., in connection with a wired network to connect to various local area networks, metropolitan area networks, Ethernet, or the Internet.
  • the transceiver 109 may include an antenna, a wireless network card, a transmitter and a receiver, etc.
  • CDMA Code division multiple access
  • the audio unit 111 is provided in the case where it is necessary to provide audio information to the user, and may include a speaker, a microphone, and the like.
  • the user interaction unit 113 can be used to provide a user with a means for interacting with the virtual reality panoramic video, and can include existing devices such as a touchpad, a keyboard, a mouse, and a game controller. It can also be realized by detecting the movement of the user's hand or body by an additional posture sensor.
  • processor 101, sensor 103, memory 105, display 107, transceiver 109, and optional audio unit 111 and user interaction unit 113 described above may be integrated together to form a system on a chip (SOC).
  • SOC system on a chip
  • FIG. 2 illustrates a method flow performed in device 100 in accordance with some embodiments.
  • the virtual reality panoramic video is first spatially divided, that is, cut. From the user's point of view, the image of the virtual reality panoramic video is equivalent to forming a spherical surface centered on the user, called the world ball. Users are free to choose to watch videos from anywhere on the world's sphere. According to the eyeball structure and imaging features of the human body, the imaging of the central concave field of the user's eyeball is relatively clear, while the imaging of the edge region is relatively blurred.
  • the spherical area corresponding to the clear area of the user's eyeball imaging should be relatively clearly imaged, and the image that is blurred or even unobservable is not required to be so clearly imaged. Accordingly, the spherical surface of the world ball can be divided into a plurality of spherical sub-regions for differential projection of different spherical sub-regions in subsequent steps.
  • the sensor 103 continuously detects the viewing angle information of the user using the various attitude detection methods described above.
  • the viewing angle information of the user corresponds to a spherical area corresponding to the direction viewed by the user.
  • the detected view information is continuously transmitted by the sensor 103 to the processor 101 for processing such that the processor 101 can determine the user's perspective in real time or near real time.
  • step 205 the processor 101 determines at least one spherical sub-region corresponding to the detected user perspective information among the plurality of spherical sub-regions determined according to the user perspective information, as the main viewing region, that is, the user needs a higher quality virtual
  • the area of the realistic panoramic video image For the user, the main view area will appear directly in front of its field of view. The determination of the primary view area will change as the user's perspective changes.
  • step 207 other spherical sub-regions other than the main viewing angle region at this time are defined by the processor 101 as non-primary viewing angle regions, that is, regions in which lower quality virtual reality video images do not affect the user experience.
  • non-primary viewing angle regions regions in which lower quality virtual reality video images do not affect the user experience.
  • only one spherical sub-region is determined to be the primary viewing region, at which point all other spherical sub-regions are non-primary viewing regions.
  • the processor 101 further optimizes the projection of the virtual reality panoramic video to define a new user observation point at a predetermined offset from the center of the world ball, i.e., the point at which the original user is located.
  • a predetermined offset from the center of the world ball i.e., the point at which the original user is located.
  • projecting based on the user's viewpoint rather than the original user's location will make the user view the object in the front of the main viewing area closer to himself, clearer, higher resolution, and farther away from himself.
  • the objects on the two sides and the rear non-main view area are more and more blurred and the resolution is lower.
  • This adjustment method is called eccentric projection.
  • the use of eccentric projection will reduce the resource consumption of the video stream in the non-main view area while enhancing the video quality of the user's main view area.
  • the processor 101 further instructs the display 107 to project an image observed by the user's perspective onto the projection plane of the corresponding mode in a suitable projection mode, thereby obtaining a planar projection image.
  • the display 107 projects the projected image with the first video quality on the primary view area and the non-primary view area with a second video quality different from the first video quality.
  • the first video quality is higher than the second video quality in at least one of resolution and frame rate. For example, more pixels in the user's main view area are distributed with the first video quality, with higher resolution (such as 4K) or frame rate (such as 90Hz).
  • Suitable projection modes at this time include, for example, but are not limited to, cube projection, isometric cube projection, isometric projection, and equilateral projection.
  • the isometric cube projection scheme can achieve a more uniform resolution distribution in the main viewing angle than the cube projection or the like, thereby ensuring stable quality of the region within the main viewing angle and further saving bandwidth.
  • the device or method of the embodiment of the present invention dynamically transmits the video stream of the corresponding spherical sub-area to the user. This not only ensures that the user can watch high-resolution video all the time, but also keeps the bandwidth required for transmission low.
  • FIG. 3 shows an example of pipe cutting a virtual reality panoramic video stream.
  • the original virtual reality panoramic video stream is divided into 18 perspectives, and the video is reconstructed by 18 directions in the vicinity of the user to obtain the optimal video reproduction effect.
  • These 18 perspectives divide the spherical surface of the world sphere into 8 spherical sub-regions located in the spherical equatorial region, 8 sub-regions in each of the 4 spherical sub-regions in the 45-degree north and south latitude, and 1 spherical sub-region in each of the two poles. region.
  • This division method takes into account the bandwidth saving performance, the quality of the video in the main viewing area, and the complexity of the algorithm.
  • the division of the spherical sub-region may be four sub-regions of the equatorial region, two sub-regions of the 45-degree region in the north-south latitude, and one spherical sub-region of each of the two poles.
  • the spherical sub-region can be further subdivided when resources permit, such that 16 sub-regions divided into equatorial regions, 8 sub-regions in the 45-degree region of the north-south latitude, and 2 spherical sub-regions in each of the two poles.
  • resources permit such that 16 sub-regions divided into equatorial regions, 8 sub-regions in the 45-degree region of the north-south latitude, and 2 spherical sub-regions in each of the two poles.
  • the offset amount can also be adjusted accordingly to allow the user to view a projected image suitable for the subject of the video presentation.
  • the adjustment of the offset can also be used, for example, to adjust the magnification, especially when viewing a higher resolution video with a lower resolution device, the offset adjustment can be utilized to optimize the playback effect. For example, when viewing a 4K, 6K or 8K video at the display 107 of the same resolution 1080P, the magnitude of the offset can be adjusted accordingly.
  • the predetermined offset may simply be chosen to be half the radius of the world sphere, ie half the sphere to the spherical distance. However, the selection of the offset is not limited to this value, but can be freely adjusted or even changed as described above to suit the specific situation of the user and the video.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Computer Interaction (AREA)
  • Geometry (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本发明的实施例关于一种对用户投影虚拟现实全景视频流的方法，包括将全景视频流划分在多个球面子区域中，根据实时跟踪的用户视角信息对与用户视角相关联的球面子区域以及不相关联的球面子区域提供不同的视频质量，并移动用户观察点，从而实现最优视频重现效果并减少传输带宽。本发明还关于一种对用户投影虚拟现实全景视频流的设备。

Description

虚拟现实全景视频流投影方法和设备 技术领域
本发明关于虚拟现实技术领域，更具体地关于一种虚拟现实全景视频流投影方法和设备。
背景技术
虚拟现实(Virtual Reality,VR)是一种目前已在视频，摄影，院线和游戏场景中得到应用的技术，其包括多媒体、人机交互、传感器、网络技术等多种技术的交叉结合。虚拟现实可以基于用户的视觉，听觉甚至触觉创造一个可主观感知且可自由观察的虚拟世界，为用户带来高度的沉浸感和参与感，是未来多媒体和在线娱乐领域的重要发展方向。虚拟现实技术一般包括相关的硬件和软件两部分。虚拟现实硬件例如包括人体跟踪器和传感器、用户输入设备、3D显示器、投影系统、头戴显示器、立体生系统、动作捕捉设备、眼球跟踪设备和其他交互设备。虚拟现实软件包括虚拟现实视频和游戏中所需要的显示驱动、数据传输和编解码算法等。
随着以光纤入户和4G网络为代表的网络接入环境的改进，目前已经大量出现利用全景相机或摄像机制作的虚拟现实全景图片或视频通过网络与例如虚拟现实显示器、投影仪、手机或游戏主机等虚拟现实设备建立数据通信，使得用户可以实时通过网络体验虚拟现实视频的应用。由于虚拟现实视频必须包括围绕用户的球面范围内的所有角度的视觉信息以便用户可以观看任意角度，这就要求实时流传播占用大量带宽和其他网络资源高清视频数据。在当前的VR视频点播与VR视频直播中虚拟现实视频普遍分辨率高，码率高的情况下，一般用户的网络条件很难满足流播虚拟现实视频的需要。而另一个方面， 用户在任一时刻的观察角度必然是有限的，不可能观看球面内的所有影像。只有在用户转头去观看其他角度的时候，才会改变所看到的影像，而其余范围内的影像等于并未起到作用还浪费了网络资源。因此，需要在保证用户观察角度内视频质量的同时尽量节约网络资源。现有技术中可能采用限制用户视角，利用突发的刺激性影像或声音吸引用户视角到影像球面特点区域的方法来减少传输带宽，但仍然缺少一种在保证主要播放视角(FOV)中虚拟现实视频质量的同时，减少传输带宽的可靠方法和设备。
发明内容
本发明的目的是解决现有技术中的上述问题，提供可保证用户视角视频质量并减少传输带宽的方法和设备。
本发明公开了一种投影虚拟现实(VR)全景视频流的方法，其特征在于在以用户为球心的球面中将全景视频流划分在多个球面子区域中，持续检测用户的视角信息，将多个球面子区域中与用户视角信息对应的至少一个球面子区域确定为主视角区域，将主视角区域以外的其他球面子区域定义为非主视角区域，确定与球心距离一预定偏移量的用户观察点，以及基于用户观察点，以第一视频质量对主视角区域进行投影并以第二视频质量对非主视角区域进行投影。
在一些实施例中，第一视频质量在分辨率和帧率之中的至少一方面高于第二视频质量。
在一些实施例中，多个球面子区域包括18个球面子区域。
在一些实施例中，18个球面子区域包括位于球面赤道区域的8个球面子区域，南北纬45度区域的各4个球面子区域和两极的各1个球面子区域。
在一些实施例中，对主视角区域和非主视角区域进行投影包括进行立方体投影，等角立方体投影，等距投影和等边投影之一。
在一些实施例中，预定偏移量为球面半径的一半。
在一些实施例中，全景视频流通过有线或无线网络进行接收。
本发明还公开了一种投影虚拟现实(VR)全景视频流的设备，其特征在于包括传感器、显示器、存储器和处理器，其中存储器存储有可由处理器运行的指令，收发器用于通过有线或无线网络接收所述虚拟现实全景视频流，处理器配置成在运行指令时执行以下动作：在以用户为球心的球面中将全景视频流划分在多个球面子区域中，读取由传感器持续检测的用户的视角信息，将多个球面子区域中与用户视角信息对应的至少一个球面子区域确定为主视角区域，将主视角区域以外的其他球面子区域定义为非主视角区域，确定与球心距离一预定偏移量的用户观察点，以及指示显示器基于用户观察点，以第一视频质量对主视角区域进行投影并以第二视频质量对非主视角区域进行投影。
在一些实施例中，第一视频质量在分辨率和帧率之中的至少一方面高于第二视频质量。
在一些实施例中，多个球面子区域包括18个球面子区域。
在一些实施例中，18个球面子区域包括位于球面赤道区域的8个球面子区域，南北纬45度区域的各4个球面子区域和两极的各1个球面子区域。
在一些实施例中，显示器对主视角区域和非主视角区域进行投影包括进行立方体投影，等角立方体投影，等距投影和等边投影之一。
在一些实施例中，预定偏移量为球面半径的一半。
本发明的实施例采用动态切流的手段提供优化的视频重构效果，可以在VR直播、VR点播、流服务器、APP播放端等各类应用场景中保证用户主要播放视角内视频质量的同时大幅减少传输视频所需的网络资源。
附图说明
本发明提供了附图以便于进一步理解所公开的内容，附图构成本申请的一部分，但仅仅是用于图示出体现本发明概念的一些非限制性示例，而不是用于做出任何限制。
图1是根据本发明一些实施例的投影虚拟现实全景视频流的设备的框图。
图2是根据本发明一些实施例的投影虚拟现实全景视频流的方法的流程图。
图3是根据本发明一些实施例的对虚拟现实全景视频进行球面子区域划分的示意图。
图4是根据本发明一些实施例的对用户观察点选取的示意图。
具体实施方式
下文将使用本领域的技术人员向本领域的其它技术人员传达他们工作的实质所通常使用的术语来描述本文说明性实施例的各个方面。然而，对于本领域的技术人员而言将显而易见的是，可以仅使用所描述的多个方面中的一些来实践备选实施例。出于解释的目的，本文阐述了特定的数值、材料和配置，以便使说明性的实施例更容易被理解。然而，对于本领域的技术人员而言将显而易见的是，在省略了特定细节的情况下也可以实践本文的备选实施例。在其它情况下，可以省略或简化众所周知的特征，以便不使本文的实施例难于理解。
本领域技术人员将理解尽管术语第一、第二等可在本文中用于描述各种元素，但这些元素不应由这些术语限制。这些术语仅用于将各种元素彼此区分开。例如，第一元素可以称作第二元素，并且相似地，第二元素可以称作第一元素，而不偏离本发明的范围。如本文使用的，术语“和/或”包括关联的列出项目中的一个或多个中的任一个或全部组合。本文使用的术语仅是为了描述特定实施例目的并且不意在限 制本发明。如本文使用的，单数形式“一”和“该”意在也包括复数形式，除非上下文另外明确指示其他意思。本领域技术人员将进一步理解术语“包括”和/或“包含”当在本文使用时，规定了陈述的特征、整体、步骤、操作、元素和/或部件的存在，但不排除一个或多个其它特征、整体、步骤、操作、元素、部件和/或其的组合的存在或增加。
如图1所示，根据一些实施例的投影虚拟现实全景视频流的设备100包括处理器101、传感器103、存储器105、显示器107、收发器109以及可选的音频单元111和用户交互单元113。
处理器101可以是用来执行指令的任何通用或者专用的处理设备，例如CISC或RISC指令集处理器、x86指令集处理器、多核处理器、单片机、控制器、逻辑控制单元或任何其他的微处理器或中央处理单元(CPU)。
传感器103用于检测在观看虚拟现实全景视频的用户的姿态，并持续将检测到的用户姿态传递到处理器101以判断用户的视角信息。在优选的实施例中，处理器101和传感器103可以实时或者接近实时地持续通讯，并确定用户的视角信息以降低延迟感，改善用户体验。传感器103可以包括眼球追踪传感器、头部姿态传感器、多轴姿态传感器、体感游戏手柄等。例如CN102946791B和CN102156537B等现有技术中已经公开了对眼球位置以及头部姿态的检测方法。传感器103可以基于类似技术跟踪用户的眼球或者面部方向，从而确定视角的变化。
存储器105用来存储可供处理器101执行以实现实施例中所述虚拟现实全景视频播放的机器可执行指令，也可在一些情况下存储所接收的虚拟现实全景视频数据以便进行缓冲或者本地播放。存储器105包括易失性存储器，例如但不限于随机存取存储器(RAM)、动态 DRAM、静态RAM。存储器105还包括非易失性存储器，例如但不限于CDROM光盘、压缩盘、DVD、蓝光光盘、软盘、磁盘、固态盘、只读存储器(ROM)、EPROM、EEPROM、闪存和/或网络存储设备。在VR直播、VR点播、流播、APP播放等情况下，存储器105还可以通过远程存储器或云端等来提供。存储器105可以根据虚拟现实的应用场景而在响应时间、读写速度等方面做出专用的优化。
显示器107包括相应的专用图形处理器(GPU)，用于向用户显示虚拟现实图像和/或交互对象。该GPU可以通过模拟或者数字接口与显示器107进行通信。显示器107例如包括电视、平板显示器、液晶显示器、头戴显示器、投影屏幕、媒体播放器等各种现有的成像手段。在一些实施例中，显示器107可以与音频单元111相结合。显示器107将包括与虚拟现实全景视频流兼容的显示接口，该显示接口包括但不限于高保真多媒体界面HDMI、无线HDMI、MHL、VGA接口、DVI接口、迷你显示端口MDP等。显示器107还包括相应的编解码器以对所要播放的虚拟现实全景视频进行编解码。
收发器109可以接入无线或者有线网络以提供连接性从而接收要播放的全景视频流或者相反地进行上传。收发器109也可以用来接收控制指令并与处理器101进行通信以进行远程开机、关机、播放、快进或停止等操作。在接入有线网络的情况下收发器109可以包括有线网卡、调制解调器、光调制解调器等，以连接到各种局域网、城域网、以太网或互联网。在接入无线网络的情况下，收发器109可以包括天线、无线网卡、发射器和接收器等以根据4GLTE长期演进、Wi-Fi、蓝牙、无线局域网WLAN、全球移动通信GSM、码分多址CDMA、WCDMA、时分复用TDM等与服务器、基站、演进节点和/或其他收发器进行通信。
音频单元111在需要向用户提供音频信息的情况下提供，可包括 扬声器、麦克风等。
用户交互单元113可用于向用户提供与虚拟现实全景视频交互的手段，可以包括触控板、键盘、鼠标、游戏控制器等现有设备。也可以通过额外的姿态传感器对用户手部或身体的动作进行探测来实现。
在一些实施例中，上述处理器101、传感器103、存储器105、显示器107、收发器109以及可选的音频单元111和用户交互单元113可以集成到一起以形成片上系统(SOC)。
图2示出了根据一些实施例在设备100中执行的方法流程。在步骤201中，首先对虚拟现实全景视频进行空间上的划分，即切流。在用户看来，虚拟现实全景视频的图像相当于以用户为中心形成了一个球面，称为世界球。用户可以自由选择观察世界球球面上任一处的视频。根据人体的眼球构造和成像特征，用户眼球的中央凹视野处成像较为清晰，而边缘区域成像较为模糊。因此，对于用户眼球成像清晰区域对应的球面区域应相应进行较清晰的成像，而对成像模糊甚至无法观察到的区域则不需进行如此清晰的成像。据此，可将世界球的球面划分为多个球面子区域，以便后续步骤中对于不同的球面子区域进行区别投影。
在步骤203中，传感器103利用上述的各种姿态检测方式持续对用户的视角信息进行检测。用户的视角信息对应用户所观看的方向对应的球面区域。所探测到的视角信息被传感器103持续传送到处理器101进行处理使得处理器101可以实时或接近实时地确定用户的视角。
在步骤205中，处理器101将根据用户视角信息所确定的多个球面子区域中与所探测到的用户视角信息对应的至少一个球面子区域确定为主视角区域，即用户需要更高质量虚拟现实全景视频图像的区 域。对于用户，主视角区域将出现在其视野的正前方。主视角区域的确定将随着用户视角的变化而不断变化。
在步骤207中，此时的主视角区域以外的其他球面子区域被处理器101定义为非主视角区域，即较低质量的虚拟现实视频图像不会对用户体验造成影响的区域。在备选实施例中，仅有一个球面子区域被确定为主视角区域，此时所有的其他球面子区域均为非主视角区域。
在步骤209中处理器101进一步对虚拟现实全景视频的投影进行优化，在距离世界球的球心即原始的用户所在点一定的预定偏移量的位置定义新的用户观察点。对于当前的用户视角，以该用户观察点而非原始用户所在地为基准进行投影将使得用户观看离自己较近的正前方中主视角区域的物体更加清晰，分辨率更高，而离自己较远的两侧及后方的非主视角区域的物体越来越模糊，分辨率也越低，这种调整方法被称为偏心投影。使用偏心投影将在增强用户主视角区域视频质量的同时，降低非主视角区域视频流的资源消耗。
在步骤211中，基于所述用户观察点，处理器101进一步指示显示器107以合适的投影模式，将用户视角观察到的图像投影到相应模式的投影平面上，从而得到一幅平面的投影图像。显示器107将该投影图像以第一视频质量对主视角区域进行投影并以不同于第一视频质量的第二视频质量对非主视角区域进行投影。优选地，第一视频质量在分辨率和帧率之中的至少一方面高于第二视频质量。例如，用户主视角区域内以第一视频质量分布更多的像素，具有更高的分辨率(如4K)或帧率(如90Hz)。在非用户主视角区域内分布较少的像素，获得较低的分辨率(如1080P)或帧率(如60Hz)。由于非用户主视角的像素数量或帧率大大降低，视频流的整体大小也随之降低，传输视频流需要的带宽也大大节约。此时合适的投影模式例如包括但不限于立方体投影，等角立方体投影，等距投影和等边投影中之 一。优选地，相比立方体投影等方式，等角立方体投影方案可以使得主视角内得到更均匀的分辨率分布，从而可以保证主视角内区域的质量稳定，并进一步节约带宽。
用户在观察虚拟现虚拟实全景视频时很可能会不断地变化主视角。在检测用户视角信息变化的情况下，本发明实施例的设备或方法将动态地传输相应球面子区域的视频流给用户。这样既保证了用户可以一直观看到高分辨率的视频，同时可以将传输需要的带宽保持为较低。
图3示出了对虚拟现实全景视频流进行切流的示例。其中将原始的虚拟现实全景视频流切分为了18个视角，在对用户进行播放时通过18个方向就近重构视频，得到最优视频重现效果。这18个视角将世界球的球面分为位于球面赤道区域的8个球面子区域，南北纬45度区域的各4个球面子区域共8个子区域以及两极的各1个球面子区域共2个子区域。这种划分方法兼顾了带宽节省的性能，主视角区域内视频的质量，以及算法的复杂度。但应注意，球面子区域的划分的此种方法仅为示例，并不限于划分为18个视角。例如，球面子区域的划分可以为赤道区域的4个子区域，南北纬45度区域的各2个子区域和两极的各1个球面子区域。又例如，球面子区域在资源允许的情况下可以进一步细分，使得分为赤道区域的16个子区域，南北纬45度区域的各8个子区域和两极的各2个球面子区域。本领域技术人员将能够从本发明的公开内容中容易地想到其他划分方式。
图4示出了根据一些实施例的偏心投影预定偏移量的选择和用户观察点的确定。如图所示，当用户的位置从世界球坐标系的球心移动到具有一偏移量的用户观察点时，视角也相应从世界场视角变为用户场视角。从而相比世界场视角进一步增强了用户主视角区域的视频质量，并降低非主视角区域视频流质量以节约传输带宽。例如对于表 现不同内容的虚拟现实场景(如表现宏观风景的远景视频或者表现细微细节的特写视频)，也可相应调整偏移量的大小使用户观赏到适合该视频表现主题的投影图像。偏移量的调整例如还可以用于调整放大倍数，尤其是在以较低分辨率的设备观看较高分辨率的视频时可以利用偏移量的调整来优化播放效果。例如在以同一分辨率1080P的显示器107观看4K、6K或者8K的视频时，可以相应地调整偏移量的大小。在一些实施例中，可以简单地将预定偏移量选为世界球半径的一半，即球心到球面距离的一半。但偏移量的选择不限于该取值，而是可以如上述进行自由调节甚至不断改变以便适合用户和视频的具体情况。
本领域技术人员在查看所示附图和描述时将明白依据于本发明概念和原则的实施例的其它各种虚拟现实视频投影装置和/或方法。所有此类另装置和/或方法都包括在本发明的公开范围内，并且在本发明概念和原则的范围内。另外，本文中公开的所有实施例能够单独实现或者以任何方式和/或在任何组合中组合。

Claims (13)

  1. 一种投影虚拟现实(VR)全景视频流的方法，其特征在于：
    在以用户为球心的球面中将所述全景视频流划分在多个球面子区域中；
    持续检测所述用户的视角信息；
    将所述多个球面子区域中与所述用户视角信息对应的至少一个球面子区域确定为主视角区域；
    将所述主视角区域以外的其他球面子区域定义为非主视角区域；
    确定与所述球心距离一预定偏移量的用户观察点；
    以及
    基于所述用户观察点，以第一视频质量对所述主视角区域进行投影并以第二视频质量对所述非主视角区域进行投影。
  2. 如权利要求1所述的方法，其特征在于所述第一视频质量在分辨率和帧率之中的至少一方面高于所述第二视频质量。
  3. 如权利要求2所述的方法，其特征在于所述多个球面子区域包括18个球面子区域。
  4. 如权利要求3所述的方法，其特征在于所述18个球面子区域包括位于所述球面赤道区域的8个球面子区域，南北纬45度区域的各4个球面子区域和两极的各1个球面子区域。
  5. 如权利要求4所述的方法，其特征在于对所述主视角区域和所述非主视角区域进行投影包括进行立方体投影，等角立方体投影，等距投影和等边投影之一。
  6. 如权利要求1所述的方法，其特征在于所述预定偏移量为所述球面半径的一半。
  7. 如权利要求1所述的方法，其特征在于所述全景视频流通过 有线或无线网络进行接收。
  8. 一种投影虚拟现实(VR)全景视频流的设备，其特征在于包括传感器、显示器、存储器、收发器和处理器，其中所述存储器存储有可由所述处理器运行的指令，所述收发器用于通过有线或无线网络接收所述虚拟现实全景视频流，所述处理器配置成在运行所述指令时执行以下动作：
    在以用户为球心的球面中将所述全景视频流划分在多个球面子区域中；
    读取由所述传感器持续检测的所述用户的视角信息；
    将所述多个球面子区域中与所述用户视角信息对应的至少一个球面子区域确定为主视角区域；
    将所述主视角区域以外的其他球面子区域定义为非主视角区域；
    确定与所述球心距离一预定偏移量的用户观察点；
    以及
    指示所述显示器基于所述用户观察点，以第一视频质量对所述主视角区域进行投影并以第二视频质量对所述非主视角区域进行投影。
  9. 如权利要求8所述的设备，其特征在于所述第一视频质量在分辨率和帧率之中的至少一方面高于所述第二视频质量。
  10. 如权利要求9所述的设备，其特征在于其特征在于所述多个球面子区域包括18个球面子区域。
  11. 如权利要求10所述的设备，其特征在于所述18个球面子区域包括位于所述球面赤道区域的8个球面子区域，南北纬45度区域的各4个球面子区域和两极的各1个球面子区域。
  12. 如权利要求11所述的设备，其特征在于所述显示器对所述主视角区域和所述非主视角区域进行投影包括进行立方体投影，等角立方体投影，等距投影和等边投影之一。
  13. 如权利要求8所述的设备，其特征在于所述预定偏移量为所述球面半径的一半。
PCT/CN2017/118537 2017-08-23 2017-12-26 虚拟现实全景视频流投影方法和设备 WO2019037365A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/640,796 US20210368148A1 (en) 2017-08-23 2017-12-26 Virtual reality panoramic video stream projection method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710728421.XA CN107396077B (zh) 2017-08-23 2017-08-23 虚拟现实全景视频流投影方法和设备
CN201710728421.X 2017-08-23

Publications (1)

Publication Number Publication Date
WO2019037365A1 true WO2019037365A1 (zh) 2019-02-28

Family

ID=60354206

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/118537 WO2019037365A1 (zh) 2017-08-23 2017-12-26 虚拟现实全景视频流投影方法和设备

Country Status (3)

Country Link
US (1) US20210368148A1 (zh)
CN (1) CN107396077B (zh)
WO (1) WO2019037365A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112437286A (zh) * 2020-11-23 2021-03-02 成都易瞳科技有限公司 一种全景原始画面视频分块传输方法
CN113242384A (zh) * 2021-05-08 2021-08-10 聚好看科技股份有限公司 一种全景视频显示方法及显示设备

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107396077B (zh) * 2017-08-23 2022-04-08 深圳看到科技有限公司 虚拟现实全景视频流投影方法和设备
US10217488B1 (en) * 2017-12-15 2019-02-26 Snap Inc. Spherical video editing
CN110022473A (zh) * 2018-01-08 2019-07-16 中国科学院计算技术研究所 全景视频图像的显示方法
CN108322727A (zh) * 2018-02-28 2018-07-24 北京搜狐新媒体信息技术有限公司 一种全景视频传输方法及装置
CN108401184A (zh) * 2018-03-06 2018-08-14 深圳市赛亿科技开发有限公司 Vr全景画面的显示方法、vr显示终端
WO2019199637A1 (en) * 2018-04-11 2019-10-17 Alcacruz Inc. Digital media system
EP3588970A1 (en) * 2018-06-22 2020-01-01 Koninklijke Philips N.V. Apparatus and method for generating an image data stream
CN110798707B (zh) 2018-08-02 2023-06-16 华为技术有限公司 传输媒体数据的方法、客户端和服务器
CN109040601B (zh) * 2018-09-05 2020-06-26 清华-伯克利深圳学院筹备办公室 一种多尺度非结构化的十亿像素vr全景摄影系统
CN110956583B (zh) * 2018-09-26 2022-05-10 华为技术有限公司 球面图像处理方法、装置及服务器
CN109587571A (zh) * 2018-11-02 2019-04-05 北京洛必达科技有限公司 适用于虚拟现实技术的音视频控制方法和装置
CN109936697B (zh) * 2019-04-29 2020-12-22 嘉兴奥恒进出口有限公司 一种视频拍摄目标跟踪方法和装置
CN112312227B (zh) * 2020-06-15 2022-09-16 青岛一舍科技有限公司 一种全景交互方法和装置
CN111954085A (zh) * 2020-08-06 2020-11-17 咪咕文化科技有限公司 Vr视频显示方法、装置、网络设备及存储介质
CN112040209B (zh) * 2020-09-14 2021-09-03 龙马智芯（珠海横琴）科技有限公司 一种vr场景的投影方法、装置、投影系统以及服务器
CN112218110B (zh) * 2020-09-28 2022-10-28 京东方科技集团股份有限公司 Vr全景视频的传输方法、装置、服务器及交互系统
CN115529449A (zh) * 2021-06-26 2022-12-27 华为技术有限公司 虚拟现实视频传输方法及装置
CN113949893A (zh) * 2021-10-15 2022-01-18 中国联合网络通信集团有限公司 直播处理方法、装置、电子设备及可读存储介质
CN115103114B (zh) * 2022-06-16 2024-06-14 京东方科技集团股份有限公司 全景视频的视角跟踪方法、装置、设备和介质
CN117784915A (zh) * 2022-09-20 2024-03-29 北京字跳网络技术有限公司 虚拟环境中的观影画面调整方法、装置、存储介质及设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016191467A1 (en) * 2015-05-27 2016-12-01 Google Inc. Capture and render of panoramic virtual reality content
CN106412424A (zh) * 2016-09-20 2017-02-15 乐视控股（北京）有限公司 全景视频的视野调整方法及装置
CN106454321A (zh) * 2016-10-26 2017-02-22 深圳市道通智能航空技术有限公司 全景视频的处理方法、装置及系统
CN106919248A (zh) * 2015-12-26 2017-07-04 华为技术有限公司 应用于虚拟现实的内容传输方法以及设备
CN107396077A (zh) * 2017-08-23 2017-11-24 深圳看到科技有限公司 虚拟现实全景视频流投影方法和设备

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6833843B2 (en) * 2001-12-03 2004-12-21 Tempest Microsystems Panoramic imaging and display system with canonical magnifier

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016191467A1 (en) * 2015-05-27 2016-12-01 Google Inc. Capture and render of panoramic virtual reality content
CN106919248A (zh) * 2015-12-26 2017-07-04 华为技术有限公司 应用于虚拟现实的内容传输方法以及设备
CN106412424A (zh) * 2016-09-20 2017-02-15 乐视控股（北京）有限公司 全景视频的视野调整方法及装置
CN106454321A (zh) * 2016-10-26 2017-02-22 深圳市道通智能航空技术有限公司 全景视频的处理方法、装置及系统
CN107396077A (zh) * 2017-08-23 2017-11-24 深圳看到科技有限公司 虚拟现实全景视频流投影方法和设备

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112437286A (zh) * 2020-11-23 2021-03-02 成都易瞳科技有限公司 一种全景原始画面视频分块传输方法
CN112437286B (zh) * 2020-11-23 2024-04-26 成都易瞳科技有限公司 一种全景原始画面视频分块传输方法
CN113242384A (zh) * 2021-05-08 2021-08-10 聚好看科技股份有限公司 一种全景视频显示方法及显示设备

Also Published As

Publication number Publication date
US20210368148A1 (en) 2021-11-25
CN107396077B (zh) 2022-04-08
CN107396077A (zh) 2017-11-24

Similar Documents

Publication Publication Date Title
WO2019037365A1 (zh) 虚拟现实全景视频流投影方法和设备
US11508125B1 (en) Navigating a virtual environment of a media content item
CN111194561B (zh) 预测性的头部跟踪的双耳音频渲染
US10270825B2 (en) Prediction-based methods and systems for efficient distribution of virtual reality media content
CN107003512B (zh) 用于虚拟现实的头戴式显示器中的扫描显示系统
US10469820B2 (en) Streaming volumetric video for six degrees of freedom virtual reality
US10367919B2 (en) Methods and systems for efficient distribution of virtual reality media content
US9613461B2 (en) Display control apparatus, display control method, and program
US20170286993A1 (en) Methods and Systems for Inserting Promotional Content into an Immersive Virtual Reality World
US10681276B2 (en) Virtual reality video processing to compensate for movement of a camera during capture
JP2018139102A (ja) 没入型コンテンツ内の関心地点を決定する方法及び装置
JP2019534500A (ja) 没入型ビデオフォーマットのための方法、機器、及びシステム
US20220321858A1 (en) Methods, systems, and media for rendering immersive video content with foveated meshes
KR20190084987A (ko) 구형 이미지 콘텐츠를 위한 배향된 이미지 스티칭
KR20220049891A (ko) 바디 움직임 기반의 클라우드 vr 장치 및 방법
US11187895B2 (en) Content generation apparatus and method
WO2020206647A1 (zh) 跟随用户运动控制播放视频内容的方法和装置
WO2019034804A2 (en) THREE-DIMENSIONAL VIDEO PROCESSING
JP7447266B2 (ja) ボリュメトリック画像データに関するビューの符号化及び復号
CN203870559U (zh) 头戴式全景播放装置
EP3330839A1 (en) Method and device for adapting an immersive content to the field of view of a user
JP7339435B2 (ja) パーソナルデバイスによって支援されるｔｖストリーミング及びレンダリングコンテンツの中心窩最適化
KR102179810B1 (ko) 가상현실영상 재생방법 및 이를 이용한 프로그램
US20210354035A1 (en) Interaction in a multi-user environment
EP3564905A1 (en) Conversion of a volumetric object in a 3d scene into a simpler representation model

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17922720

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17922720

Country of ref document: EP

Kind code of ref document: A1