US20180191868A1 - Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization - Google Patents
Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization Download PDFInfo
- Publication number
- US20180191868A1 US20180191868A1 US15/860,430 US201815860430A US2018191868A1 US 20180191868 A1 US20180191868 A1 US 20180191868A1 US 201815860430 A US201815860430 A US 201815860430A US 2018191868 A1 US2018191868 A1 US 2018191868A1
- Authority
- US
- United States
- Prior art keywords
- video
- bitrate
- resolution
- resolutions
- video segments
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000005457 optimization Methods 0.000 title description 2
- 230000000875 corresponding effect Effects 0.000 claims description 33
- 238000013507 mapping Methods 0.000 claims description 30
- 230000002596 correlated effect Effects 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 11
- 230000003247 decreasing effect Effects 0.000 abstract description 3
- 230000004048 modification Effects 0.000 description 7
- 238000012986 modification Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- H04L67/38—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/398—Synchronisation thereof; Control thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/20—Perspective computation
- G06T15/205—Image-based rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0888—Throughput
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/752—Media network packet handling adapting media to network capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/131—Protocols for games, networked simulations or virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/139—Format conversion, e.g. of frame-rate or size
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/189—Recording image signals; Reproducing recorded image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/366—Image reproducers using viewer tracking
- H04N13/378—Image reproducers using viewer tracking for tracking rotational head movements around an axis perpendicular to the screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/366—Image reproducers using viewer tracking
- H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
- H04N21/2335—Processing of audio elementary streams involving reformatting operations of audio signals, e.g. by converting from one coding standard to another
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234345—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
- H04N21/2387—Stream processing in response to a playback request from an end-user, e.g. for trick-play
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
- H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2405—Monitoring of the internal components or processes of the server, e.g. server load
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3185—Geometric adjustment, e.g. keystone or convergence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/79—Processing of colour television signals in connection with recording
- H04N9/87—Regeneration of colour television signals
- H04N9/8715—Regeneration of colour television signals involving the mixing of the reproduced video signal with a non-recorded signal, e.g. a text signal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2215/00—Indexing scheme for image rendering
- G06T2215/16—Using real world measurements to influence rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
Definitions
- the present disclosure relates to video processing technology, and more particularly, to a method and a system for downloading a VR video.
- Virtual Reality is a computer simulation technology for creating and experiencing a virtual world. For example, a three-dimensional real-time image can be presented based on a technology which tracks a user's head, eyes or hands.
- full-view video data can be pre-stored on a server, and then transmitted to a display device.
- a display device can be glasses, a head-mounted display, etc.
- a video is displayed on the display device in accordance with a field of view of the user.
- a high-resolution video data occupies large transmission of bandwidth and requires high computing power from the display devices.
- Presenting high-resolution VR video on internet is difficult, precisely, the existing video streaming technology cannot fulfill the virtual reality. Therefore, in order to present VR video smoothly in real-time, it is desirable to further improve the existing video streaming technology to save bandwidth and reduce performance requirements for display devices.
- the present disclosure provides a method and a system for downloading a VR video for optimizing video transmission.
- a method for downloading a VR video comprising: receiving a play request for a first video; calculating an average download speed; selecting a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates; determining a download link according to the plurality of first video segments; and downloading the plurality of video segments according to the download link.
- the method further comprises: obtaining a mapping table between resolutions and bitrates during a video transcoding process; and determining the plurality of first video segments by looking up the mapping table.
- the step of obtaining the mapping table between resolutions and bitrates during the video transcoding process comprises: defining a plurality of resolution levels, and calculating a plurality of resolutions in the plurality of resolution levels according to a predetermined target resolution; and calculating bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
- the step of defining a plurality of resolution levels, and calculating a plurality of resolutions in the plurality of resolution levels according to a predetermined target resolution comprises: calculating the resolution in one of the plurality of resolution levels following formulas below,
- n represents the one of the plurality of resolution levels
- R0 represents a maximum value of the plurality of resolutions
- ⁇ X 0 enhance Y 0 enhance ⁇ represents the resolution of an enhancement video corresponding to the one of the plurality of resolution levels
- q represents a resolution ratio
- R n ⁇ X n , Y n ⁇ represents the resolution in the one of the plurality of resolution levels
- the step of calculating bitrates corresponding to the plurality of resolutions in the plurality of resolution levels based on a target bitrate of the predetermined target resolution comprises: calculating the bitrates corresponding to the resolutions in the plurality of resolution levels following formulas below,
- bitrate n bitrate n enhance and bitrate n base , there is:
- bitrate n + 1 bitrate n q 2 ,
- bitrate n base > 1 q 2 - 1 ⁇ bitrate n enhance ,
- bitrate 0 represents a maximum value of the bitrates
- bitrate n represents a maximum value of the bitrates
- bitrate n base represents bitrate n + bitrate n enhance respectively represent the bitrate, the bitrate of the basic video, and the bitrate of the enhancement video corresponding to the one of the plurality of resolution levels, wherein n is an integer equal to or larger than 1.
- the play request comprises a field of view of a user
- the plurality of first video segments have a plurality of resolutions
- a resolution distribution of the plurality of first video segments is determined by the field of view of the user.
- the resolution of a first video segment within the field of view is higher than the resolution of a first video segment outside the field of view.
- the average download speed and the resolution distribution of the plurality of first video segments are positively correlated.
- a system for downloading a VR video comprising: a request receiving module configured to receive a play request of a first video; a speed calculating module configured to calculate an average download speed; a segment obtaining module configured to select a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates; a link obtaining module configured to determine a download link according to the plurality of first video segments; and a download executing module configured to download the plurality of video segments according to the download link.
- the system further comprises: a mapping table obtaining module configured to obtaining a mapping table between resolutions and bitrates during a video transcoding process, and to determine the plurality of first video segments by looking up the mapping table.
- a mapping table obtaining module configured to obtaining a mapping table between resolutions and bitrates during a video transcoding process, and to determine the plurality of first video segments by looking up the mapping table.
- the mapping table obtaining module comprises: a first calculating unit configured to define a plurality of resolution levels and calculate a plurality of resolutions in the plurality of resolution levels according to the predetermined target resolution; and calculating bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
- the present disclosure provides a plurality of video segments with proper resolutions and proper bitrates for downloading according to the average download speed.
- a resolution reflects the image precision
- a bitrate reflects bit number of data transmitted per unit time
- the resolution, the bitrate and average download speed can be synthesized to optimize the video transmission by using an optimal strategy to select the appropriate video segments.
- the video segments with high resolutions and low resolutions are selected according to the user's field of view, which can decrease the amount of transferred data without affecting the user's demands for viewing the video.
- FIG. 1 is a diagram illustrating an example network of a VR playback system
- FIG. 2 is a flowchart showing a method used in the VR playback system of FIG. 1 ;
- FIG. 3 is a flowchart presenting the method for downloading a VR video according to an embodiment of the present disclosure
- FIG. 4 is a flowchart presenting a detailed methodology of obtaining the mapping table between resolutions and bitrates during the video transcoding process.
- FIG. 5 is a structural diagram presenting a system for downloading a VR video according to an embodiment of the present disclosure.
- FIG. 1 is a diagram illustrating an example network of a VR playback system.
- the VR playback system 10 includes a server 100 and a display device 120 which are coupled with each other through a network 110 , and a VR device.
- the server 100 may be a stand-alone computer server or a server cluster.
- the server 100 is used to store various video data and to store various applications that process these video data.
- various daemons run on the server 100 in real time, so as to process various video data in the server 100 and to respond various requests from VR devices and the display device 120 .
- the network 110 may be a selected one or selected ones from the group consisting of an internet, a local area network, an internet of things, and the like.
- the display device 120 may be any of the computing devices, including a computer device having an independent display screen and a processing capability.
- the display device 120 may be a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a palmtop computer, a personal digital assistant, a smart phone, an intelligent electrical apparatus, a game console, an iPad/iPhone, a video player, a DVD recorder/player, a television, or a home entertainment system.
- the display device 120 may store VR player software as a VR player. When the VR player is started, it requests and downloads various video data from the server 100 , and renders and plays the video data in the display device.
- the VR device 130 is a stand-alone head-mounted device that can interact with the display device 120 and the server 100 , to communicate the user's current information to the display device 120 and/or the server 100 through signaling.
- the user's current information is, for example, a field of view of the user, a position modification of a helmet, a modification of the user's sight orientation, etc.
- the display device 120 can flexibly process the currently played video data.
- the display device 120 determines that a core viewing region for the user has been changed and starts to play video data with high resolution in the changed core viewing region.
- the VR device 130 is a stand-alone head-mounted device.
- the VR device 130 is not limited thereto, and the VR device 130 may also be an all-in-one head-mounted device.
- the all-in-one head-mounted device itself has a display screen, so that it is not necessary to connect the all-in-one head-mounted device with the external display device.
- the display device 120 may be eliminated.
- the all-in-one head-mounted device is configured to obtain video data from the server 100 and to perform playback operation, and the all-in-one head-mounted device is also configured to detect user's current field of view modification information and to adjust the playback operation according to the field of view modification information.
- FIG. 2 is a flowchart showing a method used in the VR playback system of FIG. 1 . The method comprises following steps.
- step S 10 a source video data process is operated on server side.
- step S 20 the display device obtains some related information of the user based on interactions with the VR device.
- step S 30 the display device requests video data and receives video data from the server according to the related information.
- step S 40 the display device side performs a rendering process based on the received video data.
- the video data is downloaded. Furthermore, the present disclosure provides a method for downloading a VR video. The method downloads the VR video by predicting a network condition according to a calculated average download speed and providing a download link according to bitrates, in order to optimize VR video streaming transmission.
- FIG. 3 is a flowchart presenting the method for downloading a VR video according to an embodiment of the present disclosure. Specifically, following steps are comprised.
- step S 100 a play request for a first video is received.
- the first video in this step refers to a specific video.
- step S 200 an average download speed is calculated.
- the average download speed is calculated by making statistics on an amount of downloaded data and download time.
- the average download speed is calculated once in real time. For example, the average download speed is calculated once before data of the first video starts to be downloaded, or the average download speed is calculated once before each video segments of the first video starts to be downloaded.
- step S 300 a plurality of first video segments are selected from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates.
- a complete source VR video can be processed into the plurality of video segments, for example, the plurality of video segments may be obtained by segmenting the source VR video in temporal domain and/or in spatial domain.
- the plurality of video segments have different resolutions and bitrates.
- the plurality of first video segments can be selected from the plurality of video segments with different resolutions and bitrates. Therefore, in this step, each of the plurality of first video segments is the video segment with a proper resolution and a proper bitrate selected from the plurality of video segments, according to the average download speed.
- the video segments with high resolutions and high bitrates are selected as the plurality of first video segments
- the video segments with low resolutions and low bitrates are selected as the plurality of first video segments.
- step S 400 a download link is determined according to the plurality of first video segments.
- the download link of the first video is determined according to a predetermined download strategy, and the plurality of first video segments waiting to be downloaded and a download order are specified by the download link.
- the download order for downloading the first video segments, which need to be downloaded is an ascending order of the bitrates.
- the present disclosure is not limited thereto. Those skilled in the art can determine the download strategy according to the bitrates and the average download speed, in order to optimize video transmission.
- step S 500 the plurality of first video segments are downloaded according to the download link.
- download processes are operated on the plurality of first video segments.
- the video segments which need to be downloaded and are selected according to the average speed, are downloaded, so as to optimize video transmission.
- the method for downloading the VR video further comprises: obtaining a mapping table between resolutions and bitrates, and determining the plurality of first video segments by looking up the mapping table.
- the mapping table defines correspondences between a plurality of resolutions and a plurality of bitrates.
- the mapping table is looked up to obtain the set bitrates of the plurality of video segments, so that the plurality of first video segments can be determined according to the average download speed and the set bitrates.
- the mapping table is generated during a video transcoding process.
- raw video data is processed into the plurality of video segments, for example, the raw video data can be processed into a basic video and an enhancement video which need to be overlaid together during playback.
- the bitrate of each of the plurality of video segments are recorded and are summarized as the mapping table between resolutions and bitrates.
- information provided by the play request comprises a field of view of the user
- the plurality of first video segments have a plurality of resolutions
- a resolution distribution of the plurality of first video segments is determined by the field of view of the user. For example, it is assumed that a video with 360-degree panoramic images is provided, the user's field of view ranges in 135 degrees, then the video segments with high resolutions are displayed in a screen region inside the user's field of view, and the video segments with low resolutions are displayed on the screen regions outside the user's field of view. Therefore, by decreasing times for downloading the high-resolution video segments, the amount of downloaded data can be reduced and the video transmission can be optimized.
- the user's field of view can be obtained according to parameters of a gyroscope in the head-mounted device.
- FIG. 4 is a flowchart presenting a detailed methodology of obtaining the mapping table between resolutions and bitrates during the video transcoding process.
- step S 1001 a plurality of resolution levels are defined, and a plurality of resolutions in the plurality of resolution levels are calculated according to a predetermined target resolution.
- step S 1002 bitrates corresponding to the resolutions in the plurality of resolution levels are calculated based on a target bitrate corresponding to the predetermined target resolution.
- step S 1003 the video transcoding process is performed according to the corresponding resolutions and the corresponding bitrates, in order to obtain a plurality of video segments of video data with different resolutions and different bitrates.
- N represents a total number of the plurality of resolution levels
- R 0 represents the predetermined target resolution
- bitrate 0 represents the target bitrate according to the target resolution.
- the resolution R 1 in the 1 st resolution level, the resolution R 2 in the 2 nd resolution level, . . . , and the resolution R N ⁇ 1 in the (N ⁇ 1) th resolution level can be calculated according to a resolution ratio between R n and R n+1 , so as to obtain the plurality of resolutions in all of the plurality of resolution levels.
- All of the bitrates can be obtained according to resolution ratios between bitrate n and bitrate n+1 , so that the mapping table between resolutions and bitrates is obtained.
- the plurality of video segments can be obtained by transcoding based on the corresponding resolutions and the corresponding bitrates.
- mapping table between resolutions and bitrates is not limited thereto.
- the mapping table can be obtained by other manners, for example, the mapping table between resolutions and bitrates may be set by experience.
- n one of the plurality of resolution level, wherein a lower n corresponding to a higher resolution, n ⁇ [0, 1, 2, . . . ] R n the resolution in the n th resolution level, a two-element array ( X n , Y n ) bitrate n the bitrate corresponding to the n th resolution level bitrate n base the bitrate of the basic video corresponding to the n th resolution level bitrate n enhance the bitrate of the enhancement video corresponding to the n th resolution level q a resolution ratio (which is, for example, 2n)
- a maximum resolution is defined as R 0 , bitrate 0 , X 0 base , Y 0 base , X 0 enhance , Y 0 enhance , and q are given by user, and also
- q X 0 enhance X 1 enhance > 1 ( 2 )
- bitrate corresponding to the n th resolution level satisfy following formulas:
- bitrate n For each one of bitrate n , bitrate n enhance , and bitrate n base there is:
- bitrate n + 1 bitrate n q 2 ( 7 )
- bitrate n base >bitrate n+1 base +bitrate n+1 enhance (8)
- bitrate n base > 1 q 2 - 1 ⁇ bitrate n enhance ( 9 )
- mapping table can be generated when X 0 base , Y 0 base , X 0 enhance , Y 0 enhance and q are provided by the user and satisfy an inequality:
- FIG. 5 is a structural diagram presenting a system for downloading a VR video according to an embodiment of the present disclosure.
- the system 500 for downloading a VR video comprises a request receiving module 501 , a speed calculation module 502 , a segment obtaining module 503 , a link obtaining module 504 and a download executing module 505 .
- the request receiving module 501 is configured to receive a play request for a first video.
- the play request is sent from a display device or a VR device.
- Information provided by the play request includes a name of the current playing video and a current field of view of a user.
- the first video here refers to a specific video.
- the speed calculating module 502 is configured to calculate an average download speed.
- the average download speed is calculated by making statistics on the amount of downloaded and download time.
- the segment obtaining module 503 is configured to select a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates.
- the plurality of video segments of the first video is obtained during a video transcoding process and have different resolutions and bitrates.
- the segment obtaining module 503 is configured to obtain the video segments for downloading selected from the plurality of video segments for downloading according to the current average download speed.
- a download strategy is set to select the video segments with low bitrates and low resolutions for downloading when the average download speed is low, and to select the video segments with high bitrates and high resolutions for downloading when the average download speed is relatively high.
- the link obtaining module 504 is configured to determine a download link according to the plurality of first video segments. Specifically, the link obtaining module 504 is configured to determine a download manner and a download order according to the predetermined download strategy, for example, the link obtaining module 504 downloads the video segments with high resolutions first, and then downloads the video segments with low resolutions.
- the download executing module 505 is configured to download the plurality of video segments according to the download link. That is, the download executing module performs a download operation according to the download link provided by the link obtaining module 505 .
- the above-described system for downloading the VR video further comprises: a mapping table obtaining module which is configured to generate a mapping table between resolutions and bitrates during the process when the plurality of video segments are obtained according to the video data.
- the mapping table obtaining module comprises: a first calculating unit which is configured to define a plurality of resolution levels, and calculate resolutions in the plurality of resolution levels according to a predetermined target resolution; and a second calculating unit which is configured to calculate bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
- the play request comprises a field of view of a user
- the plurality of first video segments have a plurality of resolutions
- a resolution distribution of the plurality of first video segments is determined by the field of view of the user.
- the video segments, which need to be downloaded are determined according to the average download speed, so that the amount of downloaded data can be decreased and the video transmission can be optimized. It should be pointed out that the video data needs to be continuously downloaded during VR video playback, therefore, the process for selecting the video segments with proper resolutions and bitrates for downloading can be continuously performed according to the current average download speed, that is, the video transmission optimization is achieved by dynamic selection processes.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Graphics (AREA)
- Human Computer Interaction (AREA)
- Geometry (AREA)
- Environmental & Geological Engineering (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Controls And Circuits For Display Device (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Image Generation (AREA)
- Processing Or Creating Images (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Image Processing (AREA)
Abstract
A method for downloading a VR video is disclosed. The method comprises: receiving a play request for a first video; calculating an average download speed; selecting a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates; determining a download link according to the plurality of first video segments; and downloading the plurality of first video segments according to the download link. The video segments, which need to be downloaded, are determined according to the average download speed, so that the amount of downloaded data can be decreased and the video transmission can be optimized. The present disclosure also provides a system for downloading a VR video.
Description
- This application claims the priority and benefit of U.S. provisional application 62/441,936, filed on Jan. 3, 2017, which is incorporated herein by reference in its entirety.
- The present disclosure relates to video processing technology, and more particularly, to a method and a system for downloading a VR video.
- Virtual Reality (VR) is a computer simulation technology for creating and experiencing a virtual world. For example, a three-dimensional real-time image can be presented based on a technology which tracks a user's head, eyes or hands. In the network-based virtual reality technology, full-view video data can be pre-stored on a server, and then transmitted to a display device. A display device can be glasses, a head-mounted display, etc. A video is displayed on the display device in accordance with a field of view of the user.
- However, a high-resolution video data occupies large transmission of bandwidth and requires high computing power from the display devices. Presenting high-resolution VR video on internet is difficult, precisely, the existing video streaming technology cannot fulfill the virtual reality. Therefore, in order to present VR video smoothly in real-time, it is desirable to further improve the existing video streaming technology to save bandwidth and reduce performance requirements for display devices.
- How to improve transmission effect by network prediction without changing a total amount of transferred data is always a problem waiting to be solved by industry.
- In view of this, the present disclosure provides a method and a system for downloading a VR video for optimizing video transmission.
- According to a first aspect of the present disclosure, there is provided a method for downloading a VR video, comprising: receiving a play request for a first video; calculating an average download speed; selecting a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates; determining a download link according to the plurality of first video segments; and downloading the plurality of video segments according to the download link.
- Preferably, the method further comprises: obtaining a mapping table between resolutions and bitrates during a video transcoding process; and determining the plurality of first video segments by looking up the mapping table.
- Preferably, the step of obtaining the mapping table between resolutions and bitrates during the video transcoding process comprises: defining a plurality of resolution levels, and calculating a plurality of resolutions in the plurality of resolution levels according to a predetermined target resolution; and calculating bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
- Preferably, the step of defining a plurality of resolution levels, and calculating a plurality of resolutions in the plurality of resolution levels according to a predetermined target resolution, comprises: calculating the resolution in one of the plurality of resolution levels following formulas below,
-
- wherein, n represents the one of the plurality of resolution levels, R0 represents a maximum value of the plurality of resolutions, {X0 base, Y0 base} represents the resolution of a basic video corresponding to the one of the plurality of the resolution levels, {X0 enhance, Y0 enhance} represents the resolution of an enhancement video corresponding to the one of the plurality of resolution levels, q represents a resolution ratio, Rn{Xn, Yn} represents the resolution in the one of the plurality of resolution levels; wherein the step of calculating bitrates corresponding to the plurality of resolutions in the plurality of resolution levels based on a target bitrate of the predetermined target resolution, comprises: calculating the bitrates corresponding to the resolutions in the plurality of resolution levels following formulas below,
-
- wherein, for each one of bitraten, bitraten enhance and bitraten base, there is:
-
- and an inequality is satisfied:
-
- wherein, bitrate0 represents a maximum value of the bitrates, bitraten, bitraten base, bitraten enhance respectively represent the bitrate, the bitrate of the basic video, and the bitrate of the enhancement video corresponding to the one of the plurality of resolution levels, wherein n is an integer equal to or larger than 1.
- Preferably, the play request comprises a field of view of a user, the plurality of first video segments have a plurality of resolutions, and a resolution distribution of the plurality of first video segments is determined by the field of view of the user.
- Preferably, the resolution of a first video segment within the field of view is higher than the resolution of a first video segment outside the field of view.
- Preferably, the average download speed and the resolution distribution of the plurality of first video segments are positively correlated.
- According to a second aspect of the present disclosure, there is provided a system for downloading a VR video, comprising: a request receiving module configured to receive a play request of a first video; a speed calculating module configured to calculate an average download speed; a segment obtaining module configured to select a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates; a link obtaining module configured to determine a download link according to the plurality of first video segments; and a download executing module configured to download the plurality of video segments according to the download link.
- Preferably, the system further comprises: a mapping table obtaining module configured to obtaining a mapping table between resolutions and bitrates during a video transcoding process, and to determine the plurality of first video segments by looking up the mapping table.
- Preferably, the mapping table obtaining module comprises: a first calculating unit configured to define a plurality of resolution levels and calculate a plurality of resolutions in the plurality of resolution levels according to the predetermined target resolution; and calculating bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
- The present disclosure provides a plurality of video segments with proper resolutions and proper bitrates for downloading according to the average download speed. As the average download speed reflects current network conditions, a resolution reflects the image precision, a bitrate reflects bit number of data transmitted per unit time, the resolution, the bitrate and average download speed can be synthesized to optimize the video transmission by using an optimal strategy to select the appropriate video segments. Furthermore, the video segments with high resolutions and low resolutions are selected according to the user's field of view, which can decrease the amount of transferred data without affecting the user's demands for viewing the video.
- The above and other objects, advantages and features of the present disclosure will become more fully understood from the detailed description given hereinbelow in connection with the appended drawings, and wherein:
-
FIG. 1 is a diagram illustrating an example network of a VR playback system; -
FIG. 2 is a flowchart showing a method used in the VR playback system ofFIG. 1 ; -
FIG. 3 is a flowchart presenting the method for downloading a VR video according to an embodiment of the present disclosure; -
FIG. 4 is a flowchart presenting a detailed methodology of obtaining the mapping table between resolutions and bitrates during the video transcoding process; and -
FIG. 5 is a structural diagram presenting a system for downloading a VR video according to an embodiment of the present disclosure. - Exemplary embodiments the present disclosure will be described in more details below with reference to the accompanying drawings. In the drawings, like reference numerals denote like members. The figures are not drawn to scale, for the sake of clarity. Moreover, some well-known parts may not be shown.
-
FIG. 1 is a diagram illustrating an example network of a VR playback system. TheVR playback system 10 includes aserver 100 and adisplay device 120 which are coupled with each other through anetwork 110, and a VR device. For example, theserver 100 may be a stand-alone computer server or a server cluster. Theserver 100 is used to store various video data and to store various applications that process these video data. For example, various daemons run on theserver 100 in real time, so as to process various video data in theserver 100 and to respond various requests from VR devices and thedisplay device 120. Thenetwork 110 may be a selected one or selected ones from the group consisting of an internet, a local area network, an internet of things, and the like. For example, thedisplay device 120 may be any of the computing devices, including a computer device having an independent display screen and a processing capability. Thedisplay device 120 may be a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a palmtop computer, a personal digital assistant, a smart phone, an intelligent electrical apparatus, a game console, an iPad/iPhone, a video player, a DVD recorder/player, a television, or a home entertainment system. Thedisplay device 120 may store VR player software as a VR player. When the VR player is started, it requests and downloads various video data from theserver 100, and renders and plays the video data in the display device. In this example, theVR device 130 is a stand-alone head-mounted device that can interact with thedisplay device 120 and theserver 100, to communicate the user's current information to thedisplay device 120 and/or theserver 100 through signaling. The user's current information is, for example, a field of view of the user, a position modification of a helmet, a modification of the user's sight orientation, etc. According to the information, thedisplay device 120 can flexibly process the currently played video data. In some embodiments, when a position of user's helmet changes, thedisplay device 120 determines that a core viewing region for the user has been changed and starts to play video data with high resolution in the changed core viewing region. - In the above embodiment, the
VR device 130 is a stand-alone head-mounted device. However, those skilled in the art should understand that theVR device 130 is not limited thereto, and theVR device 130 may also be an all-in-one head-mounted device. The all-in-one head-mounted device itself has a display screen, so that it is not necessary to connect the all-in-one head-mounted device with the external display device. For example, in this example, if the all-in-one head-mounted device is used as the VR device, thedisplay device 120 may be eliminated. At this point, the all-in-one head-mounted device is configured to obtain video data from theserver 100 and to perform playback operation, and the all-in-one head-mounted device is also configured to detect user's current field of view modification information and to adjust the playback operation according to the field of view modification information. -
FIG. 2 is a flowchart showing a method used in the VR playback system ofFIG. 1 . The method comprises following steps. - In step S10, a source video data process is operated on server side.
- In step S20, the display device obtains some related information of the user based on interactions with the VR device.
- In step S30, the display device requests video data and receives video data from the server according to the related information.
- In step S40, the display device side performs a rendering process based on the received video data.
- Wherein, in the step S30, the video data is downloaded. Furthermore, the present disclosure provides a method for downloading a VR video. The method downloads the VR video by predicting a network condition according to a calculated average download speed and providing a download link according to bitrates, in order to optimize VR video streaming transmission.
-
FIG. 3 is a flowchart presenting the method for downloading a VR video according to an embodiment of the present disclosure. Specifically, following steps are comprised. - In step S100, a play request for a first video is received.
- The first video in this step refers to a specific video.
- In step S200, an average download speed is calculated.
- The average download speed is calculated by making statistics on an amount of downloaded data and download time. In general, the average download speed is calculated once in real time. For example, the average download speed is calculated once before data of the first video starts to be downloaded, or the average download speed is calculated once before each video segments of the first video starts to be downloaded.
- In step S300, a plurality of first video segments are selected from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates.
- In the source video data process on the server side, a complete source VR video can be processed into the plurality of video segments, for example, the plurality of video segments may be obtained by segmenting the source VR video in temporal domain and/or in spatial domain. The plurality of video segments have different resolutions and bitrates. When the first video is requested, the plurality of first video segments can be selected from the plurality of video segments with different resolutions and bitrates. Therefore, in this step, each of the plurality of first video segments is the video segment with a proper resolution and a proper bitrate selected from the plurality of video segments, according to the average download speed. For example, when the average download speed is relatively high, the video segments with high resolutions and high bitrates are selected as the plurality of first video segments, and when the average download speed is relatively low, the video segments with low resolutions and low bitrates are selected as the plurality of first video segments.
- In step S400, a download link is determined according to the plurality of first video segments.
- In this step, the download link of the first video is determined according to a predetermined download strategy, and the plurality of first video segments waiting to be downloaded and a download order are specified by the download link. For example, the download order for downloading the first video segments, which need to be downloaded, is an ascending order of the bitrates. However, the present disclosure is not limited thereto. Those skilled in the art can determine the download strategy according to the bitrates and the average download speed, in order to optimize video transmission.
- In step S500, the plurality of first video segments are downloaded according to the download link.
- In this step, download processes are operated on the plurality of first video segments.
- In this embodiment, the video segments, which need to be downloaded and are selected according to the average speed, are downloaded, so as to optimize video transmission.
- In an optional embodiment, the method for downloading the VR video further comprises: obtaining a mapping table between resolutions and bitrates, and determining the plurality of first video segments by looking up the mapping table. The mapping table defines correspondences between a plurality of resolutions and a plurality of bitrates. The mapping table is looked up to obtain the set bitrates of the plurality of video segments, so that the plurality of first video segments can be determined according to the average download speed and the set bitrates. Further, the mapping table is generated during a video transcoding process. During the video transcoding process, raw video data is processed into the plurality of video segments, for example, the raw video data can be processed into a basic video and an enhancement video which need to be overlaid together during playback. In this process, the bitrate of each of the plurality of video segments are recorded and are summarized as the mapping table between resolutions and bitrates.
- In a preferred embodiment, information provided by the play request comprises a field of view of the user, the plurality of first video segments have a plurality of resolutions, and a resolution distribution of the plurality of first video segments is determined by the field of view of the user. For example, it is assumed that a video with 360-degree panoramic images is provided, the user's field of view ranges in 135 degrees, then the video segments with high resolutions are displayed in a screen region inside the user's field of view, and the video segments with low resolutions are displayed on the screen regions outside the user's field of view. Therefore, by decreasing times for downloading the high-resolution video segments, the amount of downloaded data can be reduced and the video transmission can be optimized. The user's field of view can be obtained according to parameters of a gyroscope in the head-mounted device.
-
FIG. 4 is a flowchart presenting a detailed methodology of obtaining the mapping table between resolutions and bitrates during the video transcoding process. - In step S1001, a plurality of resolution levels are defined, and a plurality of resolutions in the plurality of resolution levels are calculated according to a predetermined target resolution.
- In step S1002, bitrates corresponding to the resolutions in the plurality of resolution levels are calculated based on a target bitrate corresponding to the predetermined target resolution.
- In step S1003, the video transcoding process is performed according to the corresponding resolutions and the corresponding bitrates, in order to obtain a plurality of video segments of video data with different resolutions and different bitrates.
- This embodiment will be understood by a following example.
- It is assumed that N represents a total number of the plurality of resolution levels, R0 represents the predetermined target resolution, and bitrate0 represents the target bitrate according to the target resolution. The resolution R1 in the 1st resolution level, the resolution R2 in the 2nd resolution level, . . . , and the resolution RN−1 in the (N−1)th resolution level, can be calculated according to a resolution ratio between Rn and Rn+1, so as to obtain the plurality of resolutions in all of the plurality of resolution levels. All of the bitrates can be obtained according to resolution ratios between bitraten and bitraten+1, so that the mapping table between resolutions and bitrates is obtained. Finally, during the video transcoding process, the plurality of video segments can be obtained by transcoding based on the corresponding resolutions and the corresponding bitrates.
- Those skilled in the art should understand that the manner for obtaining the mapping table between resolutions and bitrates is not limited thereto. The mapping table can be obtained by other manners, for example, the mapping table between resolutions and bitrates may be set by experience.
- In a following example, formulas about the resolutions and the bitrates and derivations of the formulas are provided. In this example, it is assumed that the video data is processed into multiple video data each have a corresponding resolution in each of the plurality of resolution levels, and each of the multiple video data having the corresponding resolution comprises a basic video and an enhancement video which are overlaid together during playback. Table 1 shows definitions of variables included in the formulas.
-
TABLE 1 variate meanings n one of the plurality of resolution level, wherein a lower n corresponding to a higher resolution, n□[0, 1, 2, . . . ] Rn the resolution in the nth resolution level, a two-element array ( Xn, Yn) bitraten the bitrate corresponding to the nth resolution level bitraten base the bitrate of the basic video corresponding to the nth resolution level bitraten enhance the bitrate of the enhancement video corresponding to the nth resolution level q a resolution ratio (which is, for example, 2n) - In general, a maximum resolution is defined as R0, bitrate0, X0 base, Y0 base, X0 enhance, Y0 enhance, and q are given by user, and also
-
- The bitrate corresponding to the nth resolution level satisfy following formulas:
-
- For each one of bitraten, bitraten enhance, and bitraten base there is:
-
- and an inequality is satisfied:
-
bitraten base>bitraten+1 base+bitraten+1 enhance (8) - By substituting (7) into (8), there is:
-
- So that the mapping table can be generated when X0 base, Y0 base, X0 enhance, Y0 enhance and q are provided by the user and satisfy an inequality:
-
-
FIG. 5 is a structural diagram presenting a system for downloading a VR video according to an embodiment of the present disclosure. Thesystem 500 for downloading a VR video comprises arequest receiving module 501, aspeed calculation module 502, asegment obtaining module 503, alink obtaining module 504 and adownload executing module 505. - The
request receiving module 501 is configured to receive a play request for a first video. The play request is sent from a display device or a VR device. Information provided by the play request includes a name of the current playing video and a current field of view of a user. The first video here refers to a specific video. - The
speed calculating module 502 is configured to calculate an average download speed. The average download speed is calculated by making statistics on the amount of downloaded and download time. - The
segment obtaining module 503 is configured to select a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates. The plurality of video segments of the first video is obtained during a video transcoding process and have different resolutions and bitrates. Thesegment obtaining module 503 is configured to obtain the video segments for downloading selected from the plurality of video segments for downloading according to the current average download speed. In general, a download strategy is set to select the video segments with low bitrates and low resolutions for downloading when the average download speed is low, and to select the video segments with high bitrates and high resolutions for downloading when the average download speed is relatively high. - The
link obtaining module 504 is configured to determine a download link according to the plurality of first video segments. Specifically, thelink obtaining module 504 is configured to determine a download manner and a download order according to the predetermined download strategy, for example, thelink obtaining module 504 downloads the video segments with high resolutions first, and then downloads the video segments with low resolutions. - The
download executing module 505 is configured to download the plurality of video segments according to the download link. That is, the download executing module performs a download operation according to the download link provided by thelink obtaining module 505. - Preferably, the above-described system for downloading the VR video further comprises: a mapping table obtaining module which is configured to generate a mapping table between resolutions and bitrates during the process when the plurality of video segments are obtained according to the video data. The mapping table obtaining module comprises: a first calculating unit which is configured to define a plurality of resolution levels, and calculate resolutions in the plurality of resolution levels according to a predetermined target resolution; and a second calculating unit which is configured to calculate bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
- In a preferred embodiment, the play request comprises a field of view of a user, the plurality of first video segments have a plurality of resolutions, and a resolution distribution of the plurality of first video segments is determined by the field of view of the user.
- In the above-described embodiment, the video segments, which need to be downloaded, are determined according to the average download speed, so that the amount of downloaded data can be decreased and the video transmission can be optimized. It should be pointed out that the video data needs to be continuously downloaded during VR video playback, therefore, the process for selecting the video segments with proper resolutions and bitrates for downloading can be continuously performed according to the current average download speed, that is, the video transmission optimization is achieved by dynamic selection processes.
- The embodiments were chosen and described in order to best explain the principles of the disclosure and its practical applications, to thereby enable others skilled in the art to best utilize the disclosure. The disclosure is intended to cover alternatives, modifications and equivalents that may be included within the spirit and scope of the disclosure as defined by the appended claims.
- The foregoing descriptions of specific embodiments of the present disclosure have been presented, but are not intended to limit the disclosure to the precise forms disclosed. It will be readily apparent to one skilled in the art that many modifications and changes may be made in the present disclosure. Any modifications, equivalence, variations of the preferred embodiments can be made without departing from the doctrine and spirit of the present disclosure.
Claims (10)
1. A method for downloading a VR video, comprising:
receiving a play request for a first video;
calculating an average download speed;
selecting a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates;
determining a download link according to the plurality of first video segments; and
downloading the plurality of first video segments according to the download link.
2. The method according to claim 1 , further comprising: obtaining a mapping table between resolutions and bitrates during a video transcoding process; and determining the plurality of first video segments by looking up the mapping table.
3. The method according to claim 2 , wherein the step of obtaining the mapping table between resolutions and bitrates during the video transcoding process comprises:
defining a plurality of resolution levels, and calculating a plurality of resolutions in the plurality of resolution levels according to a predetermined target resolution; and
calculating bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
4. The method according to claim 3 , wherein the step of defining the plurality of resolution levels and calculating the plurality of resolutions in the plurality of resolution levels according to the predetermined target resolution comprises:
calculating the resolution in one of the plurality of resolution levels following formulas below,
wherein, n represents the one of the plurality of resolution levels, R0 represents a maximum value of the plurality of resolutions, {X0 base, Y0 base} represents the resolution of a basic video corresponding to the one of the plurality of the resolution levels, {X0 enhance, Y0 enhance} represents the resolution of an enhancement video corresponding to the one of the plurality of resolution levels, q represents a resolution ratio, Rn{Xn, Yn} represents the resolution in the one of the plurality of resolution levels; wherein the step of calculating bitrates corresponding to the plurality of resolutions in the plurality of resolution levels based on a target bitrate of the predetermined target resolution, comprises:
calculating the bitrates corresponding to the resolutions in the plurality of resolution levels following formulas below,
wherein for each one of bitraten, bitraten enhance and bitraten base, there is:
and an inequality is satisfied:
wherein, bitrate0 represents a maximum value of the bitrates, bitraten, bitraten base bitraten enhance respectively represent the bitrate, the bitrate of the basic video, and the bitrate of the enhancement video corresponding to the one of the plurality of resolution levels, wherein n is an integer equal to or larger than 1.
5. The method according to claim 1 , wherein the play request comprises a field of view of a user, the plurality of first video segments have a plurality of resolutions, and a resolution distribution of the plurality of first video segments is determined by the field of view of the user.
6. The method according to claim 5 , wherein the resolution of a first video segment within the field of view is higher than the resolution of a first video segment outside the field of view.
7. The method according to claim 1 , wherein the average download speed and the resolution distribution of the plurality of first video segments are positively correlated.
8. A system for downloading a VR video, comprising:
a request receiving module configured to receive a play request of a first video;
a speed calculating module configured to calculate an average download speed;
a segment obtaining module configured to select a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates;
a link obtaining module configured to determine a download link according to the plurality of first video segments; and
a download executing module configured to download the plurality of video segments according to the download link.
9. The system according to claim 8 , further comprising: a mapping table obtaining module configured to obtaining a mapping table between resolutions and bitrates during a video transcoding process, and to determine the plurality of first video segments by looking up the mapping table.
10. The system according to claim 9 , wherein the mapping table obtaining module comprises:
a first calculating unit configured to define a plurality of resolution levels and calculate a plurality of resolutions in the plurality of resolution levels according to the predetermined target resolution; and
a second calculating unit configured to calculate bitrates corresponding to the plurality of resolutions in the plurality of resolution levels, based on a target bitrate corresponding to the predetermined target resolution.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/860,430 US20180191868A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762441936P | 2017-01-03 | 2017-01-03 | |
US15/860,430 US20180191868A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization |
Publications (1)
Publication Number | Publication Date |
---|---|
US20180191868A1 true US20180191868A1 (en) | 2018-07-05 |
Family
ID=62711388
Family Applications (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/860,449 Expired - Fee Related US10334238B2 (en) | 2017-01-03 | 2018-01-02 | Method and system for real-time rendering displaying high resolution virtual reality (VR) video |
US15/860,494 Abandoned US20180189980A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Providing Virtual Reality (VR) Video Transcoding and Broadcasting |
US15/860,392 Abandoned US20180192044A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Providing A Viewport Division Scheme for Virtual Reality (VR) Video Streaming |
US15/860,471 Abandoned US20180192022A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Real-time Rendering Displaying Virtual Reality (VR) On Mobile Using Head-Up Display Devices |
US15/860,430 Abandoned US20180191868A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization |
US15/860,358 Abandoned US20180192063A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Virtual Reality (VR) Video Transcode By Extracting Residual From Different Resolutions |
Family Applications Before (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/860,449 Expired - Fee Related US10334238B2 (en) | 2017-01-03 | 2018-01-02 | Method and system for real-time rendering displaying high resolution virtual reality (VR) video |
US15/860,494 Abandoned US20180189980A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Providing Virtual Reality (VR) Video Transcoding and Broadcasting |
US15/860,392 Abandoned US20180192044A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Providing A Viewport Division Scheme for Virtual Reality (VR) Video Streaming |
US15/860,471 Abandoned US20180192022A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Real-time Rendering Displaying Virtual Reality (VR) On Mobile Using Head-Up Display Devices |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/860,358 Abandoned US20180192063A1 (en) | 2017-01-03 | 2018-01-02 | Method and System for Virtual Reality (VR) Video Transcode By Extracting Residual From Different Resolutions |
Country Status (2)
Country | Link |
---|---|
US (6) | US10334238B2 (en) |
CN (6) | CN108366293A (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109343518A (en) * | 2018-09-03 | 2019-02-15 | 浙江大丰实业股份有限公司 | Universal ride scene drives platform |
US10616621B2 (en) * | 2018-06-29 | 2020-04-07 | At&T Intellectual Property I, L.P. | Methods and devices for determining multipath routing for panoramic video content |
US10812774B2 (en) | 2018-06-06 | 2020-10-20 | At&T Intellectual Property I, L.P. | Methods and devices for adapting the rate of video content streaming |
CN112468806A (en) * | 2020-11-12 | 2021-03-09 | 中山大学 | Panoramic video transmission optimization method for cloud VR platform |
US11019361B2 (en) | 2018-08-13 | 2021-05-25 | At&T Intellectual Property I, L.P. | Methods, systems and devices for adjusting panoramic view of a camera for capturing video content |
US20210329255A1 (en) * | 2018-10-22 | 2021-10-21 | Bitmovin, Inc. | Video Encoding Based on Customized Bitrate Table |
US11190820B2 (en) | 2018-06-01 | 2021-11-30 | At&T Intellectual Property I, L.P. | Field of view prediction in live panoramic video streaming |
CN114979799A (en) * | 2022-05-20 | 2022-08-30 | 北京字节跳动网络技术有限公司 | Panoramic video processing method, device, equipment and storage medium |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10291910B2 (en) * | 2016-02-12 | 2019-05-14 | Gopro, Inc. | Systems and methods for spatially adaptive video encoding |
US10331862B2 (en) * | 2017-04-20 | 2019-06-25 | Cisco Technology, Inc. | Viewport decryption |
CN115842907A (en) * | 2018-03-27 | 2023-03-24 | 京东方科技集团股份有限公司 | Rendering method, computer product and display device |
US11232532B2 (en) * | 2018-05-30 | 2022-01-25 | Sony Interactive Entertainment LLC | Multi-server cloud virtual reality (VR) streaming |
CN109375369B (en) * | 2018-11-23 | 2021-05-18 | 国网天津市电力公司 | Distortion preprocessing method in VR (virtual reality) large-screen cinema mode |
CN111510777B (en) | 2019-01-30 | 2021-11-23 | 上海哔哩哔哩科技有限公司 | Method and device for measuring network speed, computer equipment and readable storage medium |
CN111669666A (en) * | 2019-03-08 | 2020-09-15 | 北京京东尚科信息技术有限公司 | Method, device and system for simulating reality |
CN111866485A (en) * | 2019-04-25 | 2020-10-30 | 中国移动通信有限公司研究院 | Stereoscopic picture projection and transmission method, device and computer readable storage medium |
CN110336994B (en) * | 2019-07-04 | 2021-06-22 | 上海索倍信息科技有限公司 | Naked eye 3D display system |
CN110381331A (en) * | 2019-07-23 | 2019-10-25 | 深圳市道通智能航空技术有限公司 | A kind of image processing method, device, equipment of taking photo by plane and storage medium |
CN110490962B (en) * | 2019-08-20 | 2023-09-15 | 武汉邦拓信息科技有限公司 | Remote rendering method based on video stream |
CN110544425A (en) * | 2019-09-13 | 2019-12-06 | 广州城市职业学院 | ancient building VR display system |
EP4032311A1 (en) | 2019-09-19 | 2022-07-27 | InterDigital CE Patent Holdings | Devices and methods for generating and rendering immersive video |
CN111489428B (en) * | 2020-04-20 | 2023-06-30 | 北京字节跳动网络技术有限公司 | Image generation method, device, electronic equipment and computer readable storage medium |
US11245911B1 (en) * | 2020-05-12 | 2022-02-08 | Whirlwind 3D, LLC | Video encoder/decoder (codec) for real-time applications and size/b and width reduction |
CN111754614B (en) * | 2020-06-30 | 2024-07-02 | 平安国际智慧城市科技股份有限公司 | VR-based video rendering method and device, electronic equipment and storage medium |
CN114286142B (en) * | 2021-01-18 | 2023-03-28 | 海信视像科技股份有限公司 | Virtual reality equipment and VR scene screen capturing method |
CN113347402A (en) * | 2021-06-28 | 2021-09-03 | 筑友建筑装饰装修工程有限公司 | Improved method, device and storage medium for rendering immersive content based on Unity |
US11812146B2 (en) * | 2021-12-02 | 2023-11-07 | Fotonation Limited | Method and system for camera motion blur reduction |
CN114466220A (en) * | 2022-01-29 | 2022-05-10 | 维沃移动通信有限公司 | Video downloading method and electronic equipment |
CN115002519A (en) * | 2022-05-31 | 2022-09-02 | 北京势也网络技术有限公司 | Method for playing 8K panoramic video file in low-bandwidth network |
CN115396731A (en) * | 2022-08-10 | 2022-11-25 | 北京势也网络技术有限公司 | Panoramic video playing method and device, electronic equipment and readable storage medium |
CN116880723B (en) * | 2023-09-08 | 2023-11-17 | 江西格如灵科技股份有限公司 | 3D scene display method and system |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120254455A1 (en) * | 2011-03-30 | 2012-10-04 | Verizon Patent And Licensing Inc. | Downloading video using excess bandwidth |
US8897370B1 (en) * | 2009-11-30 | 2014-11-25 | Google Inc. | Bitrate video transcoding based on video coding complexity estimation |
US20150208103A1 (en) * | 2012-08-08 | 2015-07-23 | National University Of Singapore | System and Method for Enabling User Control of Live Video Stream(s) |
US20160191585A1 (en) * | 2014-12-24 | 2016-06-30 | Intel Corporation | Link-aware streaming adaptation |
Family Cites Families (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3196889B2 (en) * | 1996-09-05 | 2001-08-06 | 株式会社アルファ | Three-dimensional image processing method and computer-readable recording medium storing a program for causing a computer to execute the three-dimensional image processing method |
TWI262725B (en) * | 2005-06-30 | 2006-09-21 | Cheertek Inc | Video decoding apparatus and digital audio and video display system capable of controlling presentation of subtitles and method thereof |
JP5891426B2 (en) * | 2011-03-31 | 2016-03-23 | パナソニックIpマネジメント株式会社 | An image drawing apparatus, an image drawing method, and an image drawing program for drawing an all-around stereoscopic image |
US8810598B2 (en) * | 2011-04-08 | 2014-08-19 | Nant Holdings Ip, Llc | Interference based augmented reality hosting platforms |
GB2501929B (en) * | 2012-05-11 | 2015-06-24 | Sony Comp Entertainment Europe | Apparatus and method for augmented reality |
US9129429B2 (en) * | 2012-10-24 | 2015-09-08 | Exelis, Inc. | Augmented reality on wireless mobile devices |
GB2509953B (en) * | 2013-01-18 | 2015-05-20 | Canon Kk | Method of displaying a region of interest in a video stream |
US9196199B2 (en) * | 2013-02-12 | 2015-11-24 | Pixtronix, Inc. | Display having staggered display element arrangement |
CN103702139B (en) * | 2013-12-13 | 2017-02-01 | 华中科技大学 | Video-on-demand system based on scalable coding under mobile environment |
US9398250B2 (en) * | 2014-01-06 | 2016-07-19 | Arun Sobti & Associates, Llc | System and apparatus for smart devices based conferencing |
CN105025351B (en) * | 2014-04-30 | 2018-06-29 | 深圳Tcl新技术有限公司 | The method and device of DST PLAYER buffering |
JP6337614B2 (en) * | 2014-05-23 | 2018-06-06 | セイコーエプソン株式会社 | Control device, robot, and control method |
US9918136B2 (en) * | 2014-05-29 | 2018-03-13 | Nextvr Inc. | Methods and apparatus for delivering content and/or playing back content |
CN104240281A (en) * | 2014-08-28 | 2014-12-24 | 东华大学 | Virtual reality head-mounted device based on Unity3D engine |
CN104268922B (en) * | 2014-09-03 | 2017-06-06 | 广州博冠信息科技有限公司 | A kind of image rendering method and image rendering device |
CN104616243B (en) * | 2015-01-20 | 2018-02-27 | 北京道和汇通科技发展有限公司 | A kind of efficient GPU 3 D videos fusion method for drafting |
US20160261908A1 (en) * | 2015-03-05 | 2016-09-08 | Htc Corporation | Media streaming system and control method thereof |
CN104735464A (en) * | 2015-03-31 | 2015-06-24 | 华为技术有限公司 | Panorama video interactive transmission method, server and client end |
CN104717507A (en) * | 2015-03-31 | 2015-06-17 | 北京奇艺世纪科技有限公司 | Video transcoding method and device |
US10083363B2 (en) * | 2015-05-26 | 2018-09-25 | Nbcuniversal Media, Llc | System and method for customizing content for a user |
WO2016201423A1 (en) * | 2015-06-12 | 2016-12-15 | Google Inc. | Electronic display stabilization for head mounted display |
US9956054B2 (en) * | 2015-06-25 | 2018-05-01 | EchoPixel, Inc. | Dynamic minimally invasive surgical-aware assistant |
WO2017060423A1 (en) * | 2015-10-08 | 2017-04-13 | Koninklijke Kpn N.V. | Enhancing a region of interest in video frames of a video stream |
CN106919248A (en) * | 2015-12-26 | 2017-07-04 | 华为技术有限公司 | It is applied to the content transmission method and equipment of virtual reality |
CN105916022A (en) * | 2015-12-28 | 2016-08-31 | 乐视致新电子科技(天津)有限公司 | Video image processing method and apparatus based on virtual reality technology |
CN105455285B (en) * | 2015-12-31 | 2019-02-12 | 北京小鸟看看科技有限公司 | A kind of virtual implementing helmet adaptation method |
US10412130B2 (en) * | 2016-04-04 | 2019-09-10 | Hanwha Techwin Co., Ltd. | Method and apparatus for playing media stream on web browser |
US10313417B2 (en) * | 2016-04-18 | 2019-06-04 | Qualcomm Incorporated | Methods and systems for auto-zoom based adaptive video streaming |
CN105898565A (en) * | 2016-04-28 | 2016-08-24 | 乐视控股(北京)有限公司 | Video processing method and device |
US9721393B1 (en) * | 2016-04-29 | 2017-08-01 | Immersive Enterprises, LLC | Method for processing and delivering virtual reality content to a user |
CN106060570B (en) * | 2016-06-30 | 2019-06-14 | 北京奇艺世纪科技有限公司 | A kind of full-view video image plays, coding method and device |
CN106060515B (en) * | 2016-07-14 | 2018-11-06 | 腾讯科技(深圳)有限公司 | Panorama pushing method for media files and device |
CN106231317A (en) * | 2016-09-29 | 2016-12-14 | 三星电子(中国)研发中心 | Video processing, coding/decoding method and device, VR terminal, audio/video player system |
US10595069B2 (en) * | 2016-12-05 | 2020-03-17 | Adobe Inc. | Prioritizing tile-based virtual reality video streaming using adaptive rate allocation |
US20180295375A1 (en) * | 2017-04-05 | 2018-10-11 | Lyrical Labs Video Compression Technology, LLC | Video processing and encoding |
CN107087212B (en) * | 2017-05-09 | 2019-10-29 | 杭州码全信息科技有限公司 | Interactive panoramic video transcoding and playback method and system based on spatial scalable coding |
-
2018
- 2018-01-02 US US15/860,449 patent/US10334238B2/en not_active Expired - Fee Related
- 2018-01-02 US US15/860,494 patent/US20180189980A1/en not_active Abandoned
- 2018-01-02 US US15/860,392 patent/US20180192044A1/en not_active Abandoned
- 2018-01-02 US US15/860,471 patent/US20180192022A1/en not_active Abandoned
- 2018-01-02 US US15/860,430 patent/US20180191868A1/en not_active Abandoned
- 2018-01-02 US US15/860,358 patent/US20180192063A1/en not_active Abandoned
- 2018-01-03 CN CN201810005392.9A patent/CN108366293A/en active Pending
- 2018-01-03 CN CN201810005786.4A patent/CN108377381A/en active Pending
- 2018-01-03 CN CN201810005788.3A patent/CN108391103A/en active Pending
- 2018-01-03 CN CN201810004987.2A patent/CN108419093A/en active Pending
- 2018-01-03 CN CN201810005787.9A patent/CN108366272A/en active Pending
- 2018-01-03 CN CN201810005019.3A patent/CN108419142A/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8897370B1 (en) * | 2009-11-30 | 2014-11-25 | Google Inc. | Bitrate video transcoding based on video coding complexity estimation |
US20120254455A1 (en) * | 2011-03-30 | 2012-10-04 | Verizon Patent And Licensing Inc. | Downloading video using excess bandwidth |
US20150208103A1 (en) * | 2012-08-08 | 2015-07-23 | National University Of Singapore | System and Method for Enabling User Control of Live Video Stream(s) |
US20160191585A1 (en) * | 2014-12-24 | 2016-06-30 | Intel Corporation | Link-aware streaming adaptation |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11190820B2 (en) | 2018-06-01 | 2021-11-30 | At&T Intellectual Property I, L.P. | Field of view prediction in live panoramic video streaming |
US11641499B2 (en) | 2018-06-01 | 2023-05-02 | At&T Intellectual Property I, L.P. | Field of view prediction in live panoramic video streaming |
US10812774B2 (en) | 2018-06-06 | 2020-10-20 | At&T Intellectual Property I, L.P. | Methods and devices for adapting the rate of video content streaming |
US10616621B2 (en) * | 2018-06-29 | 2020-04-07 | At&T Intellectual Property I, L.P. | Methods and devices for determining multipath routing for panoramic video content |
US11019361B2 (en) | 2018-08-13 | 2021-05-25 | At&T Intellectual Property I, L.P. | Methods, systems and devices for adjusting panoramic view of a camera for capturing video content |
US11671623B2 (en) | 2018-08-13 | 2023-06-06 | At&T Intellectual Property I, L.P. | Methods, systems and devices for adjusting panoramic view of a camera for capturing video content |
CN109343518A (en) * | 2018-09-03 | 2019-02-15 | 浙江大丰实业股份有限公司 | Universal ride scene drives platform |
US20210329255A1 (en) * | 2018-10-22 | 2021-10-21 | Bitmovin, Inc. | Video Encoding Based on Customized Bitrate Table |
US11563951B2 (en) * | 2018-10-22 | 2023-01-24 | Bitmovin, Inc. | Video encoding based on customized bitrate table |
CN112468806A (en) * | 2020-11-12 | 2021-03-09 | 中山大学 | Panoramic video transmission optimization method for cloud VR platform |
CN114979799A (en) * | 2022-05-20 | 2022-08-30 | 北京字节跳动网络技术有限公司 | Panoramic video processing method, device, equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN108366293A (en) | 2018-08-03 |
US10334238B2 (en) | 2019-06-25 |
CN108366272A (en) | 2018-08-03 |
US20180192063A1 (en) | 2018-07-05 |
US20180189980A1 (en) | 2018-07-05 |
CN108377381A (en) | 2018-08-07 |
CN108391103A (en) | 2018-08-10 |
US20180192044A1 (en) | 2018-07-05 |
US20180192022A1 (en) | 2018-07-05 |
US20180192026A1 (en) | 2018-07-05 |
CN108419142A (en) | 2018-08-17 |
CN108419093A (en) | 2018-08-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20180191868A1 (en) | Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization | |
US11303881B2 (en) | Method and client for playing back panoramic video | |
WO2017193576A1 (en) | Video resolution adaptation method and apparatus, and virtual reality terminal | |
JP6310073B2 (en) | Drawing system, control method, and storage medium | |
CN110996097B (en) | VR multimedia experience quality determination method and device | |
US20050271361A1 (en) | Image frame processing method and device for displaying moving images to a variety of displays | |
US11122245B2 (en) | Display apparatus, method for controlling the same and image providing apparatus | |
CN105430471A (en) | Method and device for displaying live commenting in video | |
US20220382053A1 (en) | Image processing method and apparatus for head-mounted display device as well as electronic device | |
CN111665940B (en) | Cursor control method and device and related equipment | |
CN112511849A (en) | Game display method, device, equipment, system and storage medium | |
US7936936B2 (en) | Method of visualizing a large still picture on a small-size display | |
US20120293638A1 (en) | Apparatus and method for providing 3d content | |
JP7550306B2 (en) | Automatic Head-Up Display Positioning Based on Eye Tracking | |
KR20230104602A (en) | Dynamic user device upscaling of media streams | |
US20240098316A1 (en) | Video encoding method and apparatus, real-time communication method and apparatus, device, and storage medium | |
CN106454417A (en) | Streaming media speed multiplication playing method and apparatus | |
US20200311947A1 (en) | Image processing apparatus, transmission method, and storage medium | |
KR20160015136A (en) | System for cloud streaming service, method of cloud streaming service using still image compression technique and apparatus for the same | |
CN115393490A (en) | Image rendering method and device, storage medium and electronic equipment | |
JP6623905B2 (en) | Server device, information processing method and program | |
US8482574B2 (en) | System, method, and computer program product for calculating statistics associated with a surface to be rendered utilizing a graphics processor | |
EP3635958B1 (en) | Image processing device and system | |
WO2024126043A1 (en) | Mechanism to control the refresh rate of the real-environment computation for augmented reality (ar) experiences | |
TWI829350B (en) | Method and system for enhancing quality of a frame sequence |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BLACK SAILS TECHNOLOGY INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WANG, ZHUO;TANG, YONGTAO;ZHAO, RUOXI;AND OTHERS;REEL/FRAME:044518/0775 Effective date: 20180102 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |