US20180191868A1 - Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization - Google Patents

Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization Download PDF

Info

Publication number
US20180191868A1
US20180191868A1 US15/860,430 US201815860430A US2018191868A1 US 20180191868 A1 US20180191868 A1 US 20180191868A1 US 201815860430 A US201815860430 A US 201815860430A US 2018191868 A1 US2018191868 A1 US 2018191868A1
Authority
US
United States
Prior art keywords
video
bitrate
resolution
resolutions
video segments
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US15/860,430
Other languages
English (en)
Inventor
Zhuo Wang
Yongtao Tang
Ruoxi Zhao
Haoyan Zu
Chia-Chi Chang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Black Sails Technology Inc
Original Assignee
Black Sails Technology Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Black Sails Technology Inc filed Critical Black Sails Technology Inc
Priority to US15/860,430 priority Critical patent/US20180191868A1/en
Assigned to Black Sails Technology Inc. reassignment Black Sails Technology Inc. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHANG, CHIA-CHI, TANG, YONGTAO, WANG, ZHUO, ZHAO, RUOXI, ZU, HAOYAN
Publication of US20180191868A1 publication Critical patent/US20180191868A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H04L67/38—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/398—Synchronisation thereof; Control thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H04L43/0888—Throughput
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/752—Media network packet handling adapting media to network capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/131—Protocols for games, networked simulations or virtual reality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/139—Format conversion, e.g. of frame-rate or size
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/189—Recording image signals; Reproducing recorded image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/378—Image reproducers using viewer tracking for tracking rotational head movements around an axis perpendicular to the screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233—Processing of audio elementary streams
    • H04N21/2335—Processing of audio elementary streams involving reformatting operations of audio signals, e.g. by converting from one coding standard to another
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234345—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2387—Stream processing in response to a playback request from an end-user, e.g. for trick-play
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2405—Monitoring of the internal components or processes of the server, e.g. server load
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H04N9/3185—Geometric adjustment, e.g. keystone or convergence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/87—Regeneration of colour television signals
    • H04N9/8715—Regeneration of colour television signals involving the mixing of the reproduced video signal with a non-recorded signal, e.g. a text signal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2215/00—Indexing scheme for image rendering
    • G06T2215/16—Using real world measurements to influence rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting

Definitions

  • the present disclosure relates to video processing technology, and more particularly, to a method and a system for downloading a VR video.
  • Virtual Reality is a computer simulation technology for creating and experiencing a virtual world. For example, a three-dimensional real-time image can be presented based on a technology which tracks a user's head, eyes or hands.
  • full-view video data can be pre-stored on a server, and then transmitted to a display device.
  • a display device can be glasses, a head-mounted display, etc.
  • a video is displayed on the display device in accordance with a field of view of the user.
  • a high-resolution video data occupies large transmission of bandwidth and requires high computing power from the display devices.
  • Presenting high-resolution VR video on internet is difficult, precisely, the existing video streaming technology cannot fulfill the virtual reality. Therefore, in order to present VR video smoothly in real-time, it is desirable to further improve the existing video streaming technology to save bandwidth and reduce performance requirements for display devices.
  • the present disclosure provides a method and a system for downloading a VR video for optimizing video transmission.
  • a method for downloading a VR video comprising: receiving a play request for a first video; calculating an average download speed; selecting a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates; determining a download link according to the plurality of first video segments; and downloading the plurality of video segments according to the download link.
  • the method further comprises: obtaining a mapping table between resolutions and bitrates during a video transcoding process; and determining the plurality of first video segments by looking up the mapping table.
  • the step of obtaining the mapping table between resolutions and bitrates during the video transcoding process comprises: defining a plurality of resolution levels, and calculating a plurality of resolutions in the plurality of resolution levels according to a predetermined target resolution; and calculating bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
  • the step of defining a plurality of resolution levels, and calculating a plurality of resolutions in the plurality of resolution levels according to a predetermined target resolution comprises: calculating the resolution in one of the plurality of resolution levels following formulas below,
  • n represents the one of the plurality of resolution levels
  • R0 represents a maximum value of the plurality of resolutions
  • ⁇ X 0 enhance Y 0 enhance ⁇ represents the resolution of an enhancement video corresponding to the one of the plurality of resolution levels
  • q represents a resolution ratio
  • R n ⁇ X n , Y n ⁇ represents the resolution in the one of the plurality of resolution levels
  • the step of calculating bitrates corresponding to the plurality of resolutions in the plurality of resolution levels based on a target bitrate of the predetermined target resolution comprises: calculating the bitrates corresponding to the resolutions in the plurality of resolution levels following formulas below,
  • bitrate n bitrate n enhance and bitrate n base , there is:
  • bitrate n + 1 bitrate n q 2 ,
  • bitrate n base > 1 q 2 - 1 ⁇ bitrate n enhance ,
  • bitrate 0 represents a maximum value of the bitrates
  • bitrate n represents a maximum value of the bitrates
  • bitrate n base represents bitrate n + bitrate n enhance respectively represent the bitrate, the bitrate of the basic video, and the bitrate of the enhancement video corresponding to the one of the plurality of resolution levels, wherein n is an integer equal to or larger than 1.
  • the play request comprises a field of view of a user
  • the plurality of first video segments have a plurality of resolutions
  • a resolution distribution of the plurality of first video segments is determined by the field of view of the user.
  • the resolution of a first video segment within the field of view is higher than the resolution of a first video segment outside the field of view.
  • the average download speed and the resolution distribution of the plurality of first video segments are positively correlated.
  • a system for downloading a VR video comprising: a request receiving module configured to receive a play request of a first video; a speed calculating module configured to calculate an average download speed; a segment obtaining module configured to select a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates; a link obtaining module configured to determine a download link according to the plurality of first video segments; and a download executing module configured to download the plurality of video segments according to the download link.
  • the system further comprises: a mapping table obtaining module configured to obtaining a mapping table between resolutions and bitrates during a video transcoding process, and to determine the plurality of first video segments by looking up the mapping table.
  • a mapping table obtaining module configured to obtaining a mapping table between resolutions and bitrates during a video transcoding process, and to determine the plurality of first video segments by looking up the mapping table.
  • the mapping table obtaining module comprises: a first calculating unit configured to define a plurality of resolution levels and calculate a plurality of resolutions in the plurality of resolution levels according to the predetermined target resolution; and calculating bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
  • the present disclosure provides a plurality of video segments with proper resolutions and proper bitrates for downloading according to the average download speed.
  • a resolution reflects the image precision
  • a bitrate reflects bit number of data transmitted per unit time
  • the resolution, the bitrate and average download speed can be synthesized to optimize the video transmission by using an optimal strategy to select the appropriate video segments.
  • the video segments with high resolutions and low resolutions are selected according to the user's field of view, which can decrease the amount of transferred data without affecting the user's demands for viewing the video.
  • FIG. 1 is a diagram illustrating an example network of a VR playback system
  • FIG. 2 is a flowchart showing a method used in the VR playback system of FIG. 1 ;
  • FIG. 3 is a flowchart presenting the method for downloading a VR video according to an embodiment of the present disclosure
  • FIG. 4 is a flowchart presenting a detailed methodology of obtaining the mapping table between resolutions and bitrates during the video transcoding process.
  • FIG. 5 is a structural diagram presenting a system for downloading a VR video according to an embodiment of the present disclosure.
  • FIG. 1 is a diagram illustrating an example network of a VR playback system.
  • the VR playback system 10 includes a server 100 and a display device 120 which are coupled with each other through a network 110 , and a VR device.
  • the server 100 may be a stand-alone computer server or a server cluster.
  • the server 100 is used to store various video data and to store various applications that process these video data.
  • various daemons run on the server 100 in real time, so as to process various video data in the server 100 and to respond various requests from VR devices and the display device 120 .
  • the network 110 may be a selected one or selected ones from the group consisting of an internet, a local area network, an internet of things, and the like.
  • the display device 120 may be any of the computing devices, including a computer device having an independent display screen and a processing capability.
  • the display device 120 may be a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a palmtop computer, a personal digital assistant, a smart phone, an intelligent electrical apparatus, a game console, an iPad/iPhone, a video player, a DVD recorder/player, a television, or a home entertainment system.
  • the display device 120 may store VR player software as a VR player. When the VR player is started, it requests and downloads various video data from the server 100 , and renders and plays the video data in the display device.
  • the VR device 130 is a stand-alone head-mounted device that can interact with the display device 120 and the server 100 , to communicate the user's current information to the display device 120 and/or the server 100 through signaling.
  • the user's current information is, for example, a field of view of the user, a position modification of a helmet, a modification of the user's sight orientation, etc.
  • the display device 120 can flexibly process the currently played video data.
  • the display device 120 determines that a core viewing region for the user has been changed and starts to play video data with high resolution in the changed core viewing region.
  • the VR device 130 is a stand-alone head-mounted device.
  • the VR device 130 is not limited thereto, and the VR device 130 may also be an all-in-one head-mounted device.
  • the all-in-one head-mounted device itself has a display screen, so that it is not necessary to connect the all-in-one head-mounted device with the external display device.
  • the display device 120 may be eliminated.
  • the all-in-one head-mounted device is configured to obtain video data from the server 100 and to perform playback operation, and the all-in-one head-mounted device is also configured to detect user's current field of view modification information and to adjust the playback operation according to the field of view modification information.
  • FIG. 2 is a flowchart showing a method used in the VR playback system of FIG. 1 . The method comprises following steps.
  • step S 10 a source video data process is operated on server side.
  • step S 20 the display device obtains some related information of the user based on interactions with the VR device.
  • step S 30 the display device requests video data and receives video data from the server according to the related information.
  • step S 40 the display device side performs a rendering process based on the received video data.
  • the video data is downloaded. Furthermore, the present disclosure provides a method for downloading a VR video. The method downloads the VR video by predicting a network condition according to a calculated average download speed and providing a download link according to bitrates, in order to optimize VR video streaming transmission.
  • FIG. 3 is a flowchart presenting the method for downloading a VR video according to an embodiment of the present disclosure. Specifically, following steps are comprised.
  • step S 100 a play request for a first video is received.
  • the first video in this step refers to a specific video.
  • step S 200 an average download speed is calculated.
  • the average download speed is calculated by making statistics on an amount of downloaded data and download time.
  • the average download speed is calculated once in real time. For example, the average download speed is calculated once before data of the first video starts to be downloaded, or the average download speed is calculated once before each video segments of the first video starts to be downloaded.
  • step S 300 a plurality of first video segments are selected from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates.
  • a complete source VR video can be processed into the plurality of video segments, for example, the plurality of video segments may be obtained by segmenting the source VR video in temporal domain and/or in spatial domain.
  • the plurality of video segments have different resolutions and bitrates.
  • the plurality of first video segments can be selected from the plurality of video segments with different resolutions and bitrates. Therefore, in this step, each of the plurality of first video segments is the video segment with a proper resolution and a proper bitrate selected from the plurality of video segments, according to the average download speed.
  • the video segments with high resolutions and high bitrates are selected as the plurality of first video segments
  • the video segments with low resolutions and low bitrates are selected as the plurality of first video segments.
  • step S 400 a download link is determined according to the plurality of first video segments.
  • the download link of the first video is determined according to a predetermined download strategy, and the plurality of first video segments waiting to be downloaded and a download order are specified by the download link.
  • the download order for downloading the first video segments, which need to be downloaded is an ascending order of the bitrates.
  • the present disclosure is not limited thereto. Those skilled in the art can determine the download strategy according to the bitrates and the average download speed, in order to optimize video transmission.
  • step S 500 the plurality of first video segments are downloaded according to the download link.
  • download processes are operated on the plurality of first video segments.
  • the video segments which need to be downloaded and are selected according to the average speed, are downloaded, so as to optimize video transmission.
  • the method for downloading the VR video further comprises: obtaining a mapping table between resolutions and bitrates, and determining the plurality of first video segments by looking up the mapping table.
  • the mapping table defines correspondences between a plurality of resolutions and a plurality of bitrates.
  • the mapping table is looked up to obtain the set bitrates of the plurality of video segments, so that the plurality of first video segments can be determined according to the average download speed and the set bitrates.
  • the mapping table is generated during a video transcoding process.
  • raw video data is processed into the plurality of video segments, for example, the raw video data can be processed into a basic video and an enhancement video which need to be overlaid together during playback.
  • the bitrate of each of the plurality of video segments are recorded and are summarized as the mapping table between resolutions and bitrates.
  • information provided by the play request comprises a field of view of the user
  • the plurality of first video segments have a plurality of resolutions
  • a resolution distribution of the plurality of first video segments is determined by the field of view of the user. For example, it is assumed that a video with 360-degree panoramic images is provided, the user's field of view ranges in 135 degrees, then the video segments with high resolutions are displayed in a screen region inside the user's field of view, and the video segments with low resolutions are displayed on the screen regions outside the user's field of view. Therefore, by decreasing times for downloading the high-resolution video segments, the amount of downloaded data can be reduced and the video transmission can be optimized.
  • the user's field of view can be obtained according to parameters of a gyroscope in the head-mounted device.
  • FIG. 4 is a flowchart presenting a detailed methodology of obtaining the mapping table between resolutions and bitrates during the video transcoding process.
  • step S 1001 a plurality of resolution levels are defined, and a plurality of resolutions in the plurality of resolution levels are calculated according to a predetermined target resolution.
  • step S 1002 bitrates corresponding to the resolutions in the plurality of resolution levels are calculated based on a target bitrate corresponding to the predetermined target resolution.
  • step S 1003 the video transcoding process is performed according to the corresponding resolutions and the corresponding bitrates, in order to obtain a plurality of video segments of video data with different resolutions and different bitrates.
  • N represents a total number of the plurality of resolution levels
  • R 0 represents the predetermined target resolution
  • bitrate 0 represents the target bitrate according to the target resolution.
  • the resolution R 1 in the 1 st resolution level, the resolution R 2 in the 2 nd resolution level, . . . , and the resolution R N ⁇ 1 in the (N ⁇ 1) th resolution level can be calculated according to a resolution ratio between R n and R n+1 , so as to obtain the plurality of resolutions in all of the plurality of resolution levels.
  • All of the bitrates can be obtained according to resolution ratios between bitrate n and bitrate n+1 , so that the mapping table between resolutions and bitrates is obtained.
  • the plurality of video segments can be obtained by transcoding based on the corresponding resolutions and the corresponding bitrates.
  • mapping table between resolutions and bitrates is not limited thereto.
  • the mapping table can be obtained by other manners, for example, the mapping table between resolutions and bitrates may be set by experience.
  • n one of the plurality of resolution level, wherein a lower n corresponding to a higher resolution, n ⁇ [0, 1, 2, . . . ] R n the resolution in the n th resolution level, a two-element array ( X n , Y n ) bitrate n the bitrate corresponding to the n th resolution level bitrate n base the bitrate of the basic video corresponding to the n th resolution level bitrate n enhance the bitrate of the enhancement video corresponding to the n th resolution level q a resolution ratio (which is, for example, 2n)
  • a maximum resolution is defined as R 0 , bitrate 0 , X 0 base , Y 0 base , X 0 enhance , Y 0 enhance , and q are given by user, and also
  • q X 0 enhance X 1 enhance > 1 ( 2 )
  • bitrate corresponding to the n th resolution level satisfy following formulas:
  • bitrate n For each one of bitrate n , bitrate n enhance , and bitrate n base there is:
  • bitrate n + 1 bitrate n q 2 ( 7 )
  • bitrate n base >bitrate n+1 base +bitrate n+1 enhance (8)
  • bitrate n base > 1 q 2 - 1 ⁇ bitrate n enhance ( 9 )
  • mapping table can be generated when X 0 base , Y 0 base , X 0 enhance , Y 0 enhance and q are provided by the user and satisfy an inequality:
  • FIG. 5 is a structural diagram presenting a system for downloading a VR video according to an embodiment of the present disclosure.
  • the system 500 for downloading a VR video comprises a request receiving module 501 , a speed calculation module 502 , a segment obtaining module 503 , a link obtaining module 504 and a download executing module 505 .
  • the request receiving module 501 is configured to receive a play request for a first video.
  • the play request is sent from a display device or a VR device.
  • Information provided by the play request includes a name of the current playing video and a current field of view of a user.
  • the first video here refers to a specific video.
  • the speed calculating module 502 is configured to calculate an average download speed.
  • the average download speed is calculated by making statistics on the amount of downloaded and download time.
  • the segment obtaining module 503 is configured to select a plurality of first video segments from a plurality of video segments of the first video according to the average download speed, wherein the plurality of video segments of the first video correspond to different resolutions and bitrates.
  • the plurality of video segments of the first video is obtained during a video transcoding process and have different resolutions and bitrates.
  • the segment obtaining module 503 is configured to obtain the video segments for downloading selected from the plurality of video segments for downloading according to the current average download speed.
  • a download strategy is set to select the video segments with low bitrates and low resolutions for downloading when the average download speed is low, and to select the video segments with high bitrates and high resolutions for downloading when the average download speed is relatively high.
  • the link obtaining module 504 is configured to determine a download link according to the plurality of first video segments. Specifically, the link obtaining module 504 is configured to determine a download manner and a download order according to the predetermined download strategy, for example, the link obtaining module 504 downloads the video segments with high resolutions first, and then downloads the video segments with low resolutions.
  • the download executing module 505 is configured to download the plurality of video segments according to the download link. That is, the download executing module performs a download operation according to the download link provided by the link obtaining module 505 .
  • the above-described system for downloading the VR video further comprises: a mapping table obtaining module which is configured to generate a mapping table between resolutions and bitrates during the process when the plurality of video segments are obtained according to the video data.
  • the mapping table obtaining module comprises: a first calculating unit which is configured to define a plurality of resolution levels, and calculate resolutions in the plurality of resolution levels according to a predetermined target resolution; and a second calculating unit which is configured to calculate bitrates corresponding to the resolutions in the plurality of resolution levels based on a target bitrate corresponding to the predetermined target resolution.
  • the play request comprises a field of view of a user
  • the plurality of first video segments have a plurality of resolutions
  • a resolution distribution of the plurality of first video segments is determined by the field of view of the user.
  • the video segments, which need to be downloaded are determined according to the average download speed, so that the amount of downloaded data can be decreased and the video transmission can be optimized. It should be pointed out that the video data needs to be continuously downloaded during VR video playback, therefore, the process for selecting the video segments with proper resolutions and bitrates for downloading can be continuously performed according to the current average download speed, that is, the video transmission optimization is achieved by dynamic selection processes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Graphics (AREA)
  • Human Computer Interaction (AREA)
  • Geometry (AREA)
  • Environmental & Geological Engineering (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Image Generation (AREA)
  • Processing Or Creating Images (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
US15/860,430 2017-01-03 2018-01-02 Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization Abandoned US20180191868A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/860,430 US20180191868A1 (en) 2017-01-03 2018-01-02 Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762441936P 2017-01-03 2017-01-03
US15/860,430 US20180191868A1 (en) 2017-01-03 2018-01-02 Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization

Publications (1)

Publication Number Publication Date
US20180191868A1 true US20180191868A1 (en) 2018-07-05

Family

ID=62711388

Family Applications (6)

Application Number Title Priority Date Filing Date
US15/860,449 Expired - Fee Related US10334238B2 (en) 2017-01-03 2018-01-02 Method and system for real-time rendering displaying high resolution virtual reality (VR) video
US15/860,392 Abandoned US20180192044A1 (en) 2017-01-03 2018-01-02 Method and System for Providing A Viewport Division Scheme for Virtual Reality (VR) Video Streaming
US15/860,494 Abandoned US20180189980A1 (en) 2017-01-03 2018-01-02 Method and System for Providing Virtual Reality (VR) Video Transcoding and Broadcasting
US15/860,358 Abandoned US20180192063A1 (en) 2017-01-03 2018-01-02 Method and System for Virtual Reality (VR) Video Transcode By Extracting Residual From Different Resolutions
US15/860,471 Abandoned US20180192022A1 (en) 2017-01-03 2018-01-02 Method and System for Real-time Rendering Displaying Virtual Reality (VR) On Mobile Using Head-Up Display Devices
US15/860,430 Abandoned US20180191868A1 (en) 2017-01-03 2018-01-02 Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization

Family Applications Before (5)

Application Number Title Priority Date Filing Date
US15/860,449 Expired - Fee Related US10334238B2 (en) 2017-01-03 2018-01-02 Method and system for real-time rendering displaying high resolution virtual reality (VR) video
US15/860,392 Abandoned US20180192044A1 (en) 2017-01-03 2018-01-02 Method and System for Providing A Viewport Division Scheme for Virtual Reality (VR) Video Streaming
US15/860,494 Abandoned US20180189980A1 (en) 2017-01-03 2018-01-02 Method and System for Providing Virtual Reality (VR) Video Transcoding and Broadcasting
US15/860,358 Abandoned US20180192063A1 (en) 2017-01-03 2018-01-02 Method and System for Virtual Reality (VR) Video Transcode By Extracting Residual From Different Resolutions
US15/860,471 Abandoned US20180192022A1 (en) 2017-01-03 2018-01-02 Method and System for Real-time Rendering Displaying Virtual Reality (VR) On Mobile Using Head-Up Display Devices

Country Status (2)

Country Link
US (6) US10334238B2 (zh)
CN (6) CN108419142A (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109343518A (zh) * 2018-09-03 2019-02-15 浙江大丰实业股份有限公司 万向车台现场驱动平台
US10616621B2 (en) * 2018-06-29 2020-04-07 At&T Intellectual Property I, L.P. Methods and devices for determining multipath routing for panoramic video content
US10812774B2 (en) 2018-06-06 2020-10-20 At&T Intellectual Property I, L.P. Methods and devices for adapting the rate of video content streaming
CN112468806A (zh) * 2020-11-12 2021-03-09 中山大学 一种用于云vr平台的全景视频传输优化方法
US11019361B2 (en) 2018-08-13 2021-05-25 At&T Intellectual Property I, L.P. Methods, systems and devices for adjusting panoramic view of a camera for capturing video content
US20210329255A1 (en) * 2018-10-22 2021-10-21 Bitmovin, Inc. Video Encoding Based on Customized Bitrate Table
US11190820B2 (en) 2018-06-01 2021-11-30 At&T Intellectual Property I, L.P. Field of view prediction in live panoramic video streaming

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10291910B2 (en) * 2016-02-12 2019-05-14 Gopro, Inc. Systems and methods for spatially adaptive video encoding
US10331862B2 (en) * 2017-04-20 2019-06-25 Cisco Technology, Inc. Viewport decryption
CN115842907A (zh) * 2018-03-27 2023-03-24 京东方科技集团股份有限公司 渲染方法、计算机产品及显示装置
US11232532B2 (en) * 2018-05-30 2022-01-25 Sony Interactive Entertainment LLC Multi-server cloud virtual reality (VR) streaming
CN109375369B (zh) * 2018-11-23 2021-05-18 国网天津市电力公司 一种vr巨幕影院模式下的畸变预处理方法
CN111510777B (zh) * 2019-01-30 2021-11-23 上海哔哩哔哩科技有限公司 测量网速的方法、装置、计算机设备及可读存储介质
CN111669666A (zh) * 2019-03-08 2020-09-15 北京京东尚科信息技术有限公司 模拟现实的方法、装置和系统
CN111866485A (zh) * 2019-04-25 2020-10-30 中国移动通信有限公司研究院 立体画面投影和传输方法、装置和计算机可读存储介质
CN110336994B (zh) * 2019-07-04 2021-06-22 上海索倍信息科技有限公司 一种裸眼3d显示系统
CN110381331A (zh) * 2019-07-23 2019-10-25 深圳市道通智能航空技术有限公司 一种图像处理方法、装置、航拍设备及存储介质
CN110490962B (zh) * 2019-08-20 2023-09-15 武汉邦拓信息科技有限公司 一种基于视频流的远程渲染方法
CN110544425A (zh) * 2019-09-13 2019-12-06 广州城市职业学院 一种古建筑vr展示系统
CN111489428B (zh) * 2020-04-20 2023-06-30 北京字节跳动网络技术有限公司 图像生成方法、装置、电子设备及计算机可读存储介质
US11245911B1 (en) * 2020-05-12 2022-02-08 Whirlwind 3D, LLC Video encoder/decoder (codec) for real-time applications and size/b and width reduction
CN111754614A (zh) * 2020-06-30 2020-10-09 平安国际智慧城市科技股份有限公司 基于vr视频渲染方法、装置、电子设备及存储介质
CN114286142B (zh) * 2021-01-18 2023-03-28 海信视像科技股份有限公司 一种虚拟现实设备及vr场景截屏方法
CN113347402A (zh) * 2021-06-28 2021-09-03 筑友建筑装饰装修工程有限公司 基于Unity的渲染沉浸式内容的改良方法、装置和存储介质
CN114466220A (zh) * 2022-01-29 2022-05-10 维沃移动通信有限公司 视频下载方法和电子设备
CN115396731A (zh) * 2022-08-10 2022-11-25 北京势也网络技术有限公司 全景视频的播放方法、装置、电子设备及可读存储介质
CN116880723B (zh) * 2023-09-08 2023-11-17 江西格如灵科技股份有限公司 一种3d场景显示方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120254455A1 (en) * 2011-03-30 2012-10-04 Verizon Patent And Licensing Inc. Downloading video using excess bandwidth
US8897370B1 (en) * 2009-11-30 2014-11-25 Google Inc. Bitrate video transcoding based on video coding complexity estimation
US20150208103A1 (en) * 2012-08-08 2015-07-23 National University Of Singapore System and Method for Enabling User Control of Live Video Stream(s)
US20160191585A1 (en) * 2014-12-24 2016-06-30 Intel Corporation Link-aware streaming adaptation

Family Cites Families (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3196889B2 (ja) * 1996-09-05 2001-08-06 株式会社アルファ 三次元画像処理方法および三次元画像処理方法をコンピュータに実行させるプログラムを記録したコンピュータ読み取り可能な記録媒体
TWI262725B (en) * 2005-06-30 2006-09-21 Cheertek Inc Video decoding apparatus and digital audio and video display system capable of controlling presentation of subtitles and method thereof
JP5891426B2 (ja) * 2011-03-31 2016-03-23 パナソニックＩｐマネジメント株式会社 全周囲立体画像の描画を行う画像描画装置、画像描画方法、画像描画プログラム
US8810598B2 (en) * 2011-04-08 2014-08-19 Nant Holdings Ip, Llc Interference based augmented reality hosting platforms
GB2501929B (en) * 2012-05-11 2015-06-24 Sony Comp Entertainment Europe Apparatus and method for augmented reality
US9129429B2 (en) * 2012-10-24 2015-09-08 Exelis, Inc. Augmented reality on wireless mobile devices
GB2509953B (en) * 2013-01-18 2015-05-20 Canon Kk Method of displaying a region of interest in a video stream
US9196199B2 (en) * 2013-02-12 2015-11-24 Pixtronix, Inc. Display having staggered display element arrangement
CN103702139B (zh) * 2013-12-13 2017-02-01 华中科技大学 一种移动环境下基于可扩展编码的视频点播系统
US9398250B2 (en) * 2014-01-06 2016-07-19 Arun Sobti & Associates, Llc System and apparatus for smart devices based conferencing
CN105025351B (zh) * 2014-04-30 2018-06-29 深圳Tcl新技术有限公司 流媒体播放器缓冲的方法及装置
JP6337614B2 (ja) * 2014-05-23 2018-06-06 セイコーエプソン株式会社 制御装置、ロボット、及び制御方法
US9918136B2 (en) * 2014-05-29 2018-03-13 Nextvr Inc. Methods and apparatus for delivering content and/or playing back content
CN104240281A (zh) * 2014-08-28 2014-12-24 东华大学 一种基于Unity3D引擎的虚拟现实头戴式设备
CN104268922B (zh) * 2014-09-03 2017-06-06 广州博冠信息科技有限公司 一种图像渲染方法及图像渲染装置
CN104616243B (zh) * 2015-01-20 2018-02-27 北京道和汇通科技发展有限公司 一种高效的gpu三维视频融合绘制方法
US20160261908A1 (en) * 2015-03-05 2016-09-08 Htc Corporation Media streaming system and control method thereof
CN104735464A (zh) * 2015-03-31 2015-06-24 华为技术有限公司 一种全景视频交互传输方法、服务器和客户端
CN104717507A (zh) * 2015-03-31 2015-06-17 北京奇艺世纪科技有限公司 一种视频转码方法及装置
US10083363B2 (en) * 2015-05-26 2018-09-25 Nbcuniversal Media, Llc System and method for customizing content for a user
JP6866297B2 (ja) * 2015-06-12 2021-04-28 グーグル エルエルシーＧｏｏｇｌｅ ＬＬＣ ヘッドマウントディスプレイの電子ディスプレイ安定化
US9956054B2 (en) * 2015-06-25 2018-05-01 EchoPixel, Inc. Dynamic minimally invasive surgical-aware assistant
CN108476324B (zh) * 2015-10-08 2021-10-29 皇家Kpn公司 增强视频流的视频帧中的感兴趣区域的方法、计算机和介质
CN106919248A (zh) * 2015-12-26 2017-07-04 华为技术有限公司 应用于虚拟现实的内容传输方法以及设备
CN105916022A (zh) * 2015-12-28 2016-08-31 乐视致新电子科技（天津）有限公司 一种基于虚拟现实技术的视频图像处理方法及装置
CN105455285B (zh) * 2015-12-31 2019-02-12 北京小鸟看看科技有限公司 一种虚拟现实头盔适配方法
US10412130B2 (en) * 2016-04-04 2019-09-10 Hanwha Techwin Co., Ltd. Method and apparatus for playing media stream on web browser
US10313417B2 (en) * 2016-04-18 2019-06-04 Qualcomm Incorporated Methods and systems for auto-zoom based adaptive video streaming
CN105898565A (zh) * 2016-04-28 2016-08-24 乐视控股（北京）有限公司 一种视频处理方法及设备
US9721393B1 (en) * 2016-04-29 2017-08-01 Immersive Enterprises, LLC Method for processing and delivering virtual reality content to a user
CN106060570B (zh) * 2016-06-30 2019-06-14 北京奇艺世纪科技有限公司 一种全景视频图像播放、编码方法及装置
CN106060515B (zh) * 2016-07-14 2018-11-06 腾讯科技（深圳）有限公司 全景媒体文件推送方法及装置
CN106231317A (zh) * 2016-09-29 2016-12-14 三星电子（中国）研发中心 视频处理、解码方法和装置、vr终端、视频播放系统
US10595069B2 (en) * 2016-12-05 2020-03-17 Adobe Inc. Prioritizing tile-based virtual reality video streaming using adaptive rate allocation
US20180295375A1 (en) * 2017-04-05 2018-10-11 Lyrical Labs Video Compression Technology, LLC Video processing and encoding
CN107087212B (zh) * 2017-05-09 2019-10-29 杭州码全信息科技有限公司 基于空间可伸缩编码的交互式全景视频转码与播放方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8897370B1 (en) * 2009-11-30 2014-11-25 Google Inc. Bitrate video transcoding based on video coding complexity estimation
US20120254455A1 (en) * 2011-03-30 2012-10-04 Verizon Patent And Licensing Inc. Downloading video using excess bandwidth
US20150208103A1 (en) * 2012-08-08 2015-07-23 National University Of Singapore System and Method for Enabling User Control of Live Video Stream(s)
US20160191585A1 (en) * 2014-12-24 2016-06-30 Intel Corporation Link-aware streaming adaptation

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11190820B2 (en) 2018-06-01 2021-11-30 At&T Intellectual Property I, L.P. Field of view prediction in live panoramic video streaming
US11641499B2 (en) 2018-06-01 2023-05-02 At&T Intellectual Property I, L.P. Field of view prediction in live panoramic video streaming
US10812774B2 (en) 2018-06-06 2020-10-20 At&T Intellectual Property I, L.P. Methods and devices for adapting the rate of video content streaming
US10616621B2 (en) * 2018-06-29 2020-04-07 At&T Intellectual Property I, L.P. Methods and devices for determining multipath routing for panoramic video content
US11019361B2 (en) 2018-08-13 2021-05-25 At&T Intellectual Property I, L.P. Methods, systems and devices for adjusting panoramic view of a camera for capturing video content
US11671623B2 (en) 2018-08-13 2023-06-06 At&T Intellectual Property I, L.P. Methods, systems and devices for adjusting panoramic view of a camera for capturing video content
CN109343518A (zh) * 2018-09-03 2019-02-15 浙江大丰实业股份有限公司 万向车台现场驱动平台
US20210329255A1 (en) * 2018-10-22 2021-10-21 Bitmovin, Inc. Video Encoding Based on Customized Bitrate Table
US11563951B2 (en) * 2018-10-22 2023-01-24 Bitmovin, Inc. Video encoding based on customized bitrate table
CN112468806A (zh) * 2020-11-12 2021-03-09 中山大学 一种用于云vr平台的全景视频传输优化方法

Also Published As

Publication number Publication date
US20180192063A1 (en) 2018-07-05
US20180192022A1 (en) 2018-07-05
US20180192044A1 (en) 2018-07-05
CN108377381A (zh) 2018-08-07
US20180192026A1 (en) 2018-07-05
CN108419093A (zh) 2018-08-17
CN108419142A (zh) 2018-08-17
CN108366293A (zh) 2018-08-03
US20180189980A1 (en) 2018-07-05
CN108366272A (zh) 2018-08-03
US10334238B2 (en) 2019-06-25
CN108391103A (zh) 2018-08-10

Similar Documents

Publication Publication Date Title
US20180191868A1 (en) Method and System for Downloading Multiple Resolutions Bitrate for Virtual Reality (VR) Video Streaming Optimization
US11303881B2 (en) Method and client for playing back panoramic video
JP6310073B2 (ja) 描画システム、制御方法、及び記憶媒体
JP2019216404A (ja) ビデオを記録および再生するシステムおよび方法
CN110996097B (zh) Vr多媒体的体验质量确定方法及装置
US20050271361A1 (en) Image frame processing method and device for displaying moving images to a variety of displays
US20220382053A1 (en) Image processing method and apparatus for head-mounted display device as well as electronic device
CN111665940B (zh) 光标控件控制方法、装置以及相关设备
CN112511849A (zh) 一种游戏显示方法、装置、设备、系统及存储介质
CN105430471A (zh) 一种视频中弹幕的显示方法及装置
US20120293638A1 (en) Apparatus and method for providing 3d content
KR20230104602A (ko) 미디어 스트림들의 동적 사용자 디바이스 업스케일링
US20240098316A1 (en) Video encoding method and apparatus, real-time communication method and apparatus, device, and storage medium
CN106454417A (zh) 流媒体倍速播放方法及装置
US7936936B2 (en) Method of visualizing a large still picture on a small-size display
US11295453B2 (en) Image processing apparatus, transmission method, and storage medium
JP6623905B2 (ja) サーバ装置、情報処理方法およびプログラム
US8482574B2 (en) System, method, and computer program product for calculating statistics associated with a surface to be rendered utilizing a graphics processor
EP3635958B1 (en) Image processing device and system
WO2024126043A1 (en) Mechanism to control the refresh rate of the real-environment computation for augmented reality (ar) experiences
TWI829350B (zh) 增強幀序列的質量的方法及系統
CN116916028A (zh) 使用虚拟分辨率调整的视频流缩放
CN117392244A (zh) 图像处理方法、装置、电子设备和存储介质
CN115393490A (zh) 图像渲染方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
AS Assignment

Owner name: BLACK SAILS TECHNOLOGY INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WANG, ZHUO;TANG, YONGTAO;ZHAO, RUOXI;AND OTHERS;REEL/FRAME:044518/0775

Effective date: 20180102

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION