CN109302602A - A kind of adaptive VR radio transmitting method based on viewing point prediction - Google Patents
A kind of adaptive VR radio transmitting method based on viewing point prediction Download PDFInfo
- Publication number
- CN109302602A CN109302602A CN201811181333.3A CN201811181333A CN109302602A CN 109302602 A CN109302602 A CN 109302602A CN 201811181333 A CN201811181333 A CN 201811181333A CN 109302602 A CN109302602 A CN 109302602A
- Authority
- CN
- China
- Prior art keywords
- viewing point
- prediction
- adaptive
- head
- radio transmitting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
Abstract
The invention discloses a kind of adaptive VR radio transmitting methods based on viewing point prediction, including the use of the aobvious viewing point coordinate position combination Kalman filtering algorithm provided of head, predict 0.5~1 second later viewing point position, thus in advance, in a planned way buffered video content.The present invention solves caused change dramatically bandwidth demand when the shaking of traditional adaptive algorithm head, realizes and saves bandwidth, the VR video wireless transmission demand of high stability.Compared to other course control method for use, Kalman filter is reliable and stable, and operation is succinctly quick, is able to satisfy real-time requirement, is particularly suitable for VR and transmits this feature of long time continuous working.
Description
Technical field
It is related to technical field of information transmission, especially a kind of adaptive virtual reality radio transmitting method.
Background technique
VR technical application is in the ascendant at present, and in order to make user obtain good experience, it is necessary that high-definition image, which is shown,
Condition.VR head-mounted display apparatus generally needs to be carried out data transmission with cable.This greatly influences user experience, restricts technology
Further development.Therefore, the demand how solved at present to VR wireless transmission is very urgent.
Traditional VR transmission is based on equivalent rectangular projection algorithm or cube (hexahedron) algorithm.Virtual reality
Head-mounted display apparatus (head is aobvious) receives all video informations, for 4K60 frame, the wireless transmission of 8K60 frame or even 4K120 frame
For data volume it is too big.Therefore in order to solve this contradiction, there are two types of paths: taking the compression of more height ratio, or reduces institute
The transmission quantity needed.
Since the visual field of people is limited, image only has in the certain angle of front and can be viewed by a user, the visual field
The content of edge and behind can not be actually noted (as shown in Figure 1).The information that can choose low resolution is passed
It is defeated to save data traffic.This resolving ideas is as follows, the effective field of view of user generally with FoV (Field of View) come into
Row description.Image within the scope of FoV uses high level rate respectively, and edge or even behind are added with the resolution ratio of several low levels
With transmission.Here it is adaptive VR transmission algorithms.
Consistency between the piecemeal image quality in the visual field for adaptively referring to user and selection transmission here.It utilizes
Overall picture block transmission mode can choose the image quality of disparate modules, Adaptive Transmission be realized, such as Fig. 2 and Fig. 3 institute
Show.
The streaming media transmission protocol that this different pictures quality obtains is MPEG-DASH or HLS, and coding mode can be
H.264 or H.265.By Adaptive Transmission algorithm, under stationary conditions, preferable laser propagation effect can achieve.
But this mode still remains many problems, most obvious one is exactly, the head rotation of user to bandwidth still
High demand is proposed, for example the scheme based on user's form and viewing point piecemeal is respectively necessary for the bandwidth of original 5 times, 3 times
Demand, as shown in Figure 4.Its reason is that adaptive algorithm can all abandon buffered content, and again from server
The content of New Century Planned Textbook is requested and caches, this will lead to needs and repeats caching partial content, and different degrees of view angle switch occurs
Delay.This results in the unstability of transmission of video to be increased sharply, and influences picture real-time.
Summary of the invention
It is asked to solve the instable technology that adaptive wireless transmission algorithm is generated when user's head shakes violent
Topic, promotes visual effect, and the present invention proposes the adaptive VR based on viewing point prediction aiming at the problems existing in the prior art
Radio transmitting method, specific technical solution are a kind of adaptive VR radio transmitting method based on viewing point prediction, feature
It is, comprising:
Initialization system obtains head and shows sensing data content；
After aobvious motion state revert to true value, viewing point prediction is carried out；
Viewing point drop point according to prediction loads the image data near viewing point in advance, and real-time measurement head shows viewing point
Mobile speed and direction.
Further, the initialization system includes that read head shows sensor content, obtains initial viewing point and to appearance
The angular speed assignment at state angle.
Further, the attitude angle includes pitch angle, yaw angle and roll angle.
Further, the viewing point prediction includes by utilizing the aobvious viewing point coordinate position combination Kalman provided of head
Filtering algorithm predicts 0.5~1 second later viewing point position.
Further, it is attached to load prediction viewing point position with higher resolution ratio in advance for the viewing point drop point according to prediction
Close image data.
Further, further includes:
Aobvious viewing point combines measured value when being moved to next viewing point position, by the image data near viewing point with
The load of highest picture quality.
Further, further includes:
Judge that head shows velocity of rotation in conjunction with measured value, if it is more than limit value that head, which shows velocity of rotation, system is with lower resolution
Loaded and displayed picture.
Compared with prior art, the core content of the method for the present invention is combined using the aobvious viewing point coordinate position provided of head
Kalman filtering algorithm predicts 0.5~1 second later viewing point position, thus in advance, in a planned way buffered video content.From
And the present invention solves caused change dramatically bandwidth demand and adaptive wireless transmission when the shaking of traditional adaptive algorithm head
Unstability that algorithm is generated when user's head shakes violent promotes visual effect, realizes and saves bandwidth, high stability
VR transmission of video demand.Compared to other course control method for use, the present invention is reliable and stable using Kalman filter, and operation is succinctly fast
Speed is able to satisfy real-time requirement, is particularly suitable for VR and transmits this feature of long time continuous working.
Detailed description of the invention
Fig. 1 is VR user visual field schematic diagram；
Fig. 2 is VR aobvious picture partitioned mode schematic diagram；
Fig. 3 is VR aobvious image quality selection mode schematic diagram；
Fig. 4 is that the aobvious head rotation of head leads to bandwidth demand data profile jumpy；
Fig. 5 is that head shows attitude angle schematic diagram；
Fig. 6 is video cache logical schematic of the present invention；
Fig. 7 is that viewing point of the present invention predicts VR Adaptive Transmission flow diagram.
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiments of the present invention will be described in further detail.Implement below
Example is not intended to limit the scope of the invention for illustrating the present invention.
Embodiment 1:
As shown in figure 5, commercial head is aobvious generally to provide one group of attitude angle to describe viewing point position/orientation, it is pitching respectively
Angle, yaw angle and roll angle (pitch/yaw/roll).Viewing point neighboring area needs to put into most resources to load.Initially
Moment, the aobvious initial value for providing attitude angle of head,And provide attitude angular velocity and angular acceleration initial value with initializing
(in fact can be set as 0 with initializing, can soon revert to true value in engineering).
For tiMoment, it is assumed that knowThe Kalman filtering algorithm of expansion can be passed through
(Extended Kalman Filter) is to ti+1The viewing point state at moment carries out one-step prediction.Obtain
The method of discretization can be set into time interval 200ms and carry out once-through operation, and with the prediction knot at this moment
Fruit loads the surrounding pictures of target direction in advance, plays the role of reinforcement reliability/reduction bandwidth demand.According to reports, 1s is pre-
Survey can reduce bandwidth until 80%.
It is knownAccording to EKF, state variable selection is a certain attitude angle plus the aobvious drift value of head
The bivector constituted, such as
Such as
The state equation (measurement and noise) of system is as follows:
∑(ti)=H (tt)X(ti)+V(ti)
The state equation of discretization is as follows:
Xi+1=KI+1, iXi+ΓiUi
Zi=HkXk
It enables
Ui=(dotangle 0)
In formula: dt is the sampling interval, and dotangle is the angle value that sensor exports within the dt time.
As shown in fig. 6, caching can be targetedly controlled after effectively predicting viewing point, Fig. 6 description
Cache policy based on predicted value.In tiMoment carries out one-step prediction, obtains subsequent time (ti+1) viewing point position.Together
When image near new viewing point is loaded with higher resolution ratio.Measured value is combined when subsequent time really arrives, and will be regarded
The picture of near focal point is with the load of highest picture quality.
Meanwhile it is known that user's head motion state, due to people in head quick rotation eyes can not quickly it is right
Coke, i.e., the scenery during can not seeing clearly rapidly.Multiple operating conditions can be set accordingly, be more than such as certain value in head rotation speed
Afterwards, the picture of real-time loading is lower resolution.This will also be effectively relieved head and rotates the inadequate problem of Time Bandwidth rapidly.
Embodiment 2:
Fig. 7 is the description of overall transfer method, in the present embodiment, is first initialized to system at 0 moment, it is therefore an objective to
Read head shows sensor content, obtains initial viewing point and to several angular speed assignment.By the operation of a bit of time
Afterwards, head shows motion state and revert to true value.At this moment it can be carried out accurately viewing point prediction, and assist adaptive pass
It is defeated.In tiMoment predicts ti+1The viewing point drop point at moment, and buffered in advance this partial content.And if it is judged that at this time
That carves head moves past certain speed (thinking that user can not correctly focus at this time), it is possible to reduce supplements the content of transmission, saves
Bandwidth-saving.Until ti+1When moment really arrives, the part do not downloaded of caching is supplemented again with measured value.This algorithm operates in nothing
On line VR head-mounted display apparatus, the VR adaptive wireless transmission of high stability can be effectively realized.
Claims (7)
1. a kind of adaptive VR radio transmitting method based on viewing point prediction characterized by comprising
Initialization system obtains head and shows sensing data content；
After aobvious motion state revert to true value, viewing point prediction is carried out；
Viewing point drop point according to prediction loads the image data near viewing point in advance, and real-time measurement head shows viewing point movement
Speed and direction.
2. the adaptive VR radio transmitting method as described in claim 1 based on viewing point prediction, which is characterized in that described first
Beginning system includes that read head shows sensor content, obtains initial viewing point and the angular speed assignment to attitude angle.
3. the adaptive VR radio transmitting method as claimed in claim 2 based on viewing point prediction, which is characterized in that the appearance
State angle includes pitch angle, yaw angle and roll angle.
4. the adaptive VR radio transmitting method as described in claim 1 based on viewing point prediction, which is characterized in that the view
Focus prediction include by using aobvious viewing point coordinate position the combination Kalman filtering algorithm provided of head, prediction 0.5~1 second with
Viewing point position afterwards.
5. the adaptive VR radio transmitting method as described in claim 1 based on viewing point prediction, which is characterized in that according to pre-
The viewing point drop point of survey loads the image data near prediction viewing point position with higher resolution ratio in advance.
6. the adaptive VR radio transmitting method as described in claim 1 based on viewing point prediction, which is characterized in that also wrap
It includes:
Aobvious viewing point combines measured value when being moved to next viewing point position, by the image data near viewing point with highest
Picture quality load.
7. the adaptive VR radio transmitting method as described in claim 1 based on viewing point prediction, which is characterized in that also wrap
It includes:
Judge that head shows velocity of rotation in conjunction with measured value, if it is more than limit value that head, which shows velocity of rotation, system is loaded with lower resolution
Show picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811181333.3A CN109302602A (en) | 2018-10-11 | 2018-10-11 | A kind of adaptive VR radio transmitting method based on viewing point prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811181333.3A CN109302602A (en) | 2018-10-11 | 2018-10-11 | A kind of adaptive VR radio transmitting method based on viewing point prediction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109302602A true CN109302602A (en) | 2019-02-01 |
Family
ID=65162344
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811181333.3A Pending CN109302602A (en) | 2018-10-11 | 2018-10-11 | A kind of adaptive VR radio transmitting method based on viewing point prediction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109302602A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110958325A (en) * | 2019-12-11 | 2020-04-03 | 联想(北京)有限公司 | Control method, control device, server and terminal |
| CN111131805A (en) * | 2019-12-31 | 2020-05-08 | 歌尔股份有限公司 | Image processing method, device and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103900473A (en) * | 2014-03-31 | 2014-07-02 | 浙江大学 | Intelligent mobile device six-degree-of-freedom fused pose estimation method based on camera and gravity inductor |
| CN103930817A (en) * | 2011-06-20 | 2014-07-16 | 谷歌公司 | Systems and methods for adaptive transmission of data |
| CN106959745A (en) * | 2016-01-12 | 2017-07-18 | 深圳多哚新技术有限责任公司 | A kind of head pose Forecasting Methodology and device |
| WO2018064287A1 (en) * | 2016-09-28 | 2018-04-05 | Ariadne's Thread (Usa), Inc. | Predictive virtual reality display system with post rendering correction |
| WO2018081418A1 (en) * | 2016-10-26 | 2018-05-03 | Alibaba Group Holding Limited | Performing virtual reality input |
| CN108463765A (en) * | 2016-04-08 | 2018-08-28 | 谷歌有限责任公司 | Based on pose information at head-mounted display apparatus coded image data |
-
2018
- 2018-10-11 CN CN201811181333.3A patent/CN109302602A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103930817A (en) * | 2011-06-20 | 2014-07-16 | 谷歌公司 | Systems and methods for adaptive transmission of data |
| CN103900473A (en) * | 2014-03-31 | 2014-07-02 | 浙江大学 | Intelligent mobile device six-degree-of-freedom fused pose estimation method based on camera and gravity inductor |
| CN106959745A (en) * | 2016-01-12 | 2017-07-18 | 深圳多哚新技术有限责任公司 | A kind of head pose Forecasting Methodology and device |
| CN108463765A (en) * | 2016-04-08 | 2018-08-28 | 谷歌有限责任公司 | Based on pose information at head-mounted display apparatus coded image data |
| WO2018064287A1 (en) * | 2016-09-28 | 2018-04-05 | Ariadne's Thread (Usa), Inc. | Predictive virtual reality display system with post rendering correction |
| WO2018081418A1 (en) * | 2016-10-26 | 2018-05-03 | Alibaba Group Holding Limited | Performing virtual reality input |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110958325A (en) * | 2019-12-11 | 2020-04-03 | 联想(北京)有限公司 | Control method, control device, server and terminal |
| CN111131805A (en) * | 2019-12-31 | 2020-05-08 | 歌尔股份有限公司 | Image processing method, device and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11553223B1 (en) | Systems and methods for video delivery based upon saccadic eye motion | |
| US11546397B2 (en) | VR 360 video for remote end users | |
| KR102111501B1 (en) | Server, device and method for providing virtual reality experience service | |
| US9781356B1 (en) | Panoramic video viewer | |
| EP3443749B1 (en) | Systems and methods for video processing and display | |
| RU2709943C2 (en) | System (embodiments) and method of data transmission from vehicle to remote control point | |
| US10687050B2 (en) | Methods and systems of reducing latency in communication of image data between devices | |
| US10499066B2 (en) | Method and apparatus for improving efficiency of content delivery based on consumption data relative to spatial data | |
| CN111627116B (en) | Image rendering control method and device and server | |
| EP3406310A1 (en) | Method and apparatuses for handling visual virtual reality content | |
| US9965830B2 (en) | Image processing apparatus, image processing method, and program | |
| CN106537894A (en) | System and method for use in playing back panorama video content | |
| CN109302602A (en) | A kind of adaptive VR radio transmitting method based on viewing point prediction | |
| KR101965746B1 (en) | Server and method for streaming virtual reality contents | |
| US11570417B2 (en) | Immersive video streaming using view-adaptive prefetching and buffer control | |
| CN109074152B (en) | Virtual reality image sending method and device | |
| WO2018049642A1 (en) | Method and device for providing image in wearable device and movable object | |
| US11711415B2 (en) | Measuring quality-of-experience (QoE) for virtual reality (VR) streaming content | |
| CN110383821B (en) | Virtual reality image reproduction method and program using the same | |
| CN108419052B (en) | Panoramic imaging method for multiple unmanned aerial vehicles | |
| CN110291776B (en) | Flight control method and aircraft | |
| US11881192B2 (en) | Compensating for latency in a streaming virtual reality environment | |
| EP4142292A1 (en) | Image content transmitting method and device using edge computing service | |
| CN113992996A (en) | Method and device for transmitting data | |
| WO2024040535A1 (en) | Video processing method and apparatus, device, and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190201 |
|
| RJ01 | Rejection of invention patent application after publication |