WO2020258427A1 - 一种QoE驱动下的VR视频自适应采集与传输方法 - Google Patents

一种QoE驱动下的VR视频自适应采集与传输方法 Download PDF

Info

Publication number
WO2020258427A1
WO2020258427A1 PCT/CN2019/097101 CN2019097101W WO2020258427A1 WO 2020258427 A1 WO2020258427 A1 WO 2020258427A1 CN 2019097101 W CN2019097101 W CN 2019097101W WO 2020258427 A1 WO2020258427 A1 WO 2020258427A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
branch
transmission
function
server
Prior art date
Application number
PCT/CN2019/097101
Other languages
English (en)
French (fr)
Inventor
黎洁
冯燃生
刘志
李奇越
孙伟
张聪
Original Assignee
合肥工业大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 合肥工业大学 filed Critical 合肥工业大学
Publication of WO2020258427A1 publication Critical patent/WO2020258427A1/zh
Priority to US17/481,211 priority Critical patent/US11831883B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/10—Architectures or entities
    • H04L65/1013—Network architectures, gateways, control or user entities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/613—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for the control of the source by the destination
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/765—Media network packet handling intermediate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194—Transmission of image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • H04N19/166—Feedback from the receiver or from the transmission channel concerning the amount of transmission errors, e.g. bit error rate [BER]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0092—Image segmentation from stereoscopic image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0096—Synchronisation or controlling aspects

Definitions

  • the invention relates to the field of multimedia video transmission, in particular to a streaming media adaptive collection and transmission method for VR video.
  • the method includes: establishing a three-dimensional VR video and a two-dimensional planar VR video mapping relationship model, based on human vision
  • the VR video is prioritized by region and motion characteristics
  • the server side slices the VR video
  • the client bandwidth estimation module uses the Kalman filter algorithm to predict the available bandwidth
  • the client video cache module smooths the available bandwidth based on the state of the buffer area.
  • the client user window perception module predicts the user window based on the motion inertia
  • the client decision module comprehensively considers the user window, network environment and buffer area to adaptively transmit VR video.
  • this method does not consider the role of QoE in VR video transmission, lacks the user's quality of experience (QoE) indicator during the transmission process, and cannot reflect the changes in the user experience during the transmission process.
  • QoE quality of experience
  • Xu Yuanyuan and others from Hohai University invented a multiple description video encoding method for VR video (public number: CN107995493A), the steps are as follows: (1) According to the viewing angle of the receiving end device, the VR video is divided into multiple spatial segments (2) Select and encode the spatial segment containing the user’s area of interest, perform slice interleaving, and generate two corresponding descriptions, which are transmitted independently on the network; (3) At the receiving end, according to the received Decoding is performed separately when a single description or two descriptions are received at the same time. Although this method reflects the priority transmission of the user's area of interest, it only considers the downlink transmission process in the VR video transmission process, and does not consider the uplink transmission process. Such a single optimization will reduce the overall performance of the entire transmission system.
  • the system includes an image acquisition module, a streaming media server module, and a user terminal module.
  • the image acquisition module includes images.
  • the streaming media server module includes a processing module, a storage module, and a second transmission module;
  • the processing module includes a real-time splicing unit, a video encoding unit, and a photographing processing unit;
  • the storage module includes a storage unit a, a storage unit b, and Storage unit c;
  • the user terminal module includes at least one user equipment, and each user equipment includes a display module, a user interaction module, and a third transmission module.
  • the present invention provides a QoE-driven VR video adaptive acquisition and transmission method, in order to better improve resource utilization and improve bandwidth under restricted conditions. Multi-user experience QoE.
  • the present invention is a QoE-driven VR video adaptive acquisition and transmission method, which is applied to a network environment composed of C cameras, a VR video server and N clients; between the cameras and the VR video server Through the uplink transmission, the VR video server and the client are transmitted through the downlink; the downlink includes a feedback channel from the client to the VR video server; its characteristic is that the VR video is self-contained To adapt to the collection and transmission method, proceed as follows:
  • Step 1 In the network environment, record C original videos captured by C cameras as ⁇ V 1 ,V 2 ,...,V c ,...,V C ⁇ , where V c represents the first Original video taken by c cameras, 1 ⁇ c ⁇ C;
  • V c e represents the original video of the e-th code rate level obtained after the c-th original video V c is compressed, and 1 ⁇ e ⁇ E;
  • Step 2 Establish the maximum total utility value constituted by the sum of the quality experience QoE of N clients as the objective function, and set the corresponding constraint conditions, thereby establishing the VR video adaptive acquisition and transmission model;
  • Step 3 Solve the VR video adaptive acquisition and transmission model by using KKT conditions and the hybrid branch and bound method to obtain uplink acquisition decision variables and downlink transmission decision variables in the network environment;
  • Step 4 The VR video server collects decision variables according to the uplink Select the original video of the e-th bitrate level for the c-th camera, and upload the original video of the e-th bitrate level selected by the c-th camera to the VR video server through the uplink, so that all The VR video server receives the original video of the corresponding bit rate level selected by each of the C cameras;
  • Step 5 The VR video server performs stitching and mapping processing on C original videos of corresponding bitrate levels, thereby synthesizing a complete VR video;
  • Step 6 The VR video server performs block processing on the complete VR video to obtain T video blocks, denoted as ⁇ T 1 ,T 2 ,...,T t ,...,T T ⁇ ,T t Represents any t-th video block, 1 ⁇ t ⁇ T;
  • the VR video server provides D code rate options for the t-th video block T t for compression processing, thereby obtaining D compressed video blocks with different code rate levels, denoted as Represents the compressed video block of the d-th code rate level obtained after the t-th video block T t is compressed, and 1 ⁇ d ⁇ D;
  • Step 7 Assuming that the modulation and coding mode in the network environment is ⁇ M 1 , M 2 ,..., M m ,..., M M ⁇ , M m represents the m-th modulation and coding mode, 1 ⁇ m ⁇ M; and the VR video server selects the m-th modulation and coding mode for the t-th video block T t ;
  • the VR video server transmits decision variables according to the downlink The value of, selects the compressed video block of the d-th code rate level of the t-th video block T t for any n-th client And through the downlink, the compressed video block of the d-th code rate level of the selected t-th video block T t Transmit to the n-th client through the m-th modulation and coding method; thereby enabling the n-th client to receive the compressed video blocks of the corresponding bit rate level of the T video blocks through the corresponding modulation and coding method;
  • Step 8 The nth client decodes, maps, and renders compressed video blocks of corresponding bitrate levels of the received T video blocks, thereby synthesizing a QoE optimized VR video.
  • Step 2.1 Use formula (1) to construct the objective function:
  • Equation (1) represents the sum of QoE of N clients, that is, the total utility value of the system; in Equation (1), Indicates the bit rate of video block t with quality level d; Represents the bit rate when the video block t is transmitted at the highest quality level D; Represents the video block covered in the FoV of the nth client; when When, it means that the t-th video block is transmitted to the client through the downlink at the d-th code rate level and the m-th modulation and coding scheme. When When, it means that the t-th video block is not transmitted to the client through the downlink at the d-th code rate level and the m-th modulation and coding scheme;
  • Step 2.2 Use formula (2)-formula (7) to construct constraint conditions:
  • Equation (2) means that any c-th camera can only select the original video of one bitrate level to upload to the server; in equation (2), when , It means that the c-th camera uploads the original video of the e-th bitrate level to the server, when , It means that the c-th camera does not upload the original video of the e-th bitrate level to the server;
  • Equation (3) indicates that the total bit rate of the transmitted C videos should not exceed the total bandwidth of the entire uplink channel; in equation (3), BW UL represents the total bandwidth value of the uplink channel;
  • Equation (4) indicates that when any t-th video block is transmitted to the client through the downlink at the quality level d, only one modulation and coding scheme can be selected;
  • Equation (5) indicates that when any t-th video block is transmitted to the client through the downlink with the m-th modulation and coding scheme, the transmitted video block can only select one bit rate level;
  • Equation (6) indicates that the total bit rate of all video blocks transmitted does not exceed the bit rate that can be provided by all resource blocks in the entire downlink channel; in equation (6), Represents the bit rate that a single resource block can provide when the m-th modulation and coding scheme is selected; Y DL represents the total number of all resource blocks in the downlink channel;
  • Equation (7) indicates that the code rate of any t-th video block in the downlink in the network environment is not greater than the code rate of the original video taken by any c-th camera in the uplink.
  • the third step is carried out as follows:
  • Step 3.1 the acquisition decision variables in the VR video adaptive acquisition and transmission model And transmission decision variables Perform relaxation operations to obtain continuous acquisition decision variables and continuous transmission decision variables within the range of [0,1];
  • Step 3.2 According to the constraints of formula (2)-(7), change As a function will As a function will As a function will As a function will As a function will As a function will As a function will As a function Thus, formula (8) is used to calculate the Lagrangian function of the relaxed VR video adaptive acquisition and transmission model
  • ⁇ represents the Lagrangian coefficient of the equality constraint in formula (2)-(7)
  • ⁇ represents the Lagrangian coefficient of the inequality constraint in formula (2)-(7)
  • ⁇ 1 means function Lagrangian coefficient of
  • ⁇ 2 represents the function Lagrangian coefficient of
  • ⁇ 3 represents the function Lagrangian coefficient of
  • ⁇ 1 represents the function Lagrangian coefficient of
  • ⁇ 2 represents the function
  • ⁇ 3 represents the function
  • the Lagrangian coefficient, QoE n represents the quality experience of the nth client, and has:
  • Step 3.3 Lagrangian function according to equation (8) Obtain the KKT conditions of the relaxed VR video adaptive acquisition and transmission model as shown in equations (10)-(15):
  • the optimal solution ⁇ relax and the optimal total utility value Z relax of the relaxed VR video adaptive acquisition and transmission model are obtained; among them, the optimal solution ⁇ relax includes the acquisition decision variable And transmission decision variables The optimal solution for relaxation;
  • Step 3.4 Use the optimal solution ⁇ relax and the optimal total utility value Z relax as the initial input parameters of the hybrid branch and bound method;
  • Step 3.5 Define the number of branches in the hybrid branch and bound method as k, define the lower bound of the optimal total utility value in the hybrid branch and bound method as L, and define the optimal total utility value in the hybrid branch and bound method The upper bound is U;
  • Step 3.9 Use ⁇ k to denote the optimal solution of the kth branch, and record the corresponding optimal total utility value as Z k , and then assign the value of ⁇ relax to ⁇ k , and use the optimal solution of the kth branch Solve ⁇ k as the root node;
  • Step 3.10 Determine whether there is a solution in ⁇ k that does not meet the 0-1 constraint. If there is, the optimal solution for relaxation in ⁇ k is divided into the solution that meets the 0-1 constraint and the solution that does not meet the 0-1 constraint. Solution ⁇ k(0,1) , and perform step 3.12; otherwise, it means that ⁇ k is the optimal solution of the non-relaxed VR video adaptive acquisition and transmission model;
  • Step 3.12 use KKT conditions to find the relaxation solutions of sub-branch I and sub-branch II of the kth branch, and use them as the optimal solution ⁇ k+1 and the optimal total utility value Z k+1 to the k+1 branch ,
  • ⁇ k+1 includes: the relaxation solutions of sub-branch I and sub-branch II of the k+1th branch;
  • Step 3.13 Determine whether the optimal solution ⁇ k+1 of the k+ 1th branch meets the 0-1 constraint, and if so, find the maximum value from the optimal total utility value Z k+1 and assign it to L , And ⁇ k+1 ⁇ 0,1 ⁇ ; otherwise, find the maximum value from the optimal total utility value Z k+1 and assign it to U, and ⁇ k+1 ⁇ (0,1);
  • Step 3.14 judge whether Z k+1 ⁇ L holds; if yes, cut off the branch where the optimal solution ⁇ k+1 of the k+1 branch is located, and assign k+1 to k, then return to step 3.10 ; Otherwise, go to step 3.15;
  • Step 3.15 Judge whether Z k+1 > L holds, if yes, then assign k+1 to k and return to step 3.10; otherwise, go to step 3.16;
  • the present invention proposes a streaming media adaptive acquisition and transmission method for VR video, and jointly considers the uplink and downlink transmission process of VR video, and uses this to optimize the multi-user QoE in the VR video transmission process, so as to better Improve the QoE of total users during VR video transmission.
  • the present invention combines the adaptive transmission of VR video streaming media with multi-user QoE, and proposes a transmission method for optimizing VR video streaming media using QoE of the total system user as a transmission guiding factor, so as to better guide And optimize the streaming media transmission process of VR video.
  • the present invention solves the proposed VR video adaptive acquisition and transmission model by applying the KKT condition and hybrid branch and bound method to improve the efficiency and accuracy of the solution method, thereby improving the performance of the VR video adaptive acquisition and transmission method High efficiency.
  • Figure 1 is an application scene diagram of the VR video streaming media collection and transmission method proposed in the present invention
  • Figure 2 is a system structure diagram of the adaptive acquisition and transmission method proposed in the present invention.
  • a method for adaptive acquisition and transmission of VR video driven by QoE is applied to a multi-user network scenario where there are C cameras, VR video servers, and N Client.
  • the transmission between the camera and the VR video server is through the uplink, and the transmission between the VR video server and the user is through the downlink;
  • the downlink includes a feedback channel from the user to the VR video server; the feedback channel can transfer users
  • the real-time viewing angle information and downlink bandwidth information are fed back to the server to help the server to collect and transmit.
  • the method specifically includes the following steps:
  • Step 1 In the application network scene, C original videos taken by C cameras are recorded as ⁇ V 1 ,V 2 ,...,V c ,...,V C ⁇ , V c represents the cth The original video taken by the camera, 1 ⁇ c ⁇ C;
  • the original video V c captured by the c-th camera can be compressed to obtain E original videos with different bit rate levels V c e e represents the kind of the original video rate level of cameras C c c c V captured original video obtained after compression, 1 ⁇ e ⁇ E;
  • Step 2 According to the maximum total utility value constituted by the sum of the quality experience QoE of N clients as the objective function, and set the corresponding constraint conditions, use equations (1)-(7) to establish VR video adaptive acquisition and Transmission model
  • Equation (1) represents the sum of QoE of N clients, that is, the total utility value of the system; in Equation (1), Indicates the bit rate of video block t with quality level d; Represents the bit rate when the video block t is transmitted at the highest quality level D; Represents the video block covered in the FoV of the nth client; when When, it means that the t-th video block is transmitted to the client through the downlink at the d-th code rate level and the m-th modulation and coding scheme. When When, it means that the t-th video block is not transmitted to the client through the downlink at the d-th code rate level and the m-th modulation and coding scheme;
  • Equation (2) means that any c-th camera can only select the original video of one bitrate level to upload to the server; in equation (2), when , It means that the c-th camera uploads the original video of the e-th bitrate level to the server, when , It means that the c-th camera does not upload the original video of the e-th bitrate level to the server;
  • Equation (3) indicates that the total bit rate of the transmitted C videos should not exceed the total bandwidth of the entire uplink channel; in equation (3), BW UL represents the total bandwidth value of the uplink channel;
  • Equation (4) indicates that when any t-th video block is transmitted to the client through the downlink at the quality level d, only one modulation and coding scheme can be selected;
  • Equation (5) indicates that when any t-th video block is transmitted to the client through the downlink with the m-th modulation and coding scheme, the transmitted video block can only select one bit rate level;
  • Equation (6) indicates that the total bit rate of all video blocks transmitted does not exceed the bit rate that can be provided by all resource blocks in the entire downlink channel; in equation (6), Represents the bit rate that a single resource block can provide when the m-th modulation and coding scheme is selected; Y DL represents the total number of all resource blocks in the downlink channel;
  • Equation (7) indicates that the bit rate of any t-th video block in the downlink in the network environment is not greater than the bit rate of the original video taken by any c-th camera in the uplink.
  • Step 3 Use KKT conditions and hybrid branch and bound method to solve the VR video adaptive acquisition and transmission model, and obtain the uplink acquisition decision variables and downlink transmission decision variables in the network environment;
  • Step 3.1 the acquisition decision variables in the VR video adaptive acquisition and transmission model And transmission decision variables Perform relaxation operations to obtain continuous acquisition decision variables and continuous transmission decision variables within the range of [0,1];
  • Step 3.2 According to the constraints of formula (2)-(7), change As a function will As a function will As a function will As a function will As a function will As a function will As a function will As a function Thus, formula (8) is used to calculate the Lagrangian function of the relaxed VR video adaptive acquisition and transmission model
  • ⁇ represents the Lagrangian coefficient of the equality constraint in formula (2)-(7)
  • ⁇ represents the Lagrangian coefficient of the inequality constraint in formula (2)-(7)
  • ⁇ 1 means function Lagrangian coefficient of
  • ⁇ 2 represents the function Lagrangian coefficient of
  • ⁇ 3 represents the function Lagrangian coefficient of
  • ⁇ 1 represents the function Lagrangian coefficient of
  • ⁇ 2 represents the function
  • ⁇ 3 represents the function
  • the Lagrangian coefficient, QoE n represents the quality experience of the nth client, and has:
  • Step 3.3 Lagrangian function according to equation (8) Obtain the KKT conditions of the relaxed VR video adaptive acquisition and transmission model as shown in equations (10)-(15):
  • Equations (10) and (11) express the Lagrangian function Necessary conditions when taking extreme values; equations (12) and (13) represent functions Equation (14) represents the constraint conditions of Lagrangian coefficients ⁇ 1 , ⁇ 2 , ⁇ 3 , ⁇ 1 , ⁇ 2 , and ⁇ 3 ; Equation (15) represents the complementary relaxation condition.
  • the optimal solution ⁇ relax and the optimal total utility value Z relax of the relaxed VR video adaptive acquisition and transmission model are obtained; among them, the optimal solution ⁇ relax includes the acquisition decision variable And transmission decision variables The optimal solution for relaxation;
  • Step 3.4 Use the optimal solution ⁇ relax and the optimal total utility value Z relax as the initial input parameters of the hybrid branch and bound method;
  • Step 3.5 Define the number of branches in the algorithm as k, define the lower bound of the optimal total utility value in the algorithm as L, and define the upper bound of the optimal total utility value in the algorithm as U;
  • ⁇ 0-1 denote the optimal solution of the non-relaxed VR video adaptive acquisition and transmission model
  • Z 0-1 denote the optimal total utility value of the non-relaxed VR video adaptive acquisition and transmission model
  • Step 3.9 Use ⁇ k to denote the optimal solution of the kth branch, and record the corresponding optimal total utility value as Z k , and then assign the value of ⁇ relax to ⁇ k , and use the optimal solution of the kth branch Solve ⁇ k as the root node;
  • Step 3.10 Determine whether there is a solution in ⁇ k that does not meet the 0-1 constraint. If there is, the optimal solution for relaxation in ⁇ k is divided into the solution that meets the 0-1 constraint and the solution that does not meet the 0-1 constraint. Solution ⁇ k(0,1) , and perform step 3.12; otherwise, it means that ⁇ k is the optimal solution of the non-relaxed VR video adaptive acquisition and transmission model;
  • Step 3.12 use KKT conditions to find the relaxation solutions of sub-branch I and sub-branch II of the kth branch, and use them as the optimal solution ⁇ k+1 and the optimal total utility value Z k+1 to the k+1 branch ,
  • ⁇ k+1 includes: the relaxation solutions of sub-branch I and sub-branch II of the k+1th branch;
  • Step 3.13 Determine whether the optimal solution ⁇ k+1 of the k+ 1th branch meets the 0-1 constraint, if so, find the maximum value from the optimal total utility value Z k+1 and assign it to L, and ⁇ k+1 ⁇ 0,1 ⁇ ; otherwise, find the maximum value from the optimal total utility value Z k+1 and assign it to U, and ⁇ k+1 ⁇ (0,1);
  • Step 3.14 judge whether Z k+1 ⁇ L holds; if yes, cut off the branch where the optimal solution ⁇ k+1 of the k+1 branch is located, and assign k+1 to k, then return to step 3.10 ; Otherwise, go to step 3.15;
  • Step 3.15 Judge whether Z k+1 > L holds, if yes, then assign k+1 to k and return to step 3.10; otherwise, go to step 3.16;
  • Step 4 The VR video server collects decision variables according to the uplink Select the original video of the e-th bitrate level for the c-th camera, and upload the original video of the e-th bitrate level selected by the c-th camera to the VR video server through the uplink, so that the VR The video server receives the original video of the corresponding bit rate level selected by each of the C cameras;
  • Step 5 The VR video server stitches and maps C original videos of corresponding bitrate levels to synthesize a complete VR video;
  • Step 6 The VR video server performs block processing on the complete VR video to obtain T video blocks, denoted as ⁇ T 1 ,T 2 ,...,T t ,...,T T ⁇ , T t means any The t-th video block, 1 ⁇ t ⁇ T;
  • the VR video server provides D code rate options for the t-th video block T t for compression processing, thereby obtaining D compressed video blocks with different code rate levels, denoted as Represents the compressed video block of the d-th code rate level obtained after the t-th video block T t is compressed, and 1 ⁇ d ⁇ D;
  • Step 7 Assuming that the modulation and coding method in the network environment is ⁇ M 1 ,M 2 ,...,M m ,...,M M ⁇ , M m represents the m-th modulation and coding method, and 1 ⁇ m ⁇ M; And the VR video server selects the m-th modulation and coding method for the t-th video block T t ;
  • VR video server transmits decision variables based on downlink The value of, selects the compressed video block of the d-th code rate level of the t-th video block T t for any n-th client And through the downlink, the compressed video block of the d-th code rate level of the selected t-th video block T t Transmit to the n-th client through the m-th modulation and coding method; thereby enabling the n-th client to receive the compressed video blocks of the corresponding bit rate level of the T video blocks through the corresponding modulation and coding method;
  • Step 8 The n-th client decodes, maps, and renders compressed video blocks of the corresponding bit rate levels of the received T video blocks, thereby synthesizing a QoE optimized VR video.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本发明公开了一种QoE驱动下的VR视频自适应采集与传输方法，其步骤包括：1、VR视频采集系统中的每个摄像机都拍摄出相同码率等级的原始视频，每个原始视频都被压缩成不同的码率等级；2、服务器为每个原始视频选择一种码率等级进行传输并把所有传输后的原始视频合成一个完整的VR视频；3、服务器再把合成的VR视频进行分块处理，同时把每块视频压缩成不同的质量等级；4、服务器根据反馈信道中的用户实时视角信息、下行信道带宽信息为每个视频块选择一种质量等级和MCS方案，再把每个视频块传输到客户端。本发明能够更好地提高资源利用率，在带宽受限的条件下提高多用户的体验感QoE。

Description

一种QoE驱动下的VR视频自适应采集与传输方法 技术领域
本发明涉及多媒体视频传输领域，具体的说是一种针对VR视频的流媒体自适应采集与传输方法。
背景技术
重庆邮电大学的雒江涛等人发明了一种基于DASH的VR视频自适应传输方法(公开号：CN108235131A),其方法包括：建立三维VR视频与二维平面VR视频的映射关系模型，基于人体视觉和运动特性对VR视频进行区域优先级划分，服务器端将VR视频进行切片化，客户端带宽估计模块利用卡尔曼滤波算法进行预测可用带宽，客户端视频缓存模块基于缓存区状态对可用带宽进行平滑处理，客户端用户视窗感知模块基于运动惯性进行用户视窗预测，客户端决策模块综合考虑用户视窗、网络环境和缓存区状态自适应传输VR视频。但是此种方法并没有考虑到QoE在VR视频传输中的作用，缺少了传输过程中用户的体验质量(QoE)指标，并不能体现传输过程中用户的体验感变化情况。
北京理工大学的费泽松等人发明了一种基于VR终端反馈的自适应编码方法(公开号：CN107529064A)，其方法思想是：通过在传输机制上做出改进，即通过对VR视频分割成多个视角视频，各视角视频信息独立编码传输，再根据终端视角跟踪技术实时传输用户需要的视角视频信息，其他视角低码率传输，在终端将各视角视频拼接成VR视频，再通过终端打分反馈机制使用户得到合适的视角信息，用户的打分指令传输到服务器，服务器根据用户打分映射不同的码率反馈到终端。虽然此种方法考虑了终端视角对于VR视频传输时的反馈作用，但是它忽略了QoE在传输过程中的指导作用，不能提高传输过程中用户的体验感。
河海大学的徐媛媛等人发明了一种VR视频的多描述视频编码方法(公开号：CN107995493A)，其步骤如下：(1)按照接收端设备的视角大小，对VR视频划分为多个空域片段；(2)选择包含用户感兴趣区域的空域片段并对之进行编码，进行片交织，生成相应的两个描述,两个描述各自独立地在网络中传输；(3)在接收端，根据接收到单个描述或同时接收到两个描述的不同情况分别进行解码。此种方法虽然体现了用户感兴趣区域的优先传输，但是它只考虑了VR视频传输过程中的下行传输过程，没有考虑上行传输过程，这样单独优化会降低整个传输系统的整体性能。
湖北工业大学的熊炜等人发明了一种高分辨率VR视频直播拍照系统与方法(公开号：CN108769755A)，该系统包括图像采集模块、流媒体服务器模块和用户终端模块,图像采集模 块包括图像采集设备和第一传输模块；流媒体服务器模块包括处理模块、存储模块和第二传输模块；处理模块包括实时拼接单元、视频编码单元和拍照处理单元；存储模块包括存储单元a、存储单元b和存储单元c；用户终端模块包括至少一个用户设备，每个用户设备包括显示模块、用户交互模块和第三传输模块。虽然此方法考虑了VR视频直播的上行传输和下行传输过程，但是它并没有考虑到QoE在VR视频系统中的重要性，并不能提高传输过程中的用户体验质量。
发明内容
本发明是为避免上述现有技术所存在的不足之处，提供一种QoE驱动下的VR视频自适应采集与传输方法，以期能够更好地提高资源利用率，以及在带宽受限条件下提高多用户的体验感QoE。
本发明为解决技术问题采用如下技术方案：
本发明一种QoE驱动下的VR视频自适应采集与传输方法，是应用于由C个摄像机、一个VR视频服务器和N个客户端所组成的网络环境中；所述摄像机和VR视频服务器之间通过上行链路传输，所述VR视频服务器和客户端之间通过下行链路传输；所述下行链路中包含有从客户端到VR视频服务器的反馈信道；其特点是，所述VR视频自适应采集与传输方法是按如下步骤进行：
步骤一、在所述网络环境中，将C个摄像机所拍摄的C个原始视频，记为{V 1,V 2,...,V c,...,V C}，V c表示第c个摄像机拍摄的原始视频，1≤c≤C；
将第c个原始视频V c压缩成E种码率等级的原始视频，记为
Figure PCTCN2019097101-appb-000001
V c e表示第c个原始视频V c经过压缩后得到的第e种码率等级的原始视频，1≤e≤E；
步骤二、建立以N个客户端的质量体验QoE之和所构成的总效用值最大为目标函数，并设置相应的约束条件，从而建立VR视频自适应采集与传输模型；
步骤三、利用KKT条件和混合分支定界法对所述VR视频自适应采集与传输模型进行求解，得到所述网络环境中上行链路采集决策变量和下行链路传输决策变量；
步骤四、所述VR视频服务器根据上行链路的采集决策变量
Figure PCTCN2019097101-appb-000002
的值，为第c个摄像机选择第e种码率等级的原始视频，并通过上行链路将第c个摄像机所选择的第e种码率等级的原始视频上传到VR视频服务器，从而使得所述VR视频服务器接收到C个摄像机各自所选择的 相应码率等级的原始视频；
步骤五、所述VR视频服务器对C个相应码率等级的原始视频进行缝合和映射处理，从而合成一个完整的VR视频；
步骤六、所述VR视频服务器对完整的VR视频进行分块处理，得到T个视频块，记为{T 1,T 2,...,T t,...,T T},T t表示任意第t个视频块，1≤t≤T；
所述VR视频服务器为第t个视频块T t提供D种码率选择用于压缩处理，从而得到D种不同码率等级的压缩视频块，记为
Figure PCTCN2019097101-appb-000003
Figure PCTCN2019097101-appb-000004
表示第t个视频块T t经过压缩处理后得到的第d种码率等级的压缩视频块，1≤d≤D；
步骤七、假设所述网络环境中的调制编码方式为{M 1,M 2,...,M m,...,M M},M m表示第m种调制编码方式，1≤m≤M；且所述VR视频服务器为第t个视频块T t选择第m种调制编码方式；
所述VR视频服务器根据下行链路传输决策变量
Figure PCTCN2019097101-appb-000005
的值，为任意第n个客户端选择第t个视频块T t的第d种码率等级的压缩视频块
Figure PCTCN2019097101-appb-000006
并通过下行链路将所选择的第t个视频块T t的第d种码率等级的压缩视频块
Figure PCTCN2019097101-appb-000007
通过第m种调制编码方式传输给第n个客户端；从而使得第n个客户端通过相应的调制编码方式接收到T个视频块的相应码率等级的压缩视频块；
步骤八、所述第n个客户端将所接收到的T个视频块的相应码率等级的压缩视频块进行解码、映射和渲染处理，从而合成QoE优化后的VR视频。
本发明所述的VR视频自适应采集与传输方法的特点是，所述步骤二是按如下过程进行：
步骤2.1、利用式(1)构建目标函数：
Figure PCTCN2019097101-appb-000008
式(1)表示N个客户端的QoE之和，即系统的总效用值；式(1)中，
Figure PCTCN2019097101-appb-000009
表示质量等级为d的视频块t的码率；
Figure PCTCN2019097101-appb-000010
表示视频块t以最高的质量等级D传输时的码率；
Figure PCTCN2019097101-appb-000011
表示第n个客户端的FoV内所覆盖的视频块；当
Figure PCTCN2019097101-appb-000012
时，表示第t个视频块以第d种码率等级和第m种调制与编码方案通过下行链路传输到客户端，当
Figure PCTCN2019097101-appb-000013
时，表示第t个视频块不以第d种码率等级和第m种调制与编码方案通过下行链路传输到客户端；
步骤2.2、利用式(2)-式(7)构建约束条件：
Figure PCTCN2019097101-appb-000014
Figure PCTCN2019097101-appb-000015
Figure PCTCN2019097101-appb-000016
Figure PCTCN2019097101-appb-000017
Figure PCTCN2019097101-appb-000018
Figure PCTCN2019097101-appb-000019
式(2)表示任意第c个摄像机只能选择一种码率等级的原始视频上传到服务器；式(2)中，当
Figure PCTCN2019097101-appb-000020
时，表示第c个摄像机以第e种码率等级的原始视频上传到服务器，当
Figure PCTCN2019097101-appb-000021
时，表示第c个摄像机不以第e种码率等级的原始视频上传到服务器；
式(3)表示传输的C个视频的总码率不应当超出整个上行信道的总带宽；式(3)中，BW UL表示上行信道的总带宽值；
式(4)表示任意第t个视频块在质量等级为d通过下行链路传输到客户端时，只能选择一种调制与编码方案；
式(5)表示任意第t个视频块在以第m种调制与编码方案通过下行链路传输到客户端时，所传输的视频块只能选择一种码率等级；
式(6)表示传输的所有视频块的总码率不超过整个下行信道中所有资源块所能提供的比特率；式(6)中，
Figure PCTCN2019097101-appb-000022
表示当选择第m种调制与编码方案时单个资源块所能提供的比特率；Y DL表示下行信道中所有资源块的总数目；
式(7)表示所述网络环境中下行链路中任意第t个视频块的码率不大于上行链路中任意第c个摄像头拍摄的原始视频的码率。
所述步骤三是按如下过程进行：
步骤3.1、对所述VR视频自适应采集与传输模型中的采集决策变量
Figure PCTCN2019097101-appb-000023
和传输决策变量
Figure PCTCN2019097101-appb-000024
进行松弛操作，分别得到[0,1]范围内的连续采集决策变量和连续传输决策变量；
步骤3.2、根据式(2)-式(7)的约束条件，将
Figure PCTCN2019097101-appb-000025
记为函数
Figure PCTCN2019097101-appb-000026
将
Figure PCTCN2019097101-appb-000027
记为函数
Figure PCTCN2019097101-appb-000028
将
Figure PCTCN2019097101-appb-000029
记为函数
Figure PCTCN2019097101-appb-000030
将
Figure PCTCN2019097101-appb-000031
记为函数
Figure PCTCN2019097101-appb-000032
将
Figure PCTCN2019097101-appb-000033
记为函数
Figure PCTCN2019097101-appb-000034
将
Figure PCTCN2019097101-appb-000035
记为函数
Figure PCTCN2019097101-appb-000036
从而利用式(8)计算松弛后的VR视频自适应采集与传输模型的拉格朗日函数
Figure PCTCN2019097101-appb-000037
Figure PCTCN2019097101-appb-000038
式(8)中，λ表示式(2)-式(7)中等式约束条件的拉格朗日系数，μ表示式(2)-式(7)中不等式约束条件的拉格朗日系数，λ 1表示函数
Figure PCTCN2019097101-appb-000039
的拉格朗日系数，λ 2表示函数
Figure PCTCN2019097101-appb-000040
的拉格朗日系数，λ 3表示函数
Figure PCTCN2019097101-appb-000041
的拉格朗日系数，μ 1表示函数
Figure PCTCN2019097101-appb-000042
的拉格朗日系数，μ 2表示函数
Figure PCTCN2019097101-appb-000043
的拉格朗日系数，μ 3表示函数
Figure PCTCN2019097101-appb-000044
的拉格朗日系数，QoE n表示第n个客户端的质量体验，并有：
Figure PCTCN2019097101-appb-000045
步骤3.3、根据式(8)的拉格朗日函数
Figure PCTCN2019097101-appb-000046
得到如式(10)-式(15)所示的松弛后的VR视频自适应采集与传输模型的KKT条件：
Figure PCTCN2019097101-appb-000047
Figure PCTCN2019097101-appb-000048
Figure PCTCN2019097101-appb-000049
Figure PCTCN2019097101-appb-000050
λ 1,λ 2,λ 3≠0,μ 1,μ 2,μ 3≥0            (14)
Figure PCTCN2019097101-appb-000051
对式(10)-式(15)进行求解，得到松弛后的VR视频自适应采集与传输模型的最优解χ relax和最优总效用值Z relax；其中，最优解χ relax包括采集决策变量
Figure PCTCN2019097101-appb-000052
和传输决策变量
Figure PCTCN2019097101-appb-000053
的松弛最优解；
步骤3.4、以最优解χ relax和最优总效用值Z relax作为混合分支定界法的初始输入参数；
步骤3.5、定义所述混合分支定界法中分支次数为k，定义所述混合分支定界法中最优总效用值的下界为L，定义所述混合分支定界法中最优总效用值的上界为U；
步骤3.6、初始化k＝0；
步骤3.7、初始化L＝0；
步骤3.8、初始化U＝Z relax；
步骤3.9、用χ k表示第k次分支的最优解，并将对应的最优总效用值记为Z k，再将χ relax的值赋给χ k，并以第k次分支的最优解χ k作为根节点；
步骤3.10、判断χ k中是否存在不符合0-1约束条件的解，若存在，则将χ k中的松弛最优解分为符合0-1约束条件的解和不符合0-1约束条件的解χ k(0,1)，并执行步骤3.12；否则，表示χ k为非松弛VR视频自适应采集与传输模型的最优解；
步骤3.11、在(0,1)范围内随机产生一个第k次分支的随机数ε k，并判断0＜χ k(0,1)＜ε k是否成立；若成立，则将约束条件“χ k(0,1)＝0”加入到非松弛VR视频自适应采集与传输模型中，形成第k次分支的子分支I；否则，则将约束条件“χ k(0,1)＝1”加入到非松弛VR视频自适应采集与传输模型中，形成第k次分支的子分支II；
步骤3.12、利用KKT条件求出第k次分支的子分支I和子分支II的松弛解，并作为到第k+1次分支的最优解χ k+1和最优总效用值Z k+1，其中χ k+1包括：第k+1次分支的子分支I 和子分支II的松弛解；
步骤3.13、判断第k+1次分支的最优解χ k+1是否符合0-1约束条件，若是，则从所述最优总效用值Z k+1中找出最大值并赋值给L，且χ k+1∈{0,1}；否则，从所述最优总效用值Z k+1中找出最大值并赋值给U，且χ k+1∈(0,1)；
步骤3.14、判断Z k+1＜L是否成立；如果成立，则剪掉第k+1次分支的最优解χ k+1所在的分支，并将k+1赋值给k后，返回步骤3.10；否则，执行步骤3.15；
步骤3.15、判断Z k+1＞L是否成立，如果成立，则将k+1赋值给k后，返回步骤3.10；否则执行步骤3.16；
步骤3.16、判断Z k+1＝L是否成立，若成立，则表示获得非松弛VR视频自适应采集与传输模型的最优解即为第k+1分支的最优解χ k+1，并将χ k+1赋值给非松弛VR视频自适应采集与传输模型的最优解χ 0-1，将χ k+1所对应的Z k+1赋值给非松弛VR视频自适应采集与传输模型的最优总效用值Z 0-1；否则，将k+1赋值给k后，返回步骤3.10。
与现有技术相比，本发明的有益效果体现在：
1.本发明通过提出VR视频的流媒体自适应采集与传输方法，联合考虑了VR视频的上下行链路传输过程，并以此来整体优化VR视频传输过程中多用户QoE，从而更好地提高了VR视频传输过程中总用户的QoE。
2.本发明将VR视频的流媒体自适应传输与多用户的QoE相结合，提出了一种以系统总用户QoE作为传输指导因素来优化VR视频流媒体的传输方法，从而能更好地指导和优化VR视频的流媒体传输过程。
3.本发明通过应用KKT条件和混合分支定界法对提出的VR视频自适应采集与传输模型进行求解，提高了解法的效率和解的准确性，从而提高了VR视频自适应采集与传输方法的高效性。
附图说明
图1为本发明中所提出的VR视频的流媒体采集与传输方法的应用场景图；
图2为本发明中所提出的自适应采集与传输方法的系统结构图。
具体实施方式
本实施例中，一种QoE驱动下的VR视频自适应采集与传输方法，如图1所示，应用于多用户网络场景中，该网络场景中存在着C个摄像机、VR视频服务器和N个客户端。摄像机和VR视频服务器端之间通过上行链路传输，VR视频服务器和用户端之间通过下行链路传输；下行链路中包含有从用户端到VR视频服务器的反馈信道；反馈信道可以把用户的实时视角信息、下行链路带宽信息反馈给服务器，帮助服务器进行采集与传输工作。如图2所示，该方法具体包括以下步骤：
步骤1、在应用网络场景中，C个摄像机所拍摄的C个原始视频，记为{V 1,V 2,...,V c,...,V C}，V c表示第c个摄像机拍摄的原始视频，1≤c≤C；
然后第c个摄像机拍摄的原始视频V c在经过压缩之后可以得到E个不同码率等级的原始视频
Figure PCTCN2019097101-appb-000054
V c e表示第c个摄像机C c拍摄的原始视频V c经过压缩后得到的第e种码率等级的原始视频,1≤e≤E；
步骤2、根据以N个客户端的质量体验QoE之和所构成的总效用值最大为目标函数，并设置相应的约束条件，从而利用式(1)-式(7)建立VR视频自适应采集与传输模型；
目标函数：
Figure PCTCN2019097101-appb-000055
式(1)表示N个客户端的QoE之和，即系统的总效用值；式(1)中，
Figure PCTCN2019097101-appb-000056
表示质量等级为d的视频块t的码率；
Figure PCTCN2019097101-appb-000057
表示视频块t以最高的质量等级D传输时的码率；
Figure PCTCN2019097101-appb-000058
表示第n个客户端的FoV内所覆盖的视频块；当
Figure PCTCN2019097101-appb-000059
时，表示第t个视频块以第d种码率等级和第m种调制与编码方案通过下行链路传输到客户端，当
Figure PCTCN2019097101-appb-000060
时，表示第t个视频块不以第d种码率等级和第m种调制与编码方案通过下行链路传输到客户端；
约束条件：
Figure PCTCN2019097101-appb-000061
Figure PCTCN2019097101-appb-000062
Figure PCTCN2019097101-appb-000063
Figure PCTCN2019097101-appb-000064
Figure PCTCN2019097101-appb-000065
Figure PCTCN2019097101-appb-000066
式(2)表示任意第c个摄像机只能选择一种码率等级的原始视频上传到服务器；式(2)中，当
Figure PCTCN2019097101-appb-000067
时，表示第c个摄像机以第e种码率等级的原始视频上传到服务器，当
Figure PCTCN2019097101-appb-000068
时，表示第c个摄像机不以第e种码率等级的原始视频上传到服务器；
式(3)表示传输的C个视频的总码率不应当超出整个上行信道的总带宽；式(3)中，BW UL表示上行信道的总带宽值；
式(4)表示任意第t个视频块在质量等级为d通过下行链路传输到客户端时，只能选择一种调制与编码方案；
式(5)表示任意第t个视频块在以第m种调制与编码方案通过下行链路传输到客户端时，所传输的视频块只能选择一种码率等级；
式(6)表示传输的所有视频块的总码率不超过整个下行信道中所有资源块所能提供的比特率；式(6)中，
Figure PCTCN2019097101-appb-000069
表示当选择第m种调制与编码方案时单个资源块所能提供的比特率；Y DL表示下行信道中所有资源块的总数目；
式(7)表示网络环境中下行链路中任意第t个视频块的码率不大于上行链路中任意第c个摄像头拍摄的原始视频的码率。
步骤3、利用KKT条件和混合分支定界法对VR视频自适应采集与传输模型进行求解，得到网络环境中上行链路采集决策变量和下行链路传输决策变量；
步骤3.1、对VR视频自适应采集与传输模型中的采集决策变量
Figure PCTCN2019097101-appb-000070
和传输决策变量
Figure PCTCN2019097101-appb-000071
进行松弛操作，分别得到[0,1]范围内的连续采集决策变量和连续传输决策变量；
步骤3.2、根据式(2)-式(7)的约束条件，将
Figure PCTCN2019097101-appb-000072
记为函数
Figure PCTCN2019097101-appb-000073
将
Figure PCTCN2019097101-appb-000074
记为函数
Figure PCTCN2019097101-appb-000075
将
Figure PCTCN2019097101-appb-000076
记为函数
Figure PCTCN2019097101-appb-000077
将
Figure PCTCN2019097101-appb-000078
记为函数
Figure PCTCN2019097101-appb-000079
将
Figure PCTCN2019097101-appb-000080
记为函数
Figure PCTCN2019097101-appb-000081
将
Figure PCTCN2019097101-appb-000082
记为函数
Figure PCTCN2019097101-appb-000083
从而利用式(8)计算松弛后的VR视频自适应采集与传输模型的拉格朗日函数
Figure PCTCN2019097101-appb-000084
Figure PCTCN2019097101-appb-000085
式(8)中，λ表示式(2)-式(7)中等式约束条件的拉格朗日系数，μ表示式(2)-式(7)中不等式约束条件的拉格朗日系数，λ 1表示函数
Figure PCTCN2019097101-appb-000086
的拉格朗日系数，λ 2表示函数
Figure PCTCN2019097101-appb-000087
的拉格朗日系数，λ 3表示函数
Figure PCTCN2019097101-appb-000088
的拉格朗日系数，μ 1表示函数
Figure PCTCN2019097101-appb-000089
的拉格朗日系数，μ 2表示函数
Figure PCTCN2019097101-appb-000090
的拉格朗日系数，μ 3表示函数
Figure PCTCN2019097101-appb-000091
的拉格朗日系数，QoE n表示第n个客户端的质量体验，并有：
Figure PCTCN2019097101-appb-000092
步骤3.3、根据式(8)的拉格朗日函数
Figure PCTCN2019097101-appb-000093
得到如式(10)-式(15)所示的松弛后的VR视频自适应采集与传输模型的KKT条件：
Figure PCTCN2019097101-appb-000094
Figure PCTCN2019097101-appb-000095
Figure PCTCN2019097101-appb-000096
Figure PCTCN2019097101-appb-000097
λ 1,λ 2,λ 3≠0,μ 1,μ 2,μ 3≥0                (14)
Figure PCTCN2019097101-appb-000098
式(10)和式(11)表示对拉格朗日函数
Figure PCTCN2019097101-appb-000099
取极值时的必要条件；式(12)和式(13)表示函数
Figure PCTCN2019097101-appb-000100
的约束条件；式(14)表示拉格朗日系数λ 1,λ 2,λ 3,μ 1,μ 2,μ 3的约束条件；式(15)表示互补松弛条件。
对式(10)-式(15)进行求解，得到松弛后的VR视频自适应采集与传输模型的最优解χ relax和最优总效用值Z relax；其中，最优解χ relax包括采集决策变量
Figure PCTCN2019097101-appb-000101
和传输决策变量
Figure PCTCN2019097101-appb-000102
的松弛最优解；
步骤3.4、以最优解χ relax和最优总效用值Z relax作为混合分支定界法的初始输入参数；
步骤3.5、定义该算法中分支次数为k，定义该算法中最优总效用值的下界为L，定义该算法中最优总效用值的上界为U；
确定混合分支定界法的输出参数：
令χ 0-1表示非松弛VR视频自适应采集与传输模型的最优解；
令Z 0-1表示非松弛VR视频自适应采集与传输模型的最优总效用值；
步骤3.6、初始化k＝0；
步骤3.7、初始化L＝0；
步骤3.8、初始化U＝Z relax；
步骤3.9、用χ k表示第k次分支的最优解，并将对应的最优总效用值记为Z k，再将χ relax的值赋给χ k，并以第k次分支的最优解χ k作为根节点；
步骤3.10、判断χ k中是否存在不符合0-1约束条件的解，若存在，则将χ k中的松弛最优解分为符合0-1约束条件的解和不符合0-1约束条件的解χ k(0,1)，并执行步骤3.12；否则，表示χ k为非松弛VR视频自适应采集与传输模型的最优解；
步骤3.11、在(0,1)范围内随机产生一个第k次分支的随机数ε k，并判断0＜χ k(0,1)＜ε k是否成立；若成立，则将约束条件“χ k(0,1)＝0”加入到非松弛VR视频自适应采集与传输模型中， 形成第k次分支的子分支I；否则，则将约束条件“χ k(0,1)＝1”加入到非松弛VR视频自适应采集与传输模型中，形成第k次分支的子分支II；
步骤3.12、利用KKT条件求出第k次分支的子分支I和子分支II的松弛解，并作为到第k+1次分支的最优解χ k+1和最优总效用值Z k+1，其中χ k+1包括：第k+1次分支的子分支I和子分支II的松弛解；
步骤3.13、判断第k+1次分支的最优解χ k+1是否符合0-1约束条件，若是，则从最优总效用值Z k+1中找出最大值并赋值给L，且χ k+1∈{0,1}；否则，从最优总效用值Z k+1中找出最大值并赋值给U，且χ k+1∈(0,1)；
步骤3.14、判断Z k+1＜L是否成立；如果成立，则剪掉第k+1次分支的最优解χ k+1所在的分支，并将k+1赋值给k后，返回步骤3.10；否则，执行步骤3.15；
步骤3.15、判断Z k+1＞L是否成立，如果成立，则将k+1赋值给k后，返回步骤3.10；否则执行步骤3.16；
步骤3.16、判断Z k+1＝L是否成立，若成立，则表示获得非松弛VR视频自适应采集与传输模型的最优解即为第k+1分支的最优解χ k+1，并将χ k+1赋值给非松弛VR视频自适应采集与传输模型的最优解χ 0-1，将χ k+1所对应的Z k+1赋值给非松弛VR视频自适应采集与传输模型的最优总效用值Z 0-1；否则，将k+1赋值给k后，返回步骤3.10。
步骤4、VR视频服务器根据上行链路的采集决策变量
Figure PCTCN2019097101-appb-000103
的值，为第c个摄像机选择第e种码率等级的原始视频，并通过上行链路将第c个摄像机所选择的第e种码率等级的原始视频上传到VR视频服务器，从而使得VR视频服务器接收到C个摄像机各自所选择的相应码率等级的原始视频；
步骤5、VR视频服务器对C个相应码率等级的原始视频进行缝合和映射处理，从而合成一个完整的VR视频；
步骤6、VR视频服务器对完整的VR视频进行分块处理，得到T个视频块，记为{T 1,T 2,...,T t,...,T T},T t表示任意第t个视频块，1≤t≤T；
VR视频服务器为第t个视频块T t提供D种码率选择用于压缩处理，从而得到D种不同码 率等级的压缩视频块，记为
Figure PCTCN2019097101-appb-000104
Figure PCTCN2019097101-appb-000105
表示第t个视频块T t经过压缩处理后得到的第d种码率等级的压缩视频块，1≤d≤D；
步骤7、假设网络环境中的调制编码方式为{M 1,M 2,...,M m,...,M M},M m表示第m种调制编码方式，1≤m≤M；且VR视频服务器为第t个视频块T t选择第m种调制编码方式；
VR视频服务器根据下行链路传输决策变量
Figure PCTCN2019097101-appb-000106
的值，为任意第n个客户端选择第t个视频块T t的第d种码率等级的压缩视频块
Figure PCTCN2019097101-appb-000107
并通过下行链路将所选择的第t个视频块T t的第d种码率等级的压缩视频块
Figure PCTCN2019097101-appb-000108
通过第m种调制编码方式传输给第n个客户端；从而使得第n个客户端通过相应的调制编码方式接收到T个视频块的相应码率等级的压缩视频块；
步骤8、第n个客户端将所接收到的T个视频块的相应码率等级的压缩视频块进行解码、映射和渲染处理，从而合成QoE优化后的VR视频。

Claims (3)

  1. 一种QoE驱动下的VR视频自适应采集与传输方法，是应用于由C个摄像机、一个VR视频服务器和N个客户端所组成的网络环境中；所述摄像机和VR视频服务器之间通过上行链路传输，所述VR视频服务器和客户端之间通过下行链路传输；所述下行链路中包含有从客户端到VR视频服务器的反馈信道；其特征是，所述VR视频自适应采集与传输方法是按如下步骤进行：
    步骤一、在所述网络环境中，将C个摄像机所拍摄的C个原始视频，记为{V 1,V 2,...,V c,...,V C}，V c表示第c个摄像机拍摄的原始视频，1≤c≤C；
    将第c个原始视频V c压缩成E种码率等级的原始视频，记为
    Figure PCTCN2019097101-appb-100001
    V c e表示第c个原始视频V c经过压缩后得到的第e种码率等级的原始视频，1≤e≤E；
    步骤二、建立以N个客户端的质量体验QoE之和所构成的总效用值最大为目标函数，并设置相应的约束条件，从而建立VR视频自适应采集与传输模型；
    步骤三、利用KKT条件和混合分支定界法对所述VR视频自适应采集与传输模型进行求解，得到所述网络环境中上行链路采集决策变量和下行链路传输决策变量；
    步骤四、所述VR视频服务器根据上行链路的采集决策变量
    Figure PCTCN2019097101-appb-100002
    的值，为第c个摄像机选择第e种码率等级的原始视频，并通过上行链路将第c个摄像机所选择的第e种码率等级的原始视频上传到VR视频服务器，从而使得所述VR视频服务器接收到C个摄像机各自所选择的相应码率等级的原始视频；
    步骤五、所述VR视频服务器对C个相应码率等级的原始视频进行缝合和映射处理，从而合成一个完整的VR视频；
    步骤六、所述VR视频服务器对完整的VR视频进行分块处理，得到T个视频块，记为{T 1,T 2,...,T t,...,T T},T t表示任意第t个视频块，1≤t≤T；
    所述VR视频服务器为第t个视频块T t提供D种码率选择用于压缩处理，从而得到D种不同码率等级的压缩视频块，记为{T t 1,T t 2,...,T t d,...,T t D},T t d表示第t个视频块T t经过压缩处理后得到的第d种码率等级的压缩视频块，1≤d≤D；
    步骤七、假设所述网络环境中的调制编码方式为{M 1,M 2,...,M m,...,M M},M m表示第m种调制编码方式，1≤m≤M；且所述VR视频服务器为第t个视频块T t选择第m种调制编码方式；
    所述VR视频服务器根据下行链路传输决策变量
    Figure PCTCN2019097101-appb-100003
    的值，为任意第n个客户端选择第 t个视频块T t的第d种码率等级的压缩视频块T t d，并通过下行链路将所选择的第t个视频块T t的第d种码率等级的压缩视频块T t d通过第m种调制编码方式传输给第n个客户端；从而使得第n个客户端通过相应的调制编码方式接收到T个视频块的相应码率等级的压缩视频块；
    步骤八、所述第n个客户端将所接收到的T个视频块的相应码率等级的压缩视频块进行解码、映射和渲染处理，从而合成QoE优化后的VR视频。
  2. 根据权利要求1所述的VR视频自适应采集与传输方法，其特征是，所述步骤二是按如下过程进行：
    步骤2.1、利用式(1)构建目标函数：
    Figure PCTCN2019097101-appb-100004
    式(1)表示N个客户端的QoE之和，即系统的总效用值；式(1)中，
    Figure PCTCN2019097101-appb-100005
    表示质量等级为d的视频块t的码率；
    Figure PCTCN2019097101-appb-100006
    表示视频块t以最高的质量等级D传输时的码率；
    Figure PCTCN2019097101-appb-100007
    表示第n个客户端的FoV内所覆盖的视频块；当
    Figure PCTCN2019097101-appb-100008
    时，表示第t个视频块以第d种码率等级和第m种调制与编码方案通过下行链路传输到客户端，当
    Figure PCTCN2019097101-appb-100009
    时，表示第t个视频块不以第d种码率等级和第m种调制与编码方案通过下行链路传输到客户端；
    步骤2.2、利用式(2)-式(7)构建约束条件：
    Figure PCTCN2019097101-appb-100010
    Figure PCTCN2019097101-appb-100011
    Figure PCTCN2019097101-appb-100012
    Figure PCTCN2019097101-appb-100013
    Figure PCTCN2019097101-appb-100014
    Figure PCTCN2019097101-appb-100015
    式(2)表示任意第c个摄像机只能选择一种码率等级的原始视频上传到服务器；式(2)中，当
    Figure PCTCN2019097101-appb-100016
    时，表示第c个摄像机以第e种码率等级的原始视频上传到服务器，当
    Figure PCTCN2019097101-appb-100017
    时，表示第c个摄像机不以第e种码率等级的原始视频上传到服务器；
    式(3)表示传输的C个视频的总码率不应当超出整个上行信道的总带宽；式(3)中，BW UL表示上行信道的总带宽值；
    式(4)表示任意第t个视频块在质量等级为d通过下行链路传输到客户端时，只能选择一种调制与编码方案；
    式(5)表示任意第t个视频块在以第m种调制与编码方案通过下行链路传输到客户端时，所传输的视频块只能选择一种码率等级；
    式(6)表示传输的所有视频块的总码率不超过整个下行信道中所有资源块所能提供的比特率；式(6)中，
    Figure PCTCN2019097101-appb-100018
    表示当选择第m种调制与编码方案时单个资源块所能提供的比特率；Y DL表示下行信道中所有资源块的总数目；
    式(7)表示所述网络环境中下行链路中任意第t个视频块的码率不大于上行链路中任意第c个摄像头拍摄的原始视频的码率。
  3. 根据权利要求1所述的VR视频自适应采集与传输方法，其特征是，所述步骤三是按如下过程进行：
    步骤3.1、对所述VR视频自适应采集与传输模型中的采集决策变量
    Figure PCTCN2019097101-appb-100019
    和传输决策变量
    Figure PCTCN2019097101-appb-100020
    进行松弛操作，分别得到[0,1]范围内的连续采集决策变量和连续传输决策变量；
    步骤3.2、根据式(2)-式(7)的约束条件，将
    Figure PCTCN2019097101-appb-100021
    记为函数
    Figure PCTCN2019097101-appb-100022
    将
    Figure PCTCN2019097101-appb-100023
    记为函数
    Figure PCTCN2019097101-appb-100024
    将
    Figure PCTCN2019097101-appb-100025
    记为函数
    Figure PCTCN2019097101-appb-100026
    将
    Figure PCTCN2019097101-appb-100027
    记为函数
    Figure PCTCN2019097101-appb-100028
    将
    Figure PCTCN2019097101-appb-100029
    记为函数
    Figure PCTCN2019097101-appb-100030
    将
    Figure PCTCN2019097101-appb-100031
    记为函数
    Figure PCTCN2019097101-appb-100032
    从而利用式(8)计算松弛后的VR视 频自适应采集与传输模型的拉格朗日函数
    Figure PCTCN2019097101-appb-100033
    Figure PCTCN2019097101-appb-100034
    式(8)中，λ表示式(2)-式(7)中等式约束条件的拉格朗日系数，μ表示式(2)-式(7)中不等式约束条件的拉格朗日系数，λ 1表示函数
    Figure PCTCN2019097101-appb-100035
    的拉格朗日系数，λ 2表示函数
    Figure PCTCN2019097101-appb-100036
    的拉格朗日系数，λ 3表示函数
    Figure PCTCN2019097101-appb-100037
    的拉格朗日系数，μ 1表示函数
    Figure PCTCN2019097101-appb-100038
    的拉格朗日系数，μ 2表示函数
    Figure PCTCN2019097101-appb-100039
    的拉格朗日系数，μ 3表示函数
    Figure PCTCN2019097101-appb-100040
    的拉格朗日系数，QoE n表示第n个客户端的质量体验，并有：
    Figure PCTCN2019097101-appb-100041
    步骤3.3、根据式(8)的拉格朗日函数
    Figure PCTCN2019097101-appb-100042
    得到如式(10)-式(15)所示的松弛后的VR视频自适应采集与传输模型的KKT条件：
    Figure PCTCN2019097101-appb-100043
    Figure PCTCN2019097101-appb-100044
    Figure PCTCN2019097101-appb-100045
    Figure PCTCN2019097101-appb-100046
    λ 1,λ 2,λ 3≠0,μ 1,μ 2,μ 3≥0                (14)
    Figure PCTCN2019097101-appb-100047
    对式(10)-式(15)进行求解，得到松弛后的VR视频自适应采集与传输模型的最优解χ relax和最优总效用值Z relax；其中，最优解χ relax包括采集决策变量
    Figure PCTCN2019097101-appb-100048
    和传输决策变量
    Figure PCTCN2019097101-appb-100049
    的松弛最优解；
    步骤3.4、以最优解χ relax和最优总效用值Z relax作为混合分支定界法的初始输入参数；
    步骤3.5、定义所述混合分支定界法中分支次数为k，定义所述混合分支定界法中最优总效用值的下界为L，定义所述混合分支定界法中最优总效用值的上界为U；
    步骤3.6、初始化k＝0；
    步骤3.7、初始化L＝0；
    步骤3.8、初始化U＝Z relax；
    步骤3.9、用χ k表示第k次分支的最优解，并将对应的最优总效用值记为Z k，再将χ relax的值赋给χ k，并以第k次分支的最优解χ k作为根节点；
    步骤3.10、判断χ k中是否存在不符合0-1约束条件的解，若存在，则将χ k中的松弛最优解分为符合0-1约束条件的解和不符合0-1约束条件的解χ k(0,1)，并执行步骤3.12；否则，表示χ k为非松弛VR视频自适应采集与传输模型的最优解；
    步骤3.11、在(0,1)范围内随机产生一个第k次分支的随机数ε k，并判断0＜χ k(0,1)＜ε k是否成立；若成立，则将约束条件“χ k(0,1)＝0”加入到非松弛VR视频自适应采集与传输模型中，形成第k次分支的子分支I；否则，则将约束条件“χ k(0,1)＝1”加入到非松弛VR视频自适应采集与传输模型中，形成第k次分支的子分支II；
    步骤3.12、利用KKT条件求出第k次分支的子分支I和子分支II的松弛解，并作为到第k+1次分支的最优解χ k+1和最优总效用值Z k+1，其中χ k+1包括：第k+1次分支的子分支I和子分支II的松弛解；
    步骤3.13、判断第k+1次分支的最优解χ k+1是否符合0-1约束条件，若是，则从所述最优总效用值Z k+1中找出最大值并赋值给L，且χ k+1∈{0,1}；否则，从所述最优总效用值Z k+1中找出最大值并赋值给U，且χ k+1∈(0,1)；
    步骤3.14、判断Z k+1＜L是否成立；如果成立，则剪掉第k+1次分支的最优解χ k+1所在的分支，并将k+1赋值给k后，返回步骤3.10；否则，执行步骤3.15；
    步骤3.15、判断Z k+1＞L是否成立，如果成立，则将k+1赋值给k后，返回步骤3.10；否则执行步骤3.16；
    步骤3.16、判断Z k+1＝L是否成立，若成立，则表示获得非松弛VR视频自适应采集与传输 模型的最优解即为第k+1分支的最优解χ k+1，并将χ k+1赋值给非松弛VR视频自适应采集与传输模型的最优解χ 0-1，将χ k+1所对应的Z k+1赋值给非松弛VR视频自适应采集与传输模型的最优总效用值Z 0-1；否则，将k+1赋值给k后，返回步骤3.10。
PCT/CN2019/097101 2019-06-28 2019-07-22 一种QoE驱动下的VR视频自适应采集与传输方法 WO2020258427A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/481,211 US11831883B2 (en) 2019-06-28 2021-09-21 QoE-based adaptive acquisition and transmission method for VR video

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910579018.4A CN110266714B (zh) 2019-06-28 2019-06-28 一种QoE驱动下的VR视频自适应采集与传输方法
CN201910579018.4 2019-06-28

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/481,211 Continuation US11831883B2 (en) 2019-06-28 2021-09-21 QoE-based adaptive acquisition and transmission method for VR video

Publications (1)

Publication Number Publication Date
WO2020258427A1 true WO2020258427A1 (zh) 2020-12-30

Family

ID=67923103

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/097101 WO2020258427A1 (zh) 2019-06-28 2019-07-22 一种QoE驱动下的VR视频自适应采集与传输方法

Country Status (3)

Country Link
US (1) US11831883B2 (zh)
CN (1) CN110266714B (zh)
WO (1) WO2020258427A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112822512A (zh) * 2021-01-26 2021-05-18 南京邮电大学 一种基于svc视频文件的分层缓存方法
CN115988267A (zh) * 2022-12-20 2023-04-18 哈尔滨工业大学（深圳） 一种基于dash的视频码率自适应调整方法及系统

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112929691B (zh) * 2021-01-29 2022-06-14 复旦大学 多用户全景视频传输方法
CN113115077B (zh) * 2021-03-12 2022-04-26 上海交通大学 一种静态点云服务器码率自适应传输方法及系统
CN113141526B (zh) * 2021-04-27 2022-06-07 合肥工业大学 QoE驱动下的联合资源分配的点云视频自适应传输方法
CN113329266B (zh) * 2021-06-08 2022-07-05 合肥工业大学 一种基于有限用户视角反馈的全景视频自适应传输方法
CN113395603B (zh) * 2021-06-25 2022-04-01 合肥工业大学 一种基于模型预测控制的点云视频流自适应传输方法
CN113395602B (zh) * 2021-06-25 2022-03-22 合肥工业大学 针对自适应点云视频流媒体中用户体验质量QoE的建模方法
CN114640870B (zh) * 2022-03-21 2023-10-03 陕西师范大学 一种QoE驱动的无线VR视频自适应传输优化方法及系统

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101222296A (zh) * 2008-01-31 2008-07-16 上海交通大学 上行蜂窝视频通信中自适应的传输方法及系统
US20120117225A1 (en) * 2010-10-28 2012-05-10 Avvasi Inc. Methods and apparatus for providing a media stream quality signal
CN102484748A (zh) * 2009-06-16 2012-05-30 高通股份有限公司 管理视频自适应算法
CN103581696A (zh) * 2012-08-10 2014-02-12 英特尔公司 用于内容感知的多媒体流送的方法与系统
CN103888846A (zh) * 2014-03-04 2014-06-25 浙江大学 基于QoE的无线视频流业务自适应速率控制方法
US20140282792A1 (en) * 2013-03-15 2014-09-18 Cygnus Broadband, Inc. Video streaming with buffer occupancy prediction based quality adaptation
CN104869651A (zh) * 2015-03-30 2015-08-26 浙江大学 基于QoE的OFDMA网络下行链路资源分配方法
US9756142B2 (en) * 2013-03-14 2017-09-05 The Regents Of The University Of California System and method for delivering video data from a server in a wireless network by caching the video data
CN108769755A (zh) * 2018-04-27 2018-11-06 湖北工业大学 高分辨率全景视频直播拍照系统与方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180309991A1 (en) * 2016-03-02 2018-10-25 MatrixView, Inc. Video encoding with adaptive rate distortion control by skipping blocks of a lower quality video into a higher quality video
CN105872540A (zh) * 2016-04-26 2016-08-17 乐视控股（北京）有限公司 视频处理的方法及装置
CN106454321A (zh) * 2016-10-26 2017-02-22 深圳市道通智能航空技术有限公司 全景视频的处理方法、装置及系统
US10680927B2 (en) * 2017-08-25 2020-06-09 Advanced Micro Devices, Inc. Adaptive beam assessment to predict available link bandwidth
CN107529064A (zh) 2017-09-04 2017-12-29 北京理工大学 一种基于vr终端反馈的自适应编码方法
CN107995493B (zh) 2017-10-30 2021-03-19 河海大学 一种全景视频的多描述视频编码方法
CN108235131B (zh) 2018-01-30 2020-07-03 重庆邮电大学 一种基于dash的全景视频自适应传输方法

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101222296A (zh) * 2008-01-31 2008-07-16 上海交通大学 上行蜂窝视频通信中自适应的传输方法及系统
CN102484748A (zh) * 2009-06-16 2012-05-30 高通股份有限公司 管理视频自适应算法
US20120117225A1 (en) * 2010-10-28 2012-05-10 Avvasi Inc. Methods and apparatus for providing a media stream quality signal
CN103581696A (zh) * 2012-08-10 2014-02-12 英特尔公司 用于内容感知的多媒体流送的方法与系统
US9756142B2 (en) * 2013-03-14 2017-09-05 The Regents Of The University Of California System and method for delivering video data from a server in a wireless network by caching the video data
US20140282792A1 (en) * 2013-03-15 2014-09-18 Cygnus Broadband, Inc. Video streaming with buffer occupancy prediction based quality adaptation
CN103888846A (zh) * 2014-03-04 2014-06-25 浙江大学 基于QoE的无线视频流业务自适应速率控制方法
CN104869651A (zh) * 2015-03-30 2015-08-26 浙江大学 基于QoE的OFDMA网络下行链路资源分配方法
CN108769755A (zh) * 2018-04-27 2018-11-06 湖北工业大学 高分辨率全景视频直播拍照系统与方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112822512A (zh) * 2021-01-26 2021-05-18 南京邮电大学 一种基于svc视频文件的分层缓存方法
CN115988267A (zh) * 2022-12-20 2023-04-18 哈尔滨工业大学（深圳） 一种基于dash的视频码率自适应调整方法及系统
CN115988267B (zh) * 2022-12-20 2023-09-15 哈尔滨工业大学（深圳） 一种基于dash的视频码率自适应调整方法及系统

Also Published As

Publication number Publication date
US20220006851A1 (en) 2022-01-06
CN110266714A (zh) 2019-09-20
CN110266714B (zh) 2020-04-21
US11831883B2 (en) 2023-11-28

Similar Documents

Publication Publication Date Title
WO2020258427A1 (zh) 一种QoE驱动下的VR视频自适应采集与传输方法
Chakareski et al. Viewport-driven rate-distortion optimized 360º video streaming
Pahalawatta et al. Content-aware resource allocation and packet scheduling for video transmission over wireless networks
CN106331711B (zh) 一种基于网络特征与视频特征的动态码率控制方法
US20040022322A1 (en) Assigning prioritization during encode of independently compressed objects
US20130044183A1 (en) Distributed video coding/decoding method, distributed video coding/decoding apparatus, and transcoding apparatus
CN113329266B (zh) 一种基于有限用户视角反馈的全景视频自适应传输方法
CN108307182A (zh) 一种基于虚拟现实技术的网络交互共享体验系统和方法
Chakareski Uplink scheduling of visual sensors: When view popularity matters
Takeuchi et al. Perceptual quality driven adaptive video coding using JND estimation
CN105430453B (zh) 媒体数据采集方法、媒体终端及在线音乐教学系统
CN106657961B (zh) 立体视频的混合数字-模拟编码
Hu et al. Multiple description coding for best-effort delivery of light field video using GNN-based compression
CN107124603A (zh) 一种基于缓存的多视点视频自适应传输方法
EP3284255B1 (en) Encoding a plurality of signals with data rate target of a signal depending on xomplexity information
KR20210059600A (ko) 라이브 비디오 수집 시스템 및 방법
Guan et al. Cloud-assisted smart camera networks for energy-efficient 3D video streaming
US20110176742A1 (en) Image processing apparatus and image processing method
Mahmoud et al. A Review of Deep Learning Solutions in 360° Video Streaming
Okamoto et al. Edge-assisted multi-user 360-degree video delivery
Moretti et al. Content/context-aware multiple camera selection and video adaptation for the support of m-health services
Gurler et al. Quality of experience aware adaptation strategies for multi-view video over p2p networks
Ozcinar et al. Dynamic bitrate allocation of interactive real-time streamed multi-view video with view-switch prediction
Pervez et al. QoE-based network-centric resource allocation for on-demand uplink adaptive HTTP streaming over LTE network
Hua et al. Multicast scheduling for stereoscopic video in wireless networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19935010

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19935010

Country of ref document: EP

Kind code of ref document: A1