CN108965949A - Meet the code rate adaptive approach of user individual experience in a kind of video traffic - Google Patents

Abstract

The scheme that personalized user is experienced in video traffic is a kind of technology for improving user's viewing experience in video display process.The purpose is to construct a function approximator by one neural network of design, influence of the code rate selection to subsequent video broadcast performance index is predicted, to meet different user experience demands.Its design cycle is 1) to assess: assessing influence of each code rate selection to different first performance indicators using neural network.2) decision: the assessed value to first performance indicator obtained using evaluation process is explicitly multiplied with optimization aim g, selects the corresponding code rate of maximum value.The present invention may be implemented under different optimization aims user experience and maximize, while when the optimization aim of user experience changes, can quick low overhead realization it is extensive on ownership goal.

Description

A Bit Rate Adaptive Method to Satisfy User's Personalized Experience in Video Service

technical field

The invention belongs to the technical field of streaming media video, relates to user experience optimization, and in particular to a code rate self-adaptive method for satisfying user personalized experience in video services.

Background technique

In recent years, video services on the Internet have risen. It is estimated that by 2019, video traffic will account for nearly 80% of the total Internet traffic. The problem of video performance is becoming more and more important, because the performance of video directly affects the experience of users, which in turn affects the length of time users watch videos, and ultimately affects the revenue of content providers. Users expect clearer videos, no stuttering during video playback, and smooth videos with low latency. However, these performance indicators are mutually contradictory and restrict each other. With the emergence of new scenarios and new forms of expression, such as live broadcast scenarios, virtual reality (VR), etc., it becomes more challenging to meet the requirements of user experience.

A tool to describe and quantify user experience and user demand for video is user quality of experience (QoE). Adaptive bit rate (ABR) algorithm is a common method to improve user QoE, which maximizes user experience by selecting an appropriate bit rate for the next video block to be played. User QoE generally includes the following meta-indicators: bit rate, video freeze duration, bit rate switching, and delay. When watching videos, different users and different viewing scenarios have different requirements for various performance indicators of QoE. For example, in the case of live game broadcasting, users prefer to have high-definition video and do not want to experience stuttering, but they have lower requirements for latency. If it is a highly interactive scene, users may have higher requirements for delay, but lower requirements for clarity. Therefore, when facing different users, it is meaningful to provide a method that meets the user's personalized experience needs. Balancing different performance indicators to maximize user experience has become a key point of concern and research in academia and industry.

Contents of the invention

Aiming at the essential problem of improving user experience in the above-mentioned video service and the problem of satisfying the user's personalized experience, the present invention proposes a code rate adaptive method for satisfying the user's personalized experience in the video service, a model with generalization ability, in order to realize The goal of personalizing the user experience in video playback. The present invention is a code rate self-adaptive algorithm based on reinforcement learning, which can select the most suitable code rate in the network scene according to the network environment, optimize various performance indicators in the video service, and meet the user's personalized experience requirements. The performance of this algorithm is better than the previous code rate adaptive algorithm, that is, it provides the best user experience under the condition of specific user QoE target. At the same time, when the user or the playback content changes, the algorithm can generalize user preferences quickly and with low overhead, ultimately improving the user's viewing experience during video playback and maximizing user experience under different optimization goals.

In order to achieve the above object, the technical scheme adopted in the present invention is:

A code rate adaptive method for satisfying user's personalized experience in video service A code rate adaptive method for satisfying user's personalized experience in video service, characterized in that neural network is used as evaluation function Q(s,a,m,g ), evaluate the impact of each code rate selection a on different meta-performance indicators m, and use the evaluation value of the meta-performance indicators obtained in the evaluation process to explicitly multiply the optimization target weight value, that is, the given user preference g, to select the largest The code rate corresponding to the value, so as to meet different user experience requirements, wherein the evaluation function Q(s, a, m, g) represents the situation of each code rate selection a in different network states s and given user preference g Next, how to affect each meta-performance index m.

The input of described evaluation process is made up of state value s and optimized target weight value g, wherein state value s has described the situation of network and buffer zone occupancy situation; Optimized target weight value g represents different user video performance demands;

The output of the evaluation process is the cumulative sum of QoE observations until the end of video playback, and the output is Q _∞ (s, a, m, g), where ∞ represents the end of video playback.

Using the linear combination of meta-performance index m and user preference g to represent user experience QoE, then

Wherein, N is the number of blocks in a video played, R _n is the code rate of the nth block, q(R _n ) is the nth video block quality, T _n is the stall time of the nth block, |q(R _n+1 )-q(R _n )| is the code rate difference between two adjacent blocks during video playback, indicating the smoothness of the video, D _n is the delay of downloading the nth block, α, β, γ, μ is the four terms of the optimization objective g.

The two-part input of the evaluation process is the state value s and the optimization target weight value g, which are processed by two neural networks respectively, and the output of the two modules is connected as the input of the next neural network, and the future QoE value is based on the connected input , the neural network outputs the future observations corresponding to each action at the same time. The neural network is divided into two modules, one is the expectation module, which predicts the average value of the future QoE observations. This part of the value is only related to the state value s and has nothing to do with the action. ; The other is the action module, which predicts the QoE observations corresponding to different actions taken in a certain state. The output of the two parts is added together as the output of the entire neural network, that is, in a certain state, different actions are taken to correspond to different four QoE meta-performance index values at the end of video playback.

When online, the calculation formula for explicitly multiplying the evaluation value of the meta-performance index obtained by the evaluation process with the optimization target weight value g is as follows:

a＝argmaxg ^T Q _∞ (s,a,m,g)

According to the above formula, the optimal code rate under a specific target can be selected. When the product of the Q value and the optimization target g is the largest, the optimal target value is obtained, and the corresponding code rate a is the code rate to be selected for this block. .

When training the neural network model, the randomly generated optimization target weight value g is used. Compared with prior art, the beneficial effect of the present invention is:

The output dimensionality of the neural network increases. The output of traditional reinforcement learning algorithms is a scalar reward value, which represents the reward obtained after taking an action, but the information content of the scalar value is small. The increase of the output dimension enhances the operability of the algorithm. At the same time, different g values can be set to meet the personalized QoE requirements of different users.

Description of drawings

Figure 1 is a model of the evaluation process, in which the input is the state and the optimization target, and the output is the cumulative impact of selecting each code rate on the meta-performance index.

Detailed ways

The implementation of the present invention will be described in detail below in conjunction with the drawings and examples.

The invention is a method for improving user experience in video services, and its goal is to use a model with generalization capability to realize personalized user experience. User QoE generally includes the following meta-indicators: bit rate, video freezing time, bit rate switching, and delay. Different users who watch videos have different requirements for video performance indicators. When there are different video optimization targets, the present invention can perform performance optimization quickly and with low overhead.

Design idea of the present invention is as follows:

(1) Overview of design ideas: designed under the framework of deep reinforcement learning. At the same time, by explicitly introducing user preference g, the evaluation process and decision-making process of ordinary reinforcement learning are decoupled. Using the neural network as the evaluation function Q(s,a,m,g), it represents: the impact of each code rate selection on each meta-performance index m under different network states and given user preferences g, Use this evaluation function to select the code rate of the next block.

(2) Evaluation process: Utilizing the idea of a universal value evaluation function, the goal is to construct a function approximator to predict future meta-performance index values.

Evaluation process input: The input consists of two parts, the state s, and the optimization target weight value g. The state value describes the state of the network and the occupancy of the buffer. g is the weight value corresponding to the optimization target, which represents different preferences of different users for video performance.

Evaluation process output: The output is the QoE observation value at the end of video playback. Divide the traditional reward value Q(s, a) into A action measurement values Q(s, a, m), where A represents the number of optional code rates. The linear combination of the meta-performance index value m and user preference g can be used to represent the user experience QoE, namely

The simple expression is:

QoE＝g ^T Q

Therefore, the QoE of each action under any preference g can be obtained computationally.

Evaluation process model description: The two parts of input are the state and the optimization target, which are processed by two neural networks respectively, and the output connection of the two modules is used as the input of the next layer of neural network. Future QoE observations are based on concatenated inputs. The neural network simultaneously outputs future observations corresponding to each action. The neural network is divided into two modules. One is the expectation module, which predicts the average value of future QoE observations. This part of the value is only related to the state value and has nothing to do with the action; the other is the action module, which predicts in a certain state Next, take different actions corresponding to QoE observations. The output of the two parts is added together as the output of the entire neural network, that is, in a certain state, different actions are taken to correspond to different four QoE meta-performance index values at the end of video playback.

(3) Decision-making process: when online, the algorithm can use the meta-performance indicators (clarity, freeze, smoothness, delay) and optimization goals obtained during the evaluation process at the end of video playback,

a＝argmaxg ^T Q _∞ (s,a,m,g)

Select the optimal bit rate under a certain target according to the above formula.

To sum up, the present invention proposes a code rate adaptive algorithm capable of realizing personalized user experience. By using the neural network to construct a function approximator, it predicts the impact of bit rate selection on subsequent video playback performance indicators, so as to meet different user experience requirements. This solution can select different bit rates according to different playback content, users, and user behaviors to maximize user experience under different optimization goals, and when the optimization goal of user experience changes, it can quickly and cost-effectively achieve user goals Generalization to meet the needs of personalized user experience.

Claims (6)

1. A code rate adaptive method that satisfies user personalized experience in a video service, is characterized in that, utilizes neural network as evaluation function Q (s, a, m, g), evaluates each code rate selection a to different element The impact of the performance index m, the evaluation value of the meta-performance index obtained by the evaluation process, is explicitly multiplied by the optimization target weight value, that is, the given user preference g, and the code rate corresponding to the maximum value is selected to meet different user experiences Requirements, wherein the evaluation function Q(s, a, m, g) represents how each code rate choice a affects each meta-performance index m in the case of different network states s and given user preference g. 2. The code rate adaptive method satisfying user personalized experience in the video service according to claim 1, wherein the input of the evaluation process is composed of a state value s and an optimization target weight value g, wherein the state value s describes The status of the network and the occupancy of the buffer zone; the optimization target weight value g represents different user video performance requirements; The output of the evaluation process is the cumulative sum of QoE observations until the end of video playback, and the output is Q _∞ (s, a, m, g), where ∞ represents the end of video playback. 3. according to the code rate self-adaptive method that satisfies user's personalized experience in the described video service of claim 2, it is characterized in that, represent user experience QoE with the linear combination of meta-performance index m and user preference g, then Wherein, N is the number of blocks in a video played, R _n is the code rate of the nth block, q(R _n ) is the nth video block quality, T _n is the stall time of the nth block, |q(R _n+1 )-q(R _n )| is the code rate difference between two adjacent blocks during video playback, indicating the smoothness of the video, D _n is the delay of downloading the nth block, α, β, γ, μ is the four terms of the optimization objective g. 4. according to the code rate self-adaptive method that satisfies user's personalized experience in the described video service of claim 2, it is characterized in that, the two parts input of described assessment process are state value s and optimization target weight value g, are divided by two respectively Neural network processing, the output connection of two modules is used as the input of the next neural network, the future QoE value is based on the connected input, and the neural network outputs the future observation values corresponding to each action at the same time, the neural network is divided into two modules, one is The expectation module predicts the average value of future QoE observations. This part of the value is only related to the state value s and has nothing to do with the action; the other is the action module, which predicts the QoE corresponding to taking different actions in a certain state Observations. The output of the two parts is added together as the output of the entire neural network, that is, in a certain state, different actions are taken to correspond to different four QoE meta-performance index values at the end of video playback. 5. according to the code rate self-adaptive method that satisfies user's personalized experience in the described video service of claim 2, it is characterized in that, when online, utilize the evaluation value of the meta-performance index that evaluation process obtains, and optimize target weight value g to show The calculation formula for multiplication is as follows: a＝argmaxg ^T Q _∞ (s,a,m,g) According to the above formula, the optimal code rate under a specific target can be selected. When the product of the Q value and the optimization target g is the largest, the optimal target value is obtained, and the corresponding code rate a is the code rate to be selected for this block. . 6. According to claim 2, the code rate adaptive method satisfying the user's personalized experience in the video service, is characterized in that, when training the neural network model, the optimization target weight value g generated randomly is utilized.