CN116828269A - Long-term code rate planning method for video transmission - Google Patents

Abstract

In a video transmission system, a client firstly predicts an average network bandwidth in a determined time scale, calculates bandwidth capacity according to the time scale and the bandwidth, then obtains a video block set consisting of a plurality of selectable video blocks with different code rate levels according to a current buffer zone, finds a video block combination which can be downloaded in the time scale and has the maximum utility function value from the selectable video block set, and selects the block number of the first video block of the video block combination and the code rate thereof to send a request to a server, thereby realizing long-term optimal code rate planning. The experimental results show that: in some cases, compared with the current code rate selection algorithm, the method provided by the patent can better improve the user experience quality.

Description

Long-term code rate planning method for video transmission

Technical Field

The invention relates to the field of streaming media transmission, in particular to a long-term code rate planning method for video transmission.

Background

With the rapid development of video streaming, in order to meet the requirement of users on video quality and adapt to changeable network environments, video providers mostly employ a rate selection algorithm (Bitrate Selection Algorithm) to optimize the video quality of experience (QoE, quality of Experience) of users, and clients select appropriate rates from video blocks with different quality levels according to current network conditions. Currently, commonly used code rate selection algorithms generally determine the video code rate based on perceived system conditions such as buffer occupancy and predicted bandwidth. They respond instantaneously to instantaneous bandwidth changes and provide locally optimal bit rate selection algorithms. However, they ignore the problem of inconsistent time scales and inconsistent evaluation metrics in selecting code rates.

In some classical rate selection algorithms, on the one hand, algorithms based on advanced video coding (AVC, advanced Video Coding), such as Buffer-based optimized linear algorithm (BOLA: buffer-based Optimized Linear Algorithm) or block-based inverse adaptive rate adjustment algorithm (BACKFILLING: backward Adaptive Chunk-based Rate Adaptation and Playout Buffer Filling) based on scalable video coding (SVC, scalable Video Coding), determine the next video data to be requested based on the download time of the next video block or video layer only. On the other hand, a dynamic adaptive video stream control theory algorithm based on HTTP (MPC: multi-Player Compatibility) and a prospective online bit rate adaptation algorithm oriented to DASH (MSPC: multi-Stream Player Compatibility) are prospective algorithms that consider the bit rates of the next N blocks to obtain the optimal bit rate of the next requested block. Thus, the timescale of the MPC and MSPC is the download time of the next N blocks. These algorithms as described above ignore differences in video data size, resulting in inconsistent time scales. This results in inconsistent scale of evaluation of the bit rate selection or matched bandwidth capacity.

In addition, most existing rate selection algorithms use the optimization objective pursued by the method as an evaluation metric when evaluating the requested video data. For example, BOLA and backfill select video data based on buffer status, while BOLA-FAST and cursor evaluate video by switching video bit rate. The MPC selects the bit rate to maximize quality of experience (QoE) for the next 5 blocks, while the MSPC combines the average bit rate switching variation and the buffer occupancy as an evaluation function of the bit rate selection. However, these rate selection algorithms lack a unified video evaluation metric.

Disclosure of Invention

The invention provides a method capable of realizing long-term code rate planning in order to solve the problem of local optimum decision caused by different time scales of code rate selection performance evaluation in the existing code rate selection algorithm.

In order to achieve the technical purpose, the technical scheme of the invention is as follows:

a global optimal code rate planning method for video transmission comprises the following steps:

step one: the client firstly determines a time scale, predicts the average bandwidth in the time scale, and calculates the bandwidth capacity according to the time scale and the predicted average bandwidth;

step two: then obtaining an optional video block set according to the current buffer zone;

step three: finding all video block combinations which can be downloaded in a time scale from the selectable video block set through a search algorithm;

step four: and calculating the utility function value of each video block combination, selecting the first video block of the video block combination with the largest utility function value and the code rate thereof as an optimal decision, and constructing a request for downloading the video blocks to a server according to the optimal decision.

In the first step, the client determines the time scale TS according to the following formula:

where k represents the number of blocks of video blocks that the current client has downloaded, D (i) represents the size of the video block that was downloaded the i-th time, C (i) represents the network bandwidth used in downloading the video block the i-th time,representing the time of the ith download of the video block,representing the downloading of past n video blocksAnd (5) summing the time.

The method, in the first step, the average bandwidth C in the time scale TS is expressed by the following formula _TS Predicted value:

wherein D (i) represents the size of the video block that the client has downloaded the ith time,representing the sum of the sizes of n video blocks downloaded in the past.

In the method, in the first step, the bandwidth capacity BC in the future TS time is estimated by the following equation:

BC＝TS*C _TS

in the second step, the process of the client obtaining the optional video block set according to the current buffer zone is as follows:

step A1, assuming that the video is divided into n video blocks, each video block has m code rate levels, then all video blocks V of the video played by the current client are represented by the following formula _set ：

V _set ＝{v _1,1 ,v _1,2 ,...v _1,m ,v _2,1 ,...v _n,m }

wherein ,v_i,j Representing an ith video block having a j-level code rate;

step A2, assume that the existing video block in the current client buffer is V _buffer ：

Step A3, V _buffer Video block set V capable of improving video block code rate level _update Is represented by the following formula:

V _update ＝{v _i,k |q≤i≤q+j,l _i-q <k≤m}

step A4, the client does not download the video block set V with any code rate level _new Can be expressed by the following formula:

V _new ＝{v _i,k |q+j<i≤n,1≤k≤m}

step A5, all video block sets V which can be selectively downloaded _selectable ＝V _update ∪V _new Is marked asWherein s is V _update ∪V _new Total number of elements of the collection->Is represented as having m _i Coding rate level a _i Number video block, i is an integer from 1 to s.

In the third step, the process of finding all the video block combinations that can be downloaded in the time scale from the selectable video block set by the search algorithm is as follows:

is provided withx _i ∈{0,1}，/>i.e {1, 2..s }, due to x _i Is different in value, and has total value of 2 ^s A different video block combination V, i.e. +.>From this 2 ^s All combinations satisfying the following conditions are found out among the different combinations of video blocks:

wherein ,representing video block->Data size of->Representing a request to download a video blockTime urgency of (2); the video block combination satisfying the condition is marked +.>Combining these video blocksIs a video block combination of each of (a)>C in (c) _i The elements are arranged completely to obtain video block combinations with different sequences> wherein c_i The following is carried out Representation c _i Is thus possible to obtain all video block combinations +.>

The method requests to download video blocksTime urgency of->The calculation is as follows:

wherein h represents the maximum block number in the video block downloaded by the client, b (h) represents the size of the buffer zone when the client finishes downloading the video block of h, L represents the playing time of one video block, b (h)/L represents the number of video blocks in the buffer zone, h-b (h)/L represents the number of video blocks played by the client, (a) _i - (h-b (h)/L)). L represents that the current client plays to the a-th _i The duration of the numbered video block,representing video block->D (h) represents the data size of video block h.

In the fourth step, the process of calculating the utility function value of each video block combination and selecting the video block combination with the largest utility function value is as follows:

step A1, calculating by the following formulaIs combined +.>Utility function value of->

Wherein the client will download the video block combination during the TS periodAnd playing the video blocks from the No. q to the No. q+p-1 in the buffer area, wherein r (i) represents the code rate level of the No. i video block played by the client, and the video blocks are +.>Representing the sum of the code rates of p video blocks played by the client within the time scale TS,/for>Representing the code rate switching value, T, between the client playing p video blocks within a time scale TS _i Representing the length of the client's click when downloading video block number i,QoE, R representing playing video blocks within TS time scale _max For the maximum code rate level in the video code rate set, AD represents the average data size of all data blocks of the whole video to be played, X _req For the video block data size downloaded and played within the TS,/sized>The influence evaluation value of the current decision on the future decision is represented, and alpha, beta and gamma are parameters of corresponding influence factor weights respectively;

step A2, video block combinationThe corresponding utility function value isSelecting +.>Corresponding video block combination->And sending a request to the server by taking the block number and the code rate of the first video block of the video block combination as optimal decisions.

The method causes playing card pause time T when the ith video block is downloaded _i Can be expressed as:

wherein b (i-1) represents the buffer size when the i-1 th video block is downloaded, D (i) represents the data size of the i-1 th video block,representing the download time of video block number i.

The method has the technical effects that the optimal utility value planning video combination is found according to the buffer occupancy rate and the bandwidth capacity on the predicted future time scale, so that long-term optimal code rate planning is realized.

The invention is further described below with reference to the accompanying drawings.

Drawings

FIG. 1 is a general frame diagram of the present invention;

FIG. 2 is a graph of performance change of a code rate planning algorithm popularization;

fig. 3 is a graph of QoE variation for GPB-SVC with different time scales;

fig. 4 is a diagram of buffer change of GPB-SVC with different time scales.

Detailed Description

As shown in fig. 1, in this embodiment, the client predicts the average network bandwidth C over a determined time scale TS _TS And calculates the bandwidth capacity BC according to the time scale and the bandwidth, and then obtains the optional video block set V according to the current buffer zone _selectable All video block combinations which can be downloaded in a time scale are found from the selectable video block set through a search algorithm, and each video is calculatedAnd the utility function value of the block combination is selected, the first video block of the video block combination with the largest utility function value and the code rate thereof are used as optimal decisions, and a request for downloading the video blocks is constructed and sent to the server according to the optimal decisions.

Step one: client predicts average network bandwidth C over a determined time scale TS _TS And calculates the bandwidth capacity BC based on the time scale and the bandwidth.

BC＝TS*C _TS

Where k represents the number of blocks of video blocks that the current client has downloaded, D (i) represents the size of the video block that was downloaded the i-th time, C (i) represents the network bandwidth used in downloading the video block the i-th time,representing the time of the ith download of the video block,representing the sum of the download times of the past n video blocks; />Representing the sum of the sizes of n video blocks downloaded in the past.

Step two: the client obtains a selectable video block set V according to the current buffer zone _selectable 。

V _update ＝{v _i,k |q≤i≤q+j,l _i-q <k≤m}

V _new ＝{v _i,k |q+j<i≤n,1≤k≤m}

V _selectable ＝V _update ∪V _new

Wherein, the video is assumed to be divided into n video blocks, each video block having m code rate levels, v _i,j Representing video block number i with j-level code rate, V _buffer V for the video block already in the current client buffer _update Represents V _buffer Video block set capable of improving video block code rate level, V _new Representing a set of video blocks not downloading any code rate level, will V _selectable Is marked ass is V _update ∪V _new Total number of elements of the collection.

Step three: the client finds all video block combinations that can be downloaded in a time scale from the selectable video block set by a search algorithm.

Is provided withx _i ∈{0,1}，/>i.e {1, 2..s }, due to x _i Is different in value, and has total value of 2 ^s A different video block combination V, i.e. +.>From this 2 ^s All combinations satisfying the following conditions are found out among the different combinations of video blocks:

representing a request to download a video block->Is calculated by the following formula:

wherein h represents the maximum block number in the video block downloaded by the client, b (h) represents the size of the buffer zone when the client finishes downloading the video block of h, L represents the playing time of one video block, b (h)/L represents the number of video blocks in the buffer zone, h-b (h)/L represents the number of video blocks played by the client, (a) _i - (h-b (h)/L)). L represents that the current client plays to the a-th _i The duration of the numbered video block,representing video block->Download time of->Representing video block->Is a data size of (a) a data size of (b).

Video block combinations meeting the conditions are noted asCombining these video blocks->Is a video block combination of each of (a)>C in (c) _i The elements are arranged completely to obtain video block combinations with different sequencesc _i The following is carried out Representation c _i Whereby all video block combinations that are downloadable in the time scale are available

Step four: the client calculates the utility function value of each video block combination and selects the first video block of the video block combination with the largest utility function value and the code rate thereof as the optimal decision.

The specific operation of the client in the fourth step is as follows:

step A1, calculating by the following formulaIs combined +.>Utility function value of->

T _i The method is characterized in that when the client side downloads the video block No. i, the client side calculates according to the following formula:

wherein b (i-1) represents the buffer size when the i-1 th video block is downloaded, D (i) represents the data size of the i-1 th video block,representing the download time of video block number i.

The client will download the video block combination during the TS periodAnd playing the video blocks from the No. q to the No. q+p-1 in the buffer area, wherein r (i) represents the code rate level of the No. i video block played by the client, and the video blocks are +.>Representing the sum of the code rates of p video blocks played by the client within the time scale TS,/for>Representing the code rate switching value, T, between the client playing p video blocks within a time scale TS _i Indicating the blocking time length of the client when the i-th video block is downloaded, and +.> QoE, R representing playing video blocks within TS time scale _max For the maximum code rate level in the video code rate set, AD represents the average data size of all data blocks of the whole video to be played, X _req For the video block data size downloaded and played within the TS,/sized>The influence evaluation value of the current decision on the future decision is represented, and alpha, beta and gamma are parameters of corresponding influence factor weights respectively;

step A2, video block compositionThe corresponding utility function value isSelecting +.>Corresponding video block combination->And sending a request to the server by taking the block number and the code rate of the first video block of the video block combination as optimal decisions.

Fig. 2 illustrates the obtained video block bit rate cases before and after generalization using the global optimum rate selection algorithm (GPB Global Optimal Planning Bitrate Selection Al gorithm) according to the BACKFILLING and Content aware based on-line video stream slide rate switching Utility optimization algorithm (CURSOR Content-aware Utility-based Rate Adaptation with Smooth Rate Switching for Online Video Streaming). The number of video blocks of the video is 29.

In fig. 2 (a), the video block bit rates obtained by the BACKFILLING and GPB-BACKFILLING are identical, and in fig. 2 (b), the video block bit rates obtained by the CURSOR and GPB-CURSOR deviate only in the bit rates of the last few video blocks. Experimental results show that the performance of the GPB generalized algorithm is substantially similar to that of the original algorithm.

Fig. 3 shows QoE at different time scales and different bandwidths, the link bandwidths being low, medium, high, respectively. The time scales TS are 2s,4s,6s,8s,10s, respectively.

In a low bandwidth scenario, increasing the time scale from 2 seconds to 10 seconds would result in a 90.6% increase in QoE. Also, in medium and high bandwidth scenarios, qoE at 10s is 30.2% and 8.4% higher than at 2s, respectively. Experimental results show that increasing the time scale significantly improves the QoE of the methods of the present patent.

Fig. 4 shows the change in the buffer over a period of time for using different time scales. The time period is 0-100 s, and the time scale TS is 2s,4s,6s,8s and 10s respectively.

Initially, the buffers were all 2 seconds, and within 100 seconds the buffer representing the most of the time of the purple polyline of the largest time scale 10s was above the other polylines, while the buffer representing the most of the time of the blue polyline of the smallest time scale 2s was below the other polylines. Experimental results indicate that a larger time scale is more likely to maintain a higher buffer occupancy, which can better cope with possible bandwidth jitter and select a higher bit rate for subsequent video blocks.

Claims (9)

1. The global optimal code rate planning method for video transmission is characterized by comprising the following steps of:

step one: the client firstly determines a time scale, predicts the average bandwidth in the time scale, and calculates the bandwidth capacity according to the time scale and the predicted average bandwidth;

step two: then obtaining an optional video block set according to the current buffer zone;

step three: finding all video block combinations which can be downloaded in a time scale from the selectable video block set through a search algorithm;

step four: and calculating the utility function value of each video block combination, selecting the first video block of the video block combination with the largest utility function value and the code rate thereof as an optimal decision, and constructing a request for downloading the video blocks to a server according to the optimal decision.

2. The method of claim 1, wherein in the first step, the client determines the time scale TS by the following formula:

where k represents the number of blocks of video blocks that the current client has downloaded, D (i) represents the size of the video block that was downloaded the i-th time, and C (i) represents the video block that was downloaded the i-th timeThe network bandwidth used in the process is,representing the time of the ith download of the video block,representing the sum of the download times of the past n video blocks.

3. The method according to claim 1, wherein in the first step, the average bandwidth C in the time scale TS is expressed by the following formula _TS Predicted value:

wherein D (i) represents the size of the video block that the client has downloaded the ith time,representing the sum of the sizes of n video blocks downloaded in the past.

4. The method of claim 1 wherein in step one, the bandwidth capacity BC in the future TS time is estimated by:

BC＝TS*C _Ts

5. the method according to claim 1, wherein in the second step, the client obtains the optional video block set according to the current buffer as follows:

step A1, assuming that the video is divided into n video blocks, each video block has m code rate levels, then all video blocks V of the video played by the current client are represented by the following formula _set ：

V _set ＝{v _1,1 ，v _1,2 ，...v _1，m ，v _2，1 ，...v _n，m }

wherein ,v_i,j Representing an ith video block having a j-level code rate;

step A2, assume that the existing video block in the current client buffer is V _buffer ：

Step A3, V _buffer Video block set V capable of improving video block code rate level _update Is represented by the following formula:

V _update ＝{v _i，k |q≤i≤q+j，l _i-q ＜k≤m}

step A4, the client does not download the video block set V with any code rate level _new Can be expressed by the following formula:

V _new ＝{v _i，k |q+j＜i≤n，1≤k≤m}

step A5, all video block sets V which can be selectively downloaded _selectable ＝V _update ∪V _new Is marked asWherein s is V _update ∪V _new Total number of elements of the collection->Is represented as having m _i Coding rate level a _i Number video block, i is an integer from 1 to s.

6. The method according to claim 1, wherein in the third step, the process of finding all the video block combinations that can be downloaded in the time scale from the selectable video block set by the search algorithm is as follows:

is provided withDue to x _i Is different in value, and has total value of 2 ^s A different video block combination V, i.e. +.>From this 2 ^s All combinations satisfying the following conditions are found out among the different combinations of video blocks:

wherein ,representing video block->Data size of->Representing a request to download a video block->Time urgency of (2); the video block combination satisfying the condition is marked +.>Combining these video blocksIs a video block combination of each of (a)>C in (c) _i The elements are arranged completely to obtain video block combinations with different sequences> wherein c_i The following is carried out Representation c _i Is thus possible to obtain all video block combinations +.>

7. The method of claim 6, wherein downloading of video blocks is requestedTime urgency of (2)The calculation is as follows:

wherein h represents the maximum block number in the video block downloaded by the client, b (h) represents the size of the buffer zone when the client finishes downloading the video block of h, L represents the playing time of one video block, b (h)/L represents the number of video blocks in the buffer zone, h-b (h)/L represents the number of video blocks played by the client, (a) _i - (h-b (h)/L)). L represents that the current client plays to the a-th _i The duration of the numbered video block,representing video block->D (h) represents the data size of video block h.

8. The method of claim 1, wherein in the fourth step, the process of calculating the utility function value of each video block combination and selecting the video block combination with the largest utility function value is:

step A1, calculating by the following formulaEach video block of (a)Combination->Utility function value of->

Wherein the client will download the video block combination during the TS periodAnd playing the video blocks from the No. q to the No. q+p-1 in the buffer area, wherein r (i) represents the code rate level of the No. i video block played by the client, and the video blocks are +.>Representing the sum of the code rates of p video blocks played by the client within the time scale TS,/for>Representing the code rate switching value, T, between the client playing p video blocks within a time scale TS _i Representing the length of the client's click when downloading video block number i,QoE, R representing playing video blocks within TS time scale _max For the maximum code rate level in the video code rate set, AD represents the average data size of all data blocks of the whole video to be played, X _req For the video block data size downloaded and played within the TS,/sized>Representing the current blockThe influence evaluation value of the strategy on future decisions, alpha, beta and gamma are parameters of corresponding influence factor weights respectively;

step A2, video block combinationThe corresponding utility function value isSelecting +.>Corresponding video block combination->And sending a request to the server by taking the block number and the code rate of the first video block of the video block combination as optimal decisions.

9. The method of claim 8, wherein a playback pause time T is created when downloading video block No. i _i Can be expressed as:

wherein b (i-1) represents the buffer size when the i-1 th video block is downloaded, D (i) represents the data size of the i-1 th video block,representing the download time of video block number i.