WO2023125970A1

WO2023125970A1 - Code rate allocation method and apparatus, storage method and apparatus, device, and storage medium

Info

Publication number: WO2023125970A1
Application number: PCT/CN2022/144132
Authority: WO
Inventors: 俞义方; 华新海; 曹洋; 李瑾熙; 朱佳欣; 江涛
Original assignee: 中兴通讯股份有限公司
Priority date: 2021-12-30
Filing date: 2022-12-30
Publication date: 2023-07-06
Also published as: CN116419016A

Abstract

The present application discloses a code rate allocation method and apparatus, a storage method and apparatus, a device, and a storage medium. The code rate allocation method comprises: obtaining target video data, the target video data comprising viewing probabilities of all video clips in a target transmission video (S100); and performing code rate allocation on the video clips according to the viewing probabilities, so that the total utility of the target transmission video satisfies a target value (S200). The total utility of the target transmission video is determined according to the user watching quality and transmission loss of the video clips, the user watching quality is determined according to non-freshness factors, and the non-freshness factors are determined according to the storage state and the viewing probabilities of the video clips in edge nodes.

Description

A code rate allocation method, storage method, device, equipment and storage medium

Cross References to Related Applications

This application is based on a Chinese patent application with application number 202111658162.0 and a filing date of December 30, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated by reference into this application.

technical field

The present application relates to the communication field, for example, to a code rate allocation method, storage method, device, equipment and storage medium.

Background technique

Based on the good support of 5G network for high-definition video, high-definition video is gradually applied in many industries such as surveillance security, remote conference, e-commerce live broadcast, VR game/video, etc., and is favored by content providers and consumers. The transmission process of high-definition video needs to consume a lot of bandwidth and storage resources. Usually, the storage of high-definition video is performed by the server, resulting in a large loss in the transmission process. Therefore, edge nodes closer to users can be used to cache high-definition video. The user can directly obtain the cached high-definition video from the edge node instead of obtaining it from the server, which reduces the loss in the transmission process. However, when the time for storing high-definition video at the edge node increases, the value of the high-definition video to users will decrease. Under the condition of limited bandwidth, if the high-definition video maintains the original bit rate, it will affect other high-definition videos, resulting in an increase in high-definition The overall transmission loss of the video and respond to the user viewing experience.

Contents of the invention

The main purpose of the embodiments of the present application is to provide a code rate allocation method, a storage method, a device, a device, and a storage medium that reduce transmission loss.

In order to achieve the above purpose at least to a certain extent, an embodiment of the present application provides a code rate allocation method, including: acquiring target video data; the target video data includes viewing probabilities of all video segments in the target transmission video; according to the Viewing probability performs code rate allocation to the video segment, so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment, and the The viewing quality of the user is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.

The embodiment of the present application also proposes a storage method, including: obtaining the viewing probability and code rate allocation results of all video clips in the target transmission video through the code rate allocation method; according to the code rate allocation results and the viewing Probability, predicting the expected effective quality and expected transmission loss; according to the expected effective quality, the expected transmission loss and the maximum capacity of the edge node, perform storage optimization processing to obtain storage decision information; the storage decision information includes each of the A third storage decision value of the video clip, where the third storage decision value represents whether the video clip needs to be stored in an edge node; and the video clip is stored according to the storage decision information.

The embodiment of the present application also proposes a code rate allocation device, including: an acquisition module, configured to acquire target video data; the target video data includes viewing probabilities of all video segments in the target transmission video; a processing module, configured to According to the viewing probability, the code rate is allocated to the video segment, so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment, so The user's viewing quality is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.

The embodiment of the present application also proposes a storage device, including: a determination module, configured to determine viewing probabilities and code rate allocation results of all video segments in the target transmission video; The viewing probability is predicted to predict the expected effective quality and the expected transmission loss; the optimization module is used to perform storage optimization processing according to the expected effective quality, the expected transmission loss and the maximum capacity of the edge node to obtain storage decision information; the storage The decision information includes a third storage decision value for each of the video clips, the third storage decision value representing whether the video clip needs to be stored in an edge node; a storage module configured to perform the video processing according to the storage decision information. The storage of segment; The viewing probability and code rate distribution result of all video clips in the described determination target transmission video, comprise: obtain target video data; Described target video data comprises the viewing probability of all video clips in target transmission video; According to the Viewing probability performs code rate allocation to the video segment, so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment, and the The viewing quality of the user is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.

The embodiment of the present application also provides an electronic device, the electronic device includes a processor and a memory, the memory stores at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the At least one segment of program, the code set or instruction set is loaded and executed by the processor to implement the code rate allocation method or storage method described above.

The embodiment of the present application also provides a computer-readable storage medium, the storage medium stores at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the The code set or instruction set is loaded and executed by the processor to implement the above code rate allocation method or storage method.

Description of drawings

Fig. 1 is a schematic flow chart of the steps of the code rate allocation method of the present application;

Fig. 2 is a schematic flow chart of the steps of the storage method in a specific embodiment of the present application;

FIG. 3 is a schematic flow diagram of a code rate allocation method and a storage method in one application scenario of a specific embodiment of the present application;

FIG. 4 is a schematic flowchart of a code rate allocation method and a storage method in another application scenario according to a specific embodiment of the present application;

FIG. 5 is a schematic diagram of a code rate allocation device according to a specific embodiment of the present application;

FIG. 6 is a schematic diagram of a storage device according to a specific embodiment of the present application;

FIG. 7 is a schematic diagram of an electronic device according to a specific embodiment of the present application.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

In the subsequent description, use of suffixes such as 'module', 'part' or 'unit' for denoting elements is only for facilitating the description of the present application and has no specific meaning by itself. Therefore, 'module', 'part' or 'unit' may be used in combination.

Embodiment one

As shown in Figure 1, the embodiment of the present application provides a code rate allocation method, at least including but not limited to steps S100-S200:

S100. Obtain target video data.

In this embodiment of the present application, the target video data includes viewing probabilities of all video segments in the target transmission video. In an example, the target video data is the data of the video that the user needs to watch, that is, the relevant data of the target transmission video, which can be obtained by the user end by generating a video request based on the user's input instruction. It should be noted that the cloud (or server side) can divide the target transmission video into blocks to generate multiple video clips in different spatial positions. In an example, the target transmission video with a duration of T is encoded into K quality levels in the cloud, and the video of each quality level is temporally divided into video segments with a length of Δt seconds (including but not limited to 2 seconds) , each video segment with a length of Δt seconds is spatially divided into M×N video segments, where M and N are the number of horizontal and vertical video segments respectively. In the embodiment of the present application, in order to obtain the region of interest (Region of Interest, RoI) of the user on the video segment, the viewing probability of each video segment can be determined through the user's attention point track, and the sum of the viewing probabilities located in the user's RoI is 1, which satisfies

m,n is the index position of the video segment, p(m,n) is the viewing probability (probability within the user RoI) corresponding to the video segment (m,n), the higher the viewing probability, the greater the user's attention. It should be noted that when the target transmission video is immersive video, such as VR (Virtual Reality, virtual reality) video and AR (Augmented Reality, augmented reality) video, the user can see 360-degree video content by wearing a head-mounted display device , at this time, the RoI area is the user's actual field of view (Field of View, FoV) area in the head-mounted display, and the viewing probability can be determined through the eye movement trajectory; and when the target transmission video is other types of video, it can be passed Acquire real-time operation area frequency of the mouse or predict the importance of the user's possible attention position through video image processing. For example, use the Saliency map (saliency map) to predict the salient area of the video, and judge the importance of the viewing area through the gray value ratio of different areas to determine the viewing probability.

S200. Perform code rate allocation on the video segment according to the viewing probability, so that the total utility of the video segment meets the target value.

In the embodiment of this application, code rate allocation refers to determining the target transmission code rate of each video segment, that is, the code rate at which the edge node transmits each video segment to the user end; the total utility of the target video transmission is based on the user viewing quality of the video segment and the transmission loss is determined. In one example, the total utility of the target transmission video is characterized by a total utility function:

Among them, Utility _t is the total utility at time t, that is, the utility of the code rate distribution system at time t, QoP _t (m,n) is the user viewing quality of video segment (m,n) at time t, TC _t (m,n) is the transmission loss of the video segment (m,n) at time t, and δ is the weight of the transmission loss. It can be seen from the formula that the utility of each video segment can be obtained by subtracting the product of the transmission loss and the weight of the transmission loss from the user viewing quality corresponding to each video segment, and the sum of the utilities of all video segments is the total utility of the target transmission video . In one example, if a video segment is stored in the cloud (cloud server), the transmission loss of a video segment includes the communication loss of the edge node obtaining the video segment from the cloud and the communication loss of the edge node transmitting the video segment to the user end; if a The video clips are stored in the edge node, and the transmission loss of the video clip includes the communication loss of the edge node transmitting the video clip to the client, or also includes the transcoding loss of the edge node. It should be noted that the total utility of the target transmission video satisfies the target value, including but not limited to making the total utility function reach the maximum value (MAX Utility _t ); or, first setting the total utility target threshold, when the total If the utility is greater than or equal to the total utility target threshold, it is considered to meet the target value. When the total utility is determined to meet the target value, it can improve the user viewing quality of the target video transmission and reduce the transmission and communication loss of video clips obtained from the cloud. Under certain conditions, users can watch the quality of the target transmission video.

In the embodiment of the present application, an example is used to satisfy the target value so that the total utility function is the maximum value, and the code rate allocation problem can be described as:

Among them, B _max is the maximum bandwidth transmitted between the edge node and the user end,

is the code rate at time t, in an example, it is the transmission code rate allocated for video segment (m, n) at time t for the edge node to transmit video segment (m, n) to the user end, c _t-1 (m , n) is the storage decision value at time t-1. When the storage decision value is 1, it means that the video segment (m, n) is stored in the edge node, and when the storage decision value is 0, it means that the video segment (m, n) is not stored in edge node;

is the code rate at time t-1, in an example, it is the storage code rate when the video segment (m,n) is stored by the edge node at time t-1. In the embodiment of this application, when calculating the total utility, the bandwidth constraints limit all

The sum cannot exceed the maximum bandwidth B _max ; for the video clips already stored in the edge node (i.e. c _t-1 (m,n)=1), the code rate constraints stipulate that the code allocated to the stored video clips at time t rate cannot be higher than

In this way, users can directly obtain video clips from edge nodes, reducing the additional communication loss caused by obtaining video clips from the cloud. in,

is the quality level assigned to the video segment (m,n) at time t, q _K is the Kth quality level, which can be measured by the ffprobe tool (a tool that can be used to view file format information in FFmpeg), and the quality level is related to the code of the video segment The rate is closely related to the content complexity o _t (m,j), namely

F() represents a mapping relationship. It should be noted that when the total utility is calculated by formula (2), the calculation complexity of the integer programming problem is O(K ^{M N} ), and then the target quality level of each video segment can be determined to solve the maximum total utility, while the present application In the embodiment, when the computational complexity of the integer programming problem is O(K ^{M N} ), the execution time is too large, so the greedy algorithm is used to obtain the optimal solution of the approximate solution to reduce the computational complexity, so that the total utility satisfies Determine the target quality level of each video segment and the corresponding target transmission bit rate based on the target value to realize bit rate allocation.

In an example, in the embodiment of the present application, in order to determine that the total utility function of formula (2) is the maximum value, in step S200, the code rate is allocated to the video segment according to the viewing probability, including steps B201 and B202, and B203 and/or B204 :

B201. Obtain the new utility value and the old utility value of the total utility of the target transmission video, and acquire the quality level and bit rate of the video segment.

In the embodiment of the present application, the new utility value is the initial value of the total utility of the target transmission video, that is, the initial value of the total utility function in formula (1), and the initial value can be a randomly determined value or a set value, denoted as newU _t , eg newU _t =0. The old utility value may be the current value of the total utility of the target video transmission determined according to the quality level and bit rate, which is denoted as oldU _t . Taking the current moment as the moment t, the quality level and the code rate can be the quality level and the code rate currently assigned to the video segment, or both can be set values, and in the embodiment of the present application, the quality level is used as the set value as an example. description, without specific limitation. In one example, the quality level of each video segment (m, n) is set to the lowest quality level 1, denoted as

A matrix of quality levels constituting all video clips (m,n)

And the corresponding code rate can be determined through the quality level or the initial code rate can be set as

So as to get the code rate matrix of all video clips (m,n)

Thus oldU _t can be calculated by substituting it into formula (1) or (2).

B202. Determine whether the sum of code rates is less than or equal to the bandwidth capacity.

In one example, it is judged whether the sum of the code rates of all video clips is less than or equal to the bandwidth capacity, that is, it is judged

Whether it is established.

B203, when the sum of the code rates is less than or equal to the bandwidth capacity, traverse the video segment to increase the quality level, according to the increase processing result, new utility value and old utility value, update the quality level and the code rate of the video segment, and return to determine the code rate The step of whether the sum of the sum is less than or equal to the bandwidth capacity, until the sum of the code rate is greater than the bandwidth capacity, to obtain the target transmission code rate of each video segment.

In the embodiment of this application, when the sum of the code rates of all video clips

The quality level is incremented by traversing the video segments. For example, the Nth video clip in the Mth column in the video clip matrix composed of M×N video clips is used as the first video clip to start the traversal process until the first video clip in the first column is traversed. Increased processing of the quality level of the video segment, for example, the quality level of the video segment

After the increase processing, the result of the increase processing is obtained

It is equivalent to an increased quality level; wherein, the increased value can be adjusted as required, and the present application uses 1 as an example without constituting a specific limit to the increased value. Then, after increasing processing, according to increasing processing result, new utility value and old utility value, update the quality level and code rate of video segment, return to step B202 until

Thus, the target transmission bit rate of each video segment is obtained.

B204, when the sum of the code rates is not less than or not equal to the bandwidth capacity, traverse the video segment to reduce the quality level, and update the quality level and the code rate of the video segment according to the reduction processing result, new utility value and old utility value, and return to judge The step of whether the sum of the code rates is less than or equal to the bandwidth capacity, until the sum of the code rates is less than or equal to the bandwidth capacity, to obtain the target transmission code rate of each video segment.

In an example, when the sum of code rates is not less than or not equal to the bandwidth capacity, that is

Iterate over the video clips to reduce the quality level. For example, the Nth video clip in the Mth column in the video clip matrix composed of M×N video clips is used as the first video clip to start the traversal process until the first video clip in the first column is traversed. The reduction processing of the quality level of the video segment, for example, the quality level of the video segment

After the reduction processing, the reduction processing result is obtained

It is equivalent to the reduced quality level; similarly, the reduced value can be adjusted as needed, and the present application uses 1 as an example without constituting a specific limit to the reduced value. Then, after reducing the processing result, new utility value and old utility value, update the quality level and code rate of the video segment, return to step B202 until

Thus, the target transmission bit rate of each video segment is obtained.

It should be noted that after step B202 or B203 is executed, when the quality level and code rate of one or more video segments are updated, an updated video segment matrix composed of M×N video segments and the updated quality level matrix, so as to determine the target transmission bit rate of the video segment according to the updated quality level matrix. It should be noted that the target transmission bit rate refers to the final determined

recorded as

The final target transmission code rate matrix obtained is

In an example, in step B203, update the quality level and code rate of the video segment according to the increase processing result, new utility value and old utility value, including step B211 or B212:

B211. When the increase processing result is less than or equal to the quality level threshold, update the new utility value according to the increase processing result, calculate the first difference between the updated new utility value and the old utility value, and determine the first impact factor of the video segment, Taking the video segment corresponding to the maximum value of the first impact factor as the first updated video segment, updating the quality level of the first updated video segment to the increase processing result corresponding to the first updated video segment, and according to the corresponding The code rate of the first updated video segment is updated by increasing the processing result.

In one example, the quality level threshold is quality level K, when the added processing result is less than or equal to the quality level threshold, namely

Assuming that m=M, n=N in the traversal process at this time, the quality level of the current video segment

The quality level matrix obtained after adding processing is

The updated new utility value, that is, the updated newU _t , can be calculated by formula (1) or formula (2). Then, according to the first difference between the updated newU _t and the old utility value oldU _t , the first impact factor diff _t (m, n) of the current video segment (m=M, n=N) is determined. It should be noted that, before the traversal starts, an initial first influence factor matrix may be generated, and then each initial first influence factor in the initial first influence factor matrix is updated during the traversal process. It can be understood that, in the traversal process from m=M and n=N to m=1 and n=1, each video segment can obtain the updated first impact factor corresponding to each video segment through the same process, Form the first impact factor matrix and write it as

First Impact Factor Matrix

Each video segment corresponds to a first impact factor, indicating the degree of change in the total utility of the video segment after the increase processing.

In an example, after the traversal of the video segment is completed, the first influencing factor matrix is determined

After that, find the position index of the maximum value

For example: if the position of the largest video segment of the first impact factor is (M-3, N-3), then m=M-3 at this moment, n=N-3, be positioned at (M-3, N-3 in the video segment matrix N-3) The video clip at the position is used as the first updated video clip, and the quality level of the first updated video clip is updated as the corresponding increase processing result of the first updated video clip

And update the code rate of the first updated video segment according to the increase processing result corresponding to the first updated video segment,

Or it can be expressed as: get a new quality level matrix according to the increase processing result corresponding to the first updated video segment

So as to determine the updated code rate matrix

Then return to step B202. Among them, F() stands for mapping.

B212. When the increase processing result is greater than the quality level threshold, use the quality level of the video segment as the updated quality level and the bit rate of the video segment as the updated bit rate.

In the embodiment of the present application, for example, the current video segment is the Mth row and the Nth column's video segment, if

At this time, keep the quality level and code rate of the video segment unchanged, that is, the quality level of the video segment is used as the updated quality level, and the bit rate of the video segment is used as the updated bit rate, and then traverse the M row and N column The next video segment of the video segment determines the relationship between the added processing result and the quality level threshold.

In an example, in step B211, the new utility value is updated according to the increase processing result, including steps B221-B226:

B221. Calculate the transmission loss at the current moment and calculate the staleness at the current moment.

In an example, the calculation steps of the transmission loss at the current moment include B2211-B2212, and B2212 includes step A1 or A2:

B2211. Determine a first storage decision value according to the storage status of the video segment at the edge node at the previous moment.

In the embodiment of the present application, the current time is recorded as time t, and the previous time is recorded as time t-1. In other embodiments, the previous time may be time t-2 or other time, which is not specifically limited.

In an example, the storage state of the video segment and the edge node at the previous moment includes whether it is stored in the edge node or not stored in the edge node. When it is stored in the edge node, the corresponding first storage decision value is 1, that is, the first When the storage decision value c _t-1 (m,n)=1, the representative video segment is stored in the edge node; on the contrary, when the first storage decision value c _t-1 (m,n)=0, the representative video segment is not stored in The edge node, at this time, the edge node needs to obtain the video clip from the cloud. It should be noted that the first storage decision value refers to the storage decision value corresponding to the storage state of the video segment at the previous moment.

A1. When the first storage decision value indicates that the video clip is stored in the edge node, obtain the storage code rate of the video clip, and calculate the code rate difference between the stored code rate and the code rate of the video clip, according to the first storage decision value, the code rate difference and The first loss coefficient is used to determine the first loss, and the second loss is determined according to the second loss coefficient and the code rate of the video segment, and the transmission loss at the current moment is determined according to the sum of the first loss and the second loss.

A2. When the first storage decision value indicates that the video segment is not stored in the edge node, the third loss is determined according to the third loss coefficient and the code rate of the video segment, and the second loss is determined according to the second loss coefficient and the code rate of the video segment, According to the sum of the third loss and the second loss, the transmission loss at the current moment is determined.

In one example, in order to facilitate the description, it is assumed that δ=1, the transmission loss of the video segment (m,n) at time t in the formula (2) is expressed as TC _t (m,n), and the formula is:

Among them, c ₁ is the first loss coefficient, c ₂ is the second loss coefficient, and c ₃ is the third loss coefficient. It should be noted that when calculating the transmission loss of a video segment, c _t-1 (m,n) is the first storage decision value of the video segment at time t-1,

is the bit rate of the video segment at time t,

is the code rate of the video segment at time t-1; the sizes of the first loss coefficient, the second loss coefficient and the third loss coefficient can be adjusted as required. It can be understood that the transmission loss of the target transmission video can be obtained by calculating the sum of the transmission losses of each video segment. In an example, it can be known from formula (6) that the transmission loss is divided into three parts, taking one of the video segments (m,n) as an example:

1,

That is, the first loss represents the transcoding loss of the edge node;

2,

That is, the second loss represents the communication loss of the edge node transmitting video clips to the client;

3.

That is, the third loss represents the communication loss for edge nodes to obtain video clips from the cloud.

Therefore, in step A1, when c _t-1 (m,n)=1, the video segment is stored in the edge node so only the first loss or the second loss may exist, and when

greater or equal to

When , the edge node can directly transmit video clips to the client to improve storage efficiency, otherwise when

less than

, it still needs to be retrieved from the cloud. And when

equal

When , the first loss is at least 0; when

more than the

, the edge node needs to transcode the video segment into the code rate allocated by the system before transmitting it to the client. In an example, the storage code rate at this time is

That is, the video clip has been

is stored in the edge node and calculates the code rate difference

Thereby determine the above-mentioned first loss, according to the second loss coefficient c ₂ and the code rate of the video segment

Determine the second loss, and according to the sum of the first loss and the second loss, determine the transmission loss at the current moment as

Similarly, when c _t-1 (m,n)=0 in step A2, the video segment is not stored in the edge node, and the edge node obtains the video segment from the cloud and uses

For storage, the first loss does not exist at this time, but the second loss and the third loss exist. In an example, according to the third loss coefficient c ₃ and the code rate of the video segment

Determine the third loss, and according to the second loss coefficient c ₂ and the code rate of the video segment

Determine the second loss, and determine the transmission loss at the current moment according to the sum of the third loss and the second loss

In an example, calculating the non-freshness at the current moment in step B221 includes steps B2213-B2215, and B2215 includes steps B1 or B2:

B2213. Determine the non-freshness at the previous moment.

Similarly, assuming that the previous moment is t-1, if the video segment at the previous moment is stored in the edge node, the non-freshness degree uf _t-1 (m,n) at the previous moment is calculated by the non-freshness formula; if the previous When the video segment is not stored in the edge node, the non-freshness uft _-1 (m,n) is 0. In other embodiments, the non-freshness can be other values, such as a smaller value close to 0.

B2214. Determine a second storage decision value according to the storage status of the video segment at the edge node at the current moment.

In one example, as in the method of step B2211, the second storage decision value is determined. When the second storage decision value is 1, it indicates that the video segment is stored in the edge node at the previous moment, and when the second storage decision value is 0, it indicates the previous moment. Video clips are not stored on edge nodes. It should be noted that the second storage decision value refers to a storage decision value corresponding to the storage state of the video segment at the current moment.

B1. When the second storage decision value indicates that the video segment is stored in the edge node, calculate the product of the preset freshness growth factor and the viewing probability, and determine the non-freshness at the current moment according to the sum of the product and the non-freshness at the previous moment .

B2. Alternatively, when the second storage decision value indicates that the video segment is not stored in the edge node, it is determined that the non-freshness at the current moment is 0.

In the embodiment of the present application, the calculation formula of non-freshness is:

Among them, _uft (m, n) is the non-freshness of the video segment (m, n) at the current moment, that is, t moment, when the second storage decision value is 0, the non-freshness is 0; when c _t (m, When n) is 1, it is calculated by the above formula (7), p _t (m,n) is the viewing probability of the video clip (m,n); α is the preset freshness growth factor, uf _t-1 (m,n) is the non-freshness of the video segment at the previous moment. It can be understood that the non-freshness of each video segment can be calculated by formula (7); in one example, uf ₀ (m,n) is 0, and the non-freshness of the previous moment can be calculated by formula (7) calculated. It should be noted that the non-freshness can be used to reflect the outdated degree of video clips. The freshness of video clips stored in edge nodes is low, and outdated video clips will weaken the quality of user viewing experience. In addition, from formula (7), it can be known that _uft (m, n) is related to the viewing probability p _t (m, n) of the video segment and the storage status of the video segment at the edge node. Quality-sensitive, the picture content in the user RoI needs to be constantly updated to present the freshest content of the user, so the larger p _t (m,n) is, the faster the freshness of the video clip will be lost.

B222. Determine the effective quality according to the non-freshness at the current moment, the viewing probability, and the quality level at the current moment.

In the embodiment of the present application, the calculation formula of the effective mass is:

Among them, EQ _t (m,n) is the effective quality of the video segment (m,n) at time t, and μ is a factor of the effective quality. It should be noted that when each sub-step in step B221 calculates the effective quality, the quality level at the current moment

It is the quality level in step B201 or it can also be the result of adding processing in step B203. It can be understood that the effective quality of each video segment can be calculated by the above formula (8). In the embodiment of this application, the purpose of setting the effective quality is to assign a higher quality level to the video clips in the user's attention area and ensure that the freshness of the video clips is less, because if the quality level of the video clip is high, but the content is outdated , then the effective quality of the video segment will become poor, and it is necessary to assign a high quality level to the video segment in the user's region of interest to ensure that the user can clearly see the content of the video segment in the region of interest.

It should be noted that, in this embodiment of the present application, non-freshness is used as a non-freshness factor to calculate effective quality, and effective quality is used to determine user viewing quality, that is, user viewing quality is determined according to non-freshness factor. In related technologies, considering the advantages of mobile edge computing (Mobile Edge Computing, MEC) nearby services, multimedia content such as popular videos is usually cached in the MEC server at the base station in advance to achieve the purpose of enhancing the viewing experience of users. However, these strategies are applicable to videos that already exist, that is, before the user sends a content request, the edge node has already obtained the information about the probability of the video piece being watched, so it cannot be applied to real-time videos such as live videos, because real-time video information (such as content, Probability of being watched) is updated in real time, recorded while playing, and cannot be obtained in advance. However, some video content is usually the most valuable when it is fresh. For example, in autonomous driving, it is essential to know the position, direction and speed of the motor vehicle in real time. When the video clips are stored in the edge node for a long time, the freshness of the video clips This will affect the user's viewing quality. Therefore, the non-freshness factor is introduced in the embodiment of this application, which can describe the obsolete degree of the content of the video clips that have been stored in the edge node and the video clips that do not exist in the edge node. The non-freshness The factor is applied to the storage method of code rate allocation and storage decision, which can improve the quality of video watched by users and reduce transmission loss.

B223. Obtain the quality level of the previous moment of the video clip, and calculate the first subjective quality at the current moment and the second subjective quality at the previous moment according to the quality level at the current moment, the viewing probability and the quality level at the previous moment, and according to The difference between the first subjective quality and the second subjective quality determines the temporal quality loss.

In the embodiment of this application, the calculation formula of subjective quality is:

The formula for calculating the time quality loss is:

TQ _t (m,n)＝|Q _t (m,n)-Q _t-1 (m,n)| (10)

Among them, Q _t (m, n) is the subjective quality (i.e. the first subjective quality) of the video segment (m, n) at time t (current time), and TQ _t (m, n) is the video segment (m, n) at time t ), Q _t-1 (m,n) is the subjective quality (ie, the second subjective quality) of the video segment (m,n) at the moment t-1 (the previous moment). Therefore, the first subjective quality can be calculated by formula (9), and the first subjective quality can be calculated by formula (9) through the quality of the video segment at the previous moment and the viewing probability at the previous moment (approximately, the viewing probability at the current moment can be used) Two subjective quality, then combined with formula (10) can calculate the temporal quality loss of the video segment. It should be noted that the subjective quality representation of a video clip depends on the user's sensitivity to the video clip. In the embodiment of the present application, the viewing probability factor of the video clip is normalized to the interval of 0-1 by using a mathematical method with the base e as the basis. Q _t (m,n) is inversely proportional to p _t (m,n). If the importance of the video segment is low, even if the video segment is assigned a lower

The subjective quality Q _t (m,n) is still acceptable to the user. In addition, since the user is sensitive to the quality difference of images in adjacent times, if the user's subjective quality differs greatly in adjacent times, even if a higher quality is assigned to the video segment, the quality difference will still have a negative impact on the user's viewing experience. Therefore the temporal mass loss is defined as Equation (10).

B224. Calculate the subjective quality mean value according to the first subjective quality and the total number of video segments, and determine the spatial quality loss according to the difference between the first subjective quality and the subjective quality mean value.

In the embodiment of this application, the calculation formula of spatial quality loss is:

Among them, SQ _t (m,n) is the spatial quality loss of the video segment (m,n) at time t,

is the mean value of subjective quality, M·N is the total number of video clips, and Q _t (m,n) at time t is the above-mentioned first subjective quality. It should be noted that, in addition to the poor quality of adjacent video clips that will weaken the user's viewing experience, the subjective quality difference in the space of video clips will also affect the user's viewing experience, so the spatial quality loss needs to be considered.

B225. Obtain the viewing quality of the user according to the effective quality, the temporal quality loss, the spatial quality loss and the corresponding preset weights.

In an example, the calculation formula of user viewing quality QoP _t (m, n) in formula (1) and formula (2) is:

QoP _t (m,n)=β ₁ EQ _t (m,n)-β ₂ TQ _t (m,n)-β ₃ SQ _t (m,n) (12)

It should be noted that the effective quality, temporal quality loss, spatial quality loss and the corresponding preset weights may be the same or different, and the corresponding preset weight of the effective quality in the embodiment of the present application is the preset first weight β ₁ , The corresponding preset weight of the temporal quality loss is the preset second weight β ₂ , and the corresponding preset weight of the spatial quality loss is the preset third weight β ₃ . In the embodiment of the present application, the viewing quality of the user corresponding to each video segment can be calculated respectively by formula (12).

B226. Obtain an updated new utility value according to the difference between the viewing quality of the user and the transmission loss at the current moment.

In an example, the updated new utility value, that is, the updated newU _t , can be calculated by formula (1). It should be noted that in the embodiment of this application, δ is 1 as an example. When δ is not 1, when calculating the difference between the user's viewing quality and the transmission loss at the current moment, the difference refers to the user's viewing quality Subtract the difference between δ and the product of transmission loss at the current moment, and the updated new utility value can be obtained according to the sum of the differences.

In an example, in step B204, according to the reduction processing result, the new utility value and the old utility value, the quality level and the code rate of the video segment are updated, including step B231 or B232:

B231. When the reduction processing result is greater than or equal to the preset threshold value, update the new utility value according to the reduction processing result, calculate the second difference between the updated new utility value and the old utility value, and determine the second impact factor of the video segment, Taking the video segment corresponding to the maximum value of the second impact factor as the second updated video segment, updating the quality level of the second updated video segment to the reduction processing result corresponding to the second updated video segment, and according to the corresponding The code rate of the second updated video segment is updated by reducing the processing result.

In an example, the preset threshold can be set as required. In this embodiment of the present application, the preset threshold is 1, which is the lowest quality level, as an example. In one example, when reducing the processing result

Greater than or equal to 1, according to the reduction processing result

The new utility value newU _t in the step B201 is updated, according to the second difference between the new utility value after updating and the old utility value, determine the second impact factor diff _t ' (m, n) of the video segment, and the second The quality level corresponding to the maximum value of the impact factor is updated as the result of the reduction process, and the video segment corresponding to the maximum value of the second impact factor is used as the second update video segment, and the quality level of the second update video segment is updated to the second update video The reduction processing result corresponding to the segment, and then update the code rate of the second updated video segment according to the reduction processing result corresponding to the second updated video segment. It should be noted that the determination method of the second impact factor diff _t ′(m,n) is similar to that of the first impact factor diff _t (m,n), by traversing the video clips according to the updated new utility value and the old utility value oldU _t to determine the second impact factor diff _t '(m,n) of each video segment, so as to obtain a new second impact factor matrix

Second Impact Factor Matrix

Each video segment corresponds to a second impact factor, which indicates the degree of change in the total utility of the video segment after reduction processing, and finds the position index of the maximum value

For example: then this moment m=M-3, n=N-3, with the video clip that is positioned at (M-3, N-3) position in the video clip matrix as the second update video clip, the second update video clip The quality level is updated as the reduction processing result corresponding to the second updated video segment

and updating the code rate of the second updated video segment according to the reduction processing result corresponding to the second updated video segment,

Or it can be expressed as: obtain a new quality level matrix according to the increase processing result corresponding to the second updated video segment

So as to determine the updated code rate matrix

Then return to step B202. Among them, F() stands for mapping.

B232. When the reduction processing result is less than the preset threshold, use the quality level of the video segment as the updated quality level and the bit rate of the video segment as the updated bit rate.

At this time, keep the quality level and code rate of the video segment unchanged, that is, the quality level of the video segment is used as the updated quality level, and the bit rate of the video segment is used as the updated bit rate, and then traverse the M row and N column The next video segment of the video segment determines the relationship between the reduction processing result and the quality level threshold.

In the embodiment of the present application, the detailed process of code rate allocation is as follows:

enter:

B _max (

is the matrix formed by the quality level of each video segment at time t-1,

is the matrix formed by each video segment c _t-1 (m,n) at time t-1,

is the matrix formed by uf _t-1 of each video segment at time t-1, B _max is the maximum bandwidth)

output:

Among them, the final output/return

is the final quality level matrix

Utility() means to use formula (1) or (2) to calculate. It should be noted that when

Execute lines 6 to 11 in the above code during the traversal of video clips, execute lines 12 to 14 after a traversal is completed, and then return to line 5 until judgment is made in line 5

Output the final quality level matrix

And when

Execute lines 17 to 22 in the above code during the traversal of video clips, execute lines 23 to 25 after a traversal is completed, and then return to line 16 until the judgment is made at line 16

Output the final quality level matrix

Embodiment two

As shown in Figure 2, the embodiment of the present application also provides a storage method, at least including but not limited to steps S300-S600:

S300. Obtain the viewing probabilities of all video clips in the target transmission video and the code rate allocation results by using a code rate allocation method.

It should be noted that the code rate allocation method refers to the code rate allocation method in the embodiment, and the code rate allocation result includes the target transmission code rate of each video segment (m, n)

and the target quality level

S400. Predict expected effective quality and expected transmission loss according to the code rate allocation result and viewing probability.

In the embodiment of the present application, when making a storage decision for an edge node, it is assumed that time t is the current time, since the effective quality and transmission loss of the video segment at time t+1 (the next time) are unknown, the code rate allocation result at time t is used , video segment, and target quality level for prediction. in,

and

Represent the expected effective quality and expected transmission loss of the video segment (m, n) respectively, and respectively predict the impact of the video segment stored at time t on the effective quality of the video segment at time t+1 and the transmission loss of obtaining the video segment from the cloud.

In an example, step S400 includes steps B401-B403:

B401. Obtain the non-freshness at the current moment and the fourth storage decision value of the video segment at the current moment, and determine the predicted non-freshness according to the non-freshness at the current moment, viewing probability and the fourth storage decision value.

In one example, predicting non-freshness

The formula is:

Among them, _uft (m,n) is the non-freshness at the current moment, c _t (m,n) is the fourth storage decision value, α is the growth factor of non-freshness, p _t (m,n) is the viewing probability . It should be noted that the fourth storage decision value refers to the storage decision value corresponding to the storage state of the video segment at the current moment, which is the same as the second storage decision value.

B402. Determine the expected effective quality according to the predicted non-freshness, viewing probability and target quality level.

In one example, the expected effective mass

The formula is:

where μ is a factor of the effective mass,

is the target quality level.

B403. Determine expected transmission loss according to the target transmission code rate, the fourth storage decision value, the second loss coefficient, and the third loss coefficient.

In one example, the expected transmission loss

The formula is:

Among them, c ₂ is the second loss coefficient, c ₃ is the third loss coefficient,

is the target transmission code rate, and c _t (m,n) is the fourth storage decision value.

S500. Perform storage optimization processing according to the expected effective quality, expected transmission loss, and maximum capacity of the edge node, to obtain storage decision information.

In the embodiment of the present application, the storage decision information includes the third storage decision value c _t '(m,n) of each video segment (m,n), and constitutes the third storage decision value matrix

The third storage decision value represents whether the video segment needs to be stored in the edge node, for example, the third storage decision value is 1, the representation needs to be stored in the edge node, and the third storage decision value is 0, the representation does not need to be stored in the edge node. It should be noted that the third storage decision value refers to a storage decision value corresponding to the video segment obtained after storage optimization processing.

In one example, the goal of the storage method is to increase the expected effective quality and reduce the expected transmission loss, so the storage optimization processing problem can be expressed as:

Among them, β ₁ is the preset first weight, s _t (m,n) is the size of the video clip (m,n), C _max is the maximum storage capacity of the edge node, that is, the maximum capacity, and δ is the weight of transmission loss. Understandably, by solving the

as well as

The optimization process can then be stored using equation (16). The constraints stipulate that the sum of the sizes of the stored video clips cannot exceed the storage capacity of the edge nodes, and the cache strategy to be solved for the storage problem is a binary variable matrix. In the embodiment of the present application, the third storage decision value is solved by the branch and bound algorithm .

In an example, step B500 includes steps B511-B513:

B511. Determine the expected utility according to the expected effective quality and expected transmission loss, determine the storage gain according to the expected utility corresponding to the video segment stored in the edge node and not stored in the edge node at the current moment, and determine the video value of each video segment according to the storage gain .

In one example, the expected utility is

It is divided into two cases. The expected utility of the first case is recorded as the first expected utility DC _t (m,n), that is, the expected utility of the video segment stored in the edge node at the current moment, and the expected utility of the second case is recorded as the first expected utility. 2. Expected utility DC _t ′(m,n), that is, the expected utility of not storing the video segment in the edge node at the current moment, in an example:

1), when the video segment is stored in the edge node at time t (ie the current time), that is, the fourth storage decision value c _t (m,n)=1:

Among them, μ is a factor of effective quality, β ₁ is the preset first weight, δ is the weight of transmission loss, α is the growth factor of non-freshness, and c ₂ is the second loss coefficient.

2) When the video segment is not stored in the edge node at time t (ie the current time), that is, the fourth storage decision value c _t (m,n)=0:

Among them, c ₃ is the third loss coefficient.

In the embodiment of this application, the calculation formula of video value is:

Among them, v _t (m,n) is the video value of the video segment (m,n), and DC _t (m,n)-DC _t '(m,n) is the storage gain. It should be noted that in the embodiment of this application, the 0-1 knapsack problem is used to solve the optimal solution through the 0-1 branch and bound algorithm to obtain the storage decision information. Therefore, assuming that C _max is the size of the knapsack, the video clip is equivalent to the The items in the backpack, the corresponding size and value are defined as w _t (m,n) and v _t (m,n), where w _t (m,n)=s _t (m,n), find the optimal solution The goal of obtaining storage decision information is to make the size of the video segment less than or equal to C _max and the sum of the values reaches the maximum value as much as possible.

B512. Obtain the sizes of all the video clips and determine the value density according to the sizes of the video clips and the storage gain.

In the embodiment of this application, the value density ds _t (m, n) formula is:

B513. Obtain a third storage decision value for each video segment through a branch and bound algorithm according to the video value and value density.

In this embodiment of the present application, the third storage decision value calculated through the branch and bound algorithm may be a first value or a second value, for example, the first value is 1, and the second value is 0. In an example: when the third storage decision value of the video segment c _t (m,n) is 1, it means that the video segment needs to be stored in the edge node, and when the third storage decision value of the video segment is c _t When (m,n) is 0, it means that the video segment does not need to be stored in the edge node

In an example, the branch and bound algorithm actually generates a binary tree with multiple nodes, and the processing flow is as follows:

1) Arrange the video clips according to the value density (for example, from high to low: nodesList=upToLow(ds _t (m,n))), initialize the upper limit value upprofit through nodesList and C _max , so that upprofit is (modesList ,C _max ) to get the initial upper bound value.

2), initialize the queue pq:

It should be noted that there is a node initialized to 0 in the initialization queue pq (referred to as the initial node), and the data type of the node in the binary tree is a structure containing variables such as storage quality (such as weight, value, etc.), allocated code rate, etc. body, and the storage quality of the initial node is 0, which is assigned to the parent node during initialization, and the head node of the queue pq is returned through pq.get() in the subsequent traversal as the new parent node and the head node is deleted and the queue pq The rest of the nodes move forward. The left and right nodes are initialized to 0 in each round of iteration. Every time it is judged whether the left and right nodes may have a solution, a new node will be added to the queue, and the parent nodes will be updated in turn.

3) Iterative process:

In one example, after initialization, the initial node is used as the parent node, and then the left node is judged according to the video segment with the highest value density in the modesList: a fictitious left node is generated, and the video segment with the highest value density is used as the segment to be confirmed, when the segment to be confirmed The total capacity after adding the knapsack is less than C _max , where the total capacity after adding the knapsack refers to the sum of the sizes of all nodes on the branch where the imaginary left node is located in the binary tree, for example: the total capacity at this time is the size of the segment to be confirmed and the initial The sum of the sizes of the nodes. And when the total capacity is less than C _max after adding the knapsack, at this time the imaginary left node is determined to be the actual left node and the fourth storage decision value of the fragment to be confirmed is the first value, and the left node is added to the queue pq, At this time, the size of the segment to be confirmed and the video value of the segment to be confirmed are saved in the left node; then, the value threshold, the upper limit value upprofit, is updated by using the left node to generate a new value threshold upprofit. Among them, the upprofit value threshold represents the maximum video value that the current edge node can hold, which is the maximum value of the remaining knapsack nodes that can be loaded into the knapsack calculated by the greedy algorithm during the process of solving the 0-1 knapsack problem. It should be noted that when the total capacity of the segment to be confirmed is greater than or equal to C _max after being added to the backpack, that is, the sum of the size of the segment to be confirmed and the size of the initial node is greater than or equal to C _max , the actual left node will not be generated and the upper bound will not be updated at this time Value upprofit.

In one example, after determining the left node, the video segment with the highest value density is also used as the segment to be confirmed, and the segment to be confirmed is used to determine the right node: generate a fictitious right node, and determine whether the maximum utility value of the segment to be confirmed is not added to the backpack is greater than the updated upper limit value upprofit, if the maximum utility value of the segment to be confirmed not added to the backpack is greater than the updated upper limit value upprofit, then the imaginary right node is determined to be the actually generated right node, and the value saved in the right node at this time is The size of the segment to be confirmed and the video value of the segment to be confirmed, then mark the fourth storage decision value of the segment to be confirmed as the second value and add the right node to the queue pq; and if the segment to be confirmed is not added to the maximum utility value of the backpack If it is less than or equal to the updated upper bound value upprofit, the actual right node will not be generated. It should be noted that the maximum utility value refers to the sum of the video values of all nodes on the branch of the fictitious right node in the binary tree. For example: in the above process, the maximum utility value is the difference between the segment to be confirmed and the video value of the initial node. and.

It should be noted that, in the embodiment of this application, the queue adopts the queue data structure Queue(pq) of python, and the queue data structure Queue(pq) of python obeys the order of first-in-first-out, and pq.get() indicates the return queue The head node of pq, and delete the head node, and then determine the new parent node from the queue pq. In the iterative process 3), the parent node will be deleted, and then the first node arranged in order in the queue will be used as the new parent node, and the above-mentioned steps of judging the left node and judging the left node will be performed, and each time it is determined When confirming a segment, the video segment that has not been used as a segment to be confirmed and has the highest value density is used as a new segment to be confirmed, and the steps of judging the left node and judging the left node are continuously iterated. Among them, in the iterative process, when the actual left node is generated, the corresponding segment to be confirmed is marked as the first value, and when the actual right node is generated, the corresponding segment to be confirmed is marked as the second value, and in In the process of continuous iteration, the iteration ends when the queue pq is empty, and the third storage decision value of each video segment is obtained.

In the embodiment of the present application, in order to make the storage gain non-negative, the storage decision value matrix is updated, and step B514 is also included:

B514. Update the third storage decision value corresponding to the video segment whose third storage decision value is the first value and the storage gain is less than 0 to the second value to obtain storage decision information.

In one example, the third storage decision value of the M·N video clips obtained in step B513 is searched, c _t (m,n)=1 and the storage gain DC _t (m,n)-DC′ _t The third storage decision value c _t (m,n) of the video segment (m,n)<0 is updated to 0, and the other third storage decision values remain unchanged, and the storage decision information is obtained, that is, the target storage matrix

S600. Store the video segment according to the storage decision information.

In an example, the edge node can store decision information according to the target storage matrix

The third storage decision value in determines the storage policy. For example: store the video segment with the third storage decision value of 1 in the edge node, update the storage content of the edge node, and use it for the next moment, that is, the service when the user end requests to obtain the target video transmission. In the embodiment of the present application, when the target transmission video acquired by the client is real-time video, the code rate allocation method and the storage method of the embodiment of the present application can be used as an online algorithm to respond to the request information of the client in real time, and the storage of edge nodes updated at the current moment The content can provide services for users to obtain video clips in the future.

In the embodiment of this application, the process of obtaining storage decision information in the storage method is as follows:

enter:

_Cmax

output:

Among them, when executing the judgment on line 7, when the queue is not empty, execute lines 8-15, and then return to line 7 to judge until the queue is empty, then execute lines 16 to 20, and output

It should be noted,

is the storage decision matrix at time t, return

that is

In the following, the code rate allocation method and the storage method of the embodiment of the present application are described in two specific application scenarios:

As shown in Figure 3, when the target transmission video is an immersive video, the server slices the video, that is, divides it into blocks, obtains the video segment, and then records the movement track of the user's focus; when the user sends a request through the client, the server The motion trajectory calculates the probability that the video segment is located in the user's FOV, obtains the viewing probability, and then performs code rate allocation (refer to step S200 for details), and judges whether all video segments (i.e. video segments) have all been sent to the user: if so, carry out storage decisions, that is Execute the above storage method to update the video slices (i.e. video clips) stored in the edge nodes; The request is forwarded to the cloud, so that the cloud server in the cloud sends the video clip to the edge node, and the edge node then sends the video clip (that is, the video clip) to the client, and if the video clip (that is, the video clip) is stored in the edge node, the user The content of the video piece (ie video clip) can be read from the edge node.

As shown in Figure 4, when the target transmission video is a video in a cloud desktop application scenario, the server slices the video, that is, divides it into blocks, obtains the video segment, and then records the interaction track between the user and the cloud desktop (such as the operation of the mouse or touch screen) region frequency); when the user sends a request through the client terminal, the server calculates the probability that the video slice is located in the ROI of the user according to the cloud desktop interactive trajectory, obtains the viewing probability, and then performs code rate allocation (refer to step S200 for details), and judges that all video slices (i.e. video segment) has been sent to the user: if so, make a storage decision, that is, execute the above storage method, thereby updating the video segment stored in the edge node; if not, determine whether the video segment (ie video segment) is stored in the edge node, if the video If the video clips (that is, video clips) are not stored in the edge nodes, the request needs to be forwarded to the cloud, so that the cloud server in the cloud sends the video clips (that is, video clips) to the edge nodes, and the edge nodes then send the video clips (that is, video clips) sent to the user end, and if the video piece (ie video clip) is stored in the edge node, the user can read the content of the video piece (ie video clip) from the edge node.

It should be noted that the code rate allocation method in the embodiment of the present application is executed by the terminal of the user end, and in other embodiments, it can also be executed by the cloud server in the cloud, and the storage method can be executed by the edge node or executed by the cloud server in the cloud, Edge nodes store video clips according to storage decision information. Among them, the edge node refers to a network node with caching capability close to the user side in the Internet of Things.

As shown in Figure 5, the embodiment of the present application also provides a code rate allocation device, including:

The obtaining module is used to obtain target video data; the target video data includes the viewing probability of all video clips in the target transmission video;

The processing module is used to allocate the code rate to the video segment according to the viewing probability, so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment, and the user's viewing quality is determined according to The non-freshness factor is determined, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.

As shown in Figure 6, the embodiment of the present application also provides a storage device, including:

Determine module, be used for determining the viewing probability of all video clips in the target transmission video and the code rate allocation result;

A prediction module, configured to predict expected effective quality and expected transmission loss according to the code rate allocation result and viewing probability;

The optimization module is used to perform storage optimization processing according to the expected effective quality, the expected transmission loss and the maximum capacity of the edge node to obtain storage decision information; the storage decision information includes the third storage decision value of each video clip, the third storage decision value Indicating whether video clips need to be stored in edge nodes;

A storage module, configured to store video clips according to storage decision information;

Obtain target video data; the target video data includes viewing probabilities of all video clips in the target transmission video;

According to the viewing probability, the code rate of the video segment is allocated so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment, and the user's viewing quality is determined according to the non-freshness factor , the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.

The content in the above-mentioned method embodiment is applicable to this device embodiment, and the specific functions realized by this device embodiment are the same as those of the above-mentioned method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned method embodiment.

As shown in Figure 7, the embodiment of the present application also provides an electronic device, the electronic device includes a processor and a memory, at least one instruction, at least one program, code set or instruction set are stored in the memory, at least one instruction, at least one The program, code set or instruction set is loaded and executed by the processor to implement the code rate allocation or storage method of the foregoing embodiments. The electronic devices in the embodiments of the present application include, but are not limited to, mobile phones, tablet computers, computers, vehicle-mounted computers, servers, and the like.

The content in the above-mentioned method embodiment is applicable to this device embodiment. The specific functions realized by this device embodiment are the same as those of the above-mentioned method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned method embodiment.

The embodiment of the present application also provides a computer-readable storage medium. At least one instruction, at least one program, code set or instruction set is stored in the storage medium, and at least one instruction, at least one program, code set or instruction set is loaded by the processor. And execute to implement the code rate allocation or storage method of the foregoing embodiments.

The embodiment of the present application also provides a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the code rate allocation or storage method of the foregoing embodiments.

A code rate allocation method, storage method, device, device, and storage medium proposed in the embodiments of the present application, by acquiring target video data, the target video data includes viewing probabilities of all video segments in the target transmission video, and the introduction of viewing probabilities is beneficial to Improve the user's viewing quality; according to the viewing probability, the code rate is allocated to the video segment, so that the total utility of the target transmission video meets the target value; and the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment, said User viewing quality is determined according to the non-freshness factor. The non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability. The goal of reducing transmission loss while improving user viewing quality of video clips.

Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and an appropriate combination thereof.

In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit . Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Several embodiments of the present application have been described above with reference to the accompanying drawings, and the scope of rights of the present application is not limited thereto. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the present application shall fall within the scope of rights of the present application.

Claims

A code rate allocation method, comprising:

Obtain target video data; the target video data includes the viewing probability of all video clips in the target transmission video;

According to the viewing probability, the code rate is allocated to the video segment, so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment , the viewing quality of the user is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.
The code rate allocation method according to claim 1, wherein: said performing code rate allocation on said video clips according to said viewing probability comprises:

Obtaining the new utility value and the old utility value of the total utility of the target transmission video, and acquiring the quality level and bit rate of the video segment;

Judging whether the sum of the code rates is less than or equal to the bandwidth capacity;

When the sum of the code rates is less than or equal to the bandwidth capacity, traverse the video segment to increase the quality level, and update the video segment according to the increase processing result, the new utility value and the old utility value Quality level and code rate, return to the step of judging whether the sum of the code rates is less than or equal to the bandwidth capacity, until the sum of the code rates is greater than the bandwidth capacity, obtain the target transmission code rate of each of the video segments; The above-mentioned increase processing result is the increased quality level;

When the sum of the code rates is not less than or not equal to the bandwidth capacity, traverse the video segment to reduce the quality level, and update the video according to the reduction processing result, the new utility value and the old utility value The quality level and code rate of the segment, return to the step of judging whether the sum of the code rates is less than or equal to the bandwidth capacity, until the sum of the code rates is less than or equal to the bandwidth capacity, and obtain the target transmission of each video segment code rate; the reduction processing result is the reduced quality level.
The code rate allocation method according to claim 2, wherein: updating the quality level and code rate of the video segment according to the increase processing result, the new utility value and the old utility value, comprising:

When the increase processing result is less than or equal to the quality level threshold, update the new utility value according to the increase processing result, calculate a first difference between the updated new utility value and the old utility value, and determine the The first impact factor of the video segment, the video segment corresponding to the maximum value of the first impact factor is used as the first updated video segment, and the quality level of the first updated video segment is updated to the corresponding increase of the first updated video segment processing results, and update the code rate of the first updated video segment according to the increase processing result corresponding to the first updated video segment.
The code rate allocation method according to claim 3, wherein: said updating said new utility value according to said increase processing result comprises:

Calculate the transmission loss at the current moment and calculate the non-freshness at the current moment;

Determine the effective quality according to the non-freshness at the current moment, the viewing probability, and the quality level at the current moment;

Acquire the quality level of the video clip at the previous moment, and calculate the first subjective quality at the current moment and the previous The second subjective quality at a moment, and determine the time quality loss according to the difference between the first subjective quality and the second subjective quality;

calculating a subjective quality mean based on the first subjective quality and the total number of video segments, and determining a spatial quality loss based on a difference between the first subjective quality and the subjective quality mean;

Obtain user viewing quality according to the effective quality, the temporal quality loss, the spatial quality loss and the corresponding preset weights;

An updated new utility value is obtained according to the difference between the viewing quality of the user and the transmission loss at the current moment.
The code rate allocation method according to claim 4, wherein: said calculating the transmission loss at the current moment comprises:

determining a first storage decision value according to the storage status of the video segment at the edge node at the previous moment;

When the first storage decision value indicates that the video clip is stored in the edge node, obtain the storage code rate of the video clip, calculate the code rate difference between the stored code rate and the video clip code rate, according to the The first storage decision value, the code rate difference and the first loss coefficient determine the first loss, determine the second loss according to the second loss coefficient and the code rate of the video segment, and determine the second loss according to the first loss and the second The sum of losses, to determine the transmission loss at the current moment;

Or, when the first storage decision value indicates that the first segment is not stored in the edge node, determine the third loss according to the third loss coefficient and the code rate of the video segment, and determine the third loss according to the second loss coefficient and the Determine the second loss based on the code rate of the video segment, and determine the transmission loss at the current moment according to the sum of the third loss and the second loss;

Wherein, the first loss is the transcoding loss of the edge node, the second loss is the communication loss of the edge node transmitting the video segment to the client, and the third loss is the edge node's acquisition of the video segment from the cloud communication loss.
The code rate allocation method according to claim 4, wherein: said calculation of the non-freshness at the current moment includes:

Determine the non-freshness of the previous moment;

Determine a second storage decision value according to the storage status of the video segment at the edge node at the current moment;

When the second storage decision value indicates that the video segment is stored in the edge node, calculate the product of the preset freshness growth factor and the viewing probability, and calculate the product according to the sum of the product and the non-freshness at the previous moment , to determine the non-freshness of the current moment;

Or, when the second storage decision value indicates that the video clip is not stored in the edge node, it is determined that the non-freshness at the current moment is 0.
The code rate allocation method according to claim 2, wherein: updating the quality level and code rate of the video segment according to the reduction processing result, the new utility value and the old utility value, comprising:

When the reduction processing result is greater than or equal to a preset threshold, update the new utility value according to the reduction processing result, calculate a second difference between the updated new utility value and the old utility value, and determine the The second impact factor of the video segment, the video segment corresponding to the maximum value of the second impact factor is used as the second updated video segment, and the quality level of the second updated video segment is updated to the corresponding reduction of the second updated video segment processing results, and update the code rate of the second updated video segment according to the reduction processing result corresponding to the second updated video segment.
A storage method comprising:

Obtain the viewing probability and the code rate distribution result of all video clips in the target transmission video by the code rate distribution method as described in any one of claims 1-7;

Predict expected effective quality and expected transmission loss according to the code rate allocation result and the viewing probability;

According to the expected effective quality, the expected transmission loss and the maximum capacity of the edge node, perform storage optimization processing to obtain storage decision information; the storage decision information includes a third storage decision value for each of the video segments, the The third storage decision value represents whether the video segment needs to be stored in the edge node;

The video segment is stored according to the storage decision information.
The storage method according to claim 8, wherein: said code rate allocation result includes a target transmission code rate of each said video segment, and each said target transmission code rate has a corresponding target quality level, said according to said The code rate allocation result and the viewing probability predict the expected effective quality and expected transmission loss, including:

Obtain the non-freshness at the current moment and the fourth storage decision value of the video segment at the current moment, and determine the predicted non-freshness according to the non-freshness at the current moment, the viewing probability and the fourth storage decision value ;

determining the expected effective quality based on the predicted non-freshness, the viewing probability, and the target quality level;

The expected transmission loss is determined according to the target transmission code rate, the fourth stored decision value, the second loss coefficient, and the third loss coefficient.
The storage method according to claim 8, wherein: said code rate allocation result includes a target transmission code rate of each said video segment, and each said target transmission code rate has a corresponding target quality level, said according to said The expected effective quality, the expected transmission loss and the maximum capacity of the edge node are processed for storage optimization to obtain storage decision information, including:

Determine the expected utility according to the expected effective quality and the expected transmission loss, determine the storage gain according to the expected utility corresponding to the video segment stored in the edge node and not stored in the edge node at the current moment, and determine each according to the storage gain a video value of said video segment;

Obtaining the size of the video clip, and determining the value density of the video clip according to the size of the video clip and the storage gain;

According to the video value and the value density, the third storage decision value of each video segment is obtained through a branch and bound algorithm; the third storage decision value is a first value or a second value, and the first value is The representative video segment needs to be stored in the edge node, and the second value representative video segment does not need to be stored in the edge node.
The storage method according to claim 10, further comprising:

Update the third storage decision value corresponding to the video segment whose third storage decision value is the first value and whose storage gain is less than 0 to the second value to obtain storage decision information.
A code rate distribution device, comprising:

An acquisition module, configured to acquire target video data; the target video data includes the viewing probability of all video clips in the target transmission video;

A processing module, configured to perform code rate allocation on the video segment according to the viewing probability, so that the total utility of the target transmission video meets a target value; the total utility of the target transmission video is based on the user viewing of the video segment The quality and transmission loss are determined. The user viewing quality is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.
A storage device comprising:

Determine module, be used for determining the viewing probability of all video clips in the target transmission video and the code rate allocation result;

A prediction module, configured to predict expected effective quality and expected transmission loss according to the code rate allocation result and the viewing probability;

An optimization module, configured to perform storage optimization processing according to the expected effective quality, the expected transmission loss, and the maximum capacity of the edge node to obtain storage decision information; the storage decision information includes the third storage of each video segment Decision value, the third storage decision value represents whether the video segment needs to be stored in the edge node;

a storage module, configured to store the video clips according to the storage decision information;

The viewing probability and code rate allocation results of all video clips in the determined target transmission video include:

Obtain target video data; the target video data includes the viewing probability of all video clips in the target transmission video;

According to the viewing probability, the code rate is allocated to the video segment, so that the total utility of the target transmission video meets the target value; the total utility of the target transmission video is determined according to the user's viewing quality and transmission loss of the video segment , the viewing quality of the user is determined according to the non-freshness factor, and the non-freshness factor is determined according to the storage status of the video segment at the edge node and the viewing probability.
An electronic device, including a processor and a memory, at least one instruction, at least one program, code set or instruction set are stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set It is loaded and executed by the processor to implement the code rate allocation method according to any one of claims 1-7 or the storage method according to any one of claims 8-11.
A computer-readable storage medium, wherein at least one instruction, at least one program, code set or instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set are processed by The code rate allocation method according to any one of claims 1-7 or the storage method according to any one of claims 8-11 is implemented.