CN116866622A - Method, device, equipment and storage medium for controlling multimedia data transmission rate - Google Patents

Method, device, equipment and storage medium for controlling multimedia data transmission rate Download PDF

Info

Publication number
CN116866622A
CN116866622A CN202210312241.4A CN202210312241A CN116866622A CN 116866622 A CN116866622 A CN 116866622A CN 202210312241 A CN202210312241 A CN 202210312241A CN 116866622 A CN116866622 A CN 116866622A
Authority
CN
China
Prior art keywords
live
multimedia data
client
time delay
live broadcast
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210312241.4A
Other languages
Chinese (zh)
Inventor
吴昊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202210312241.4A priority Critical patent/CN116866622A/en
Publication of CN116866622A publication Critical patent/CN116866622A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate

Abstract

The invention discloses a method, a device, equipment and a storage medium for controlling the transmission rate of multimedia data.A live client determines whether a communication network between the live client and a live server is congested and the congestion degree based on the total transmission time length and the total reception time length of at least two groups of multimedia data received from the live server, so as to avoid whether the congestion occurs by determining whether the communication network loses packets or not.

Description

Method, device, equipment and storage medium for controlling multimedia data transmission rate
Technical Field
The present invention relates to the field of communications, and in particular, to a method, an apparatus, a device, and a storage medium for controlling a multimedia data transmission rate.
Background
In a multimedia system composed of a live broadcast server and a live broadcast client, multimedia data transmission can be performed between the live broadcast server and the live broadcast client through a communication network, and when the data transmission is performed, network resources of the communication network are limited due to too much transmitted multimedia data, so that network congestion is generated in the communication network. In order to improve the utilization rate of network resources and optimize network transmission, congestion control is required. For example, congestion control may be performed by a Cubic algorithm, a BBR algorithm, an SRT algorithm, and/or a GCC algorithm in WebRTC, or the like. In the research and practice process of the prior art, the inventor of the present invention finds that the existing Cubic algorithm, BBR algorithm, SRT algorithm and GCC algorithm all determine whether network congestion occurs according to whether packet loss occurs during multimedia data transmission, that is, determine the network state of the communication network according to packet loss, and can cause the problem of reducing the sending rate of multimedia data when the network bandwidth is sufficient but the packet loss rate is high.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a storage medium for controlling a multimedia data transmission rate, which can avoid reducing the multimedia data transmission rate under the condition that network bandwidth is enough but packet loss rate is higher.
The embodiment of the invention provides a multimedia data sending rate control method, which is applied to a multimedia system, wherein the multimedia system comprises a live broadcast server and a live broadcast client, and comprises the following steps:
the live client receives at least two groups of multimedia data sent by the live server, and obtains the total sending duration spent by the live server in each group of multimedia data and the total receiving duration spent by the live client when the live client receives the live multimedia data;
the live broadcast client determines a time delay gradient generated when each group of multimedia data is transmitted in a communication network between the live broadcast server and the live broadcast client according to the total sending time length and the total receiving time length of each group of multimedia data;
the live broadcast client performs smooth processing on the time delay gradients to obtain a plurality of processed time delay gradients;
the live client determines congestion degree information of the communication network based on the change condition between the processed time delay gradients, and sends the congestion degree information to the live server;
And the live broadcast server adjusts the size of the multimedia data sending rate aiming at the live broadcast client according to the congestion degree information.
Correspondingly, the embodiment of the invention also provides a multimedia data sending rate control device, which is applied to a multimedia system, wherein the multimedia system comprises a live broadcast server and a live broadcast client, and comprises:
the time length determining unit is used for receiving at least two groups of multimedia data transmitted by the live broadcast server through the live broadcast client, and acquiring the total transmission time length spent by the live broadcast server in each group of multimedia data and the total reception time length spent by the live broadcast client when the live broadcast client is received;
the time delay determining unit is used for determining a time delay gradient generated when each group of multimedia data is transmitted in a communication network between the live broadcast server and the live broadcast client according to the total sending time length and the total receiving time length of each group of multimedia data through the live broadcast client;
the smoothing processing unit is used for carrying out smoothing processing on the time delay gradient through the live broadcast client to obtain a plurality of processed time delay gradients;
the congestion determining unit is used for determining congestion degree information of the communication network based on the change condition between the processed time delay gradients through the live client and sending the congestion degree information to the live server;
And the rate adjusting unit is used for adjusting the size of the multimedia data sending rate aiming at the live client according to the congestion degree information through the live server.
Optionally, the smoothing processing unit is further configured to:
the live broadcast client determines a plurality of groups of to-be-processed time delay gradients from a plurality of time delay gradients, wherein one group of to-be-processed time delay gradients comprises at least two time delay gradients;
the live client acquires weights corresponding to the time delay gradients included in each group of time delay gradients to be processed;
the live broadcast client forms weighted time delay gradients corresponding to each group of time delay gradients to be processed according to the time delay gradients included in each group of time delay gradients to be processed and the weight corresponding to each time delay gradient;
the live client obtains a plurality of processed time delay gradients based on each weighted time delay gradient.
Optionally, the congestion determining unit is further configured to:
the live broadcast client determines a first arrangement sequence of the time delay gradients based on the receiving sequence of each group of multimedia data;
the live broadcast client acquires the corresponding relation between each processed time delay gradient and each time delay gradient;
The live broadcast client determines a second arrangement sequence of the processed time delay gradient based on the first arrangement sequence and the corresponding relation;
the live broadcast client determines the change condition of each processed time delay gradient under the second arrangement sequence;
and the live client determines congestion degree information of the communication network based on the change condition and sends the congestion degree information to the live server.
Optionally, the congestion determining unit is further configured to:
the live broadcast client performs linear regression on each processed time delay gradient based on the second arrangement sequence to obtain a linear regression equation;
the live client determines the change trend and the change rate of each processed time delay gradient based on the slope of the linear regression equation;
and the live client takes the change trend and the change rate as the change condition of each processed time delay gradient under the second arrangement sequence.
Optionally, the congestion determining unit is further configured to:
if the change trend is an increasing trend and the last processed time delay gradient in the second arrangement sequence is larger than a time delay gradient threshold, the live client determines that the communication network is in a congestion state, and determines the congestion degree of the communication network in the congestion state according to the change rate;
The live client takes the congestion state and the congestion degree as congestion degree information of the communication network;
if the change trend is a decreasing trend and/or the last processed time delay gradient in the second arrangement sequence is not greater than the time delay gradient threshold, the live client determines that the communication network is in an uncongested state, and determines the uncongested degree of the communication network when the communication network is in the uncongested state according to the change rate;
the live client takes the uncongested state and the uncongested degree as congestion degree information of the communication network.
Optionally, the rate adjustment unit is further configured to:
if the communication network is determined to be in a congestion state according to the congestion degree information, the live broadcast server reduces the multimedia data sending rate based on the congestion degree;
and if the communication network is determined to be in an uncongested state according to the congestion degree information, the live broadcast server promotes the multimedia data sending rate based on the uncongested state.
Optionally, the rate adjustment unit is further configured to:
the live broadcast server acquires an inverse proportion relation between the change rate corresponding to the congestion degree and a first coefficient, and determines the first coefficient based on the inverse proportion relation and the change rate corresponding to the congestion degree;
The live broadcast server reduces the multimedia data transmission rate based on the first coefficient;
the direct broadcast server obtains a proportional relation between the change rate corresponding to the uncongested degree and a second coefficient, and determines the second coefficient based on the proportional relation and the change rate corresponding to the uncongested degree;
the live broadcast server increases the multimedia data transmission rate based on the second coefficient.
Optionally, the rate adjustment unit is further configured to:
the live broadcast server obtains an input code rate of the multimedia data sent to the live broadcast client by the current application layer;
the live broadcast server determines the retransmission rate when retransmitting the multimedia data to the live broadcast client;
the live broadcast server determines the size of the multimedia data transmission rate currently aiming at the live broadcast client based on the input code rate and the retransmission rate.
Optionally, the duration determining unit is further configured to:
and the live broadcast client calculates the difference value between the receiving total time length and the sending total time length corresponding to each group of multimedia data, and the difference value is used as a time delay gradient generated when each group of multimedia data is transmitted in the communication network.
Also, an embodiment of the present application further provides a computer device, including:
A memory for storing a computer program;
a processor for implementing any one of the steps of the multimedia data transmission rate control method when executing the computer program.
In addition, the embodiment of the application also provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program realizes any step of the multimedia data transmission rate control method when being executed by a processor.
The embodiment of the application provides a method, a device, equipment and a storage medium for controlling a multimedia data sending rate, wherein a live client can determine whether a communication network between the live client and a live server is congested and the congestion degree based on the sending total time length and the receiving total time length of at least two groups of multimedia data received from the live server, so that whether the congestion occurs or not is avoided through packet loss of the communication network, then the live client informs the live server of the congestion degree of the communication network, and further the live server can adjust the multimedia data sending rate according to the congestion degree, so that the problem that the multimedia data sending rate is reduced when the network bandwidth is enough but the packet loss rate is higher is avoided, the multimedia data can be stably sent when the network bandwidth is enough but the packet loss rate is higher, the multimedia data sending rate can be timely adjusted when the congestion scene is judged, and serious congestion is avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of a scenario of a multimedia data transmission rate control method according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for controlling a multimedia data transmission rate according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a transmission scenario of a set of multimedia data according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of linear regression provided by an embodiment of the present invention;
fig. 5 is another flowchart of a method for controlling a multimedia data transmission rate according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a method for controlling a multimedia data transmission rate according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a multimedia data transmission rate control device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The embodiment of the invention provides a method, a device, equipment and a storage medium for controlling a multimedia data sending rate, and particularly the method for controlling the multimedia data sending rate can be applied to a live broadcast scene, namely a multimedia system comprising a live broadcast server and a live broadcast client, wherein the live broadcast server sends multimedia data (such as audio data and/or video data in live broadcast and the like) to the live broadcast client, so that the live broadcast client analyzes the multimedia data to present live broadcast pictures to a user. The method is applied to a live broadcast platform, realizes global consistency of slices through an algorithm mechanism, reduces downlink jamming, and has higher system disaster tolerance capability and robustness, so that user experience of watching live broadcast is optimized.
When the multimedia data transmission rate control method is operated at a terminal, the user terminal includes, but is not limited to, a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, an aircraft, and the like. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, a cloud multimedia database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as a large amount of multimedia data and an artificial intelligence platform. The user terminal and the server may be directly or indirectly connected through wired or wireless communication, and the present application is not limited herein.
For example, the terminal device may store a live application and be used to present scenes in a live view. The terminal device is used for interacting with a user through a live broadcast interface, for example, a live broadcast application program is downloaded and installed through the terminal device and operated. The manner in which the terminal device provides the live interface to the user may include a variety of ways, for example, it may be rendered for display on a display screen of the terminal device, or the live interface may be presented by holographic projection. For example, the terminal device may include a touch display screen for presenting a live interface including live pictures and receiving operation instructions generated by a user acting on the live interface, and a processor for running the live application, generating the live interface, responding to the operation instructions, and controlling display of the live interface on the touch display screen.
In addition, the multimedia data transmission rate control method can be applied to a cloud live scene. Cloud live broadcast refers to a live broadcast mode based on cloud computing. In the cloud live broadcast operation mode, an operation main body of the live broadcast application program and a live broadcast picture presentation main body are separated, live broadcast picture presentation is completed at a cloud live broadcast client side, and the cloud live broadcast client side is mainly used for receiving and sending multimedia data and presenting live broadcast pictures. When live broadcasting is carried out, a user operates the cloud live broadcasting client to send an operation instruction to the cloud live broadcasting server, the cloud live broadcasting server operates a live broadcasting program according to the operation instruction, multimedia data such as live broadcasting pictures and the like are encoded and compressed, the multimedia data are returned to the cloud live broadcasting client through a network, and finally, the cloud live broadcasting client is used for decoding and outputting the live broadcasting pictures.
The method for controlling the multimedia data transmission rate can be executed on the terminal, can be executed on the server, and can be executed by the terminal and the server together. The above examples should not be construed as limiting the invention.
Referring to fig. 1, fig. 1 is a schematic diagram of a scenario of a multimedia data transmission rate control method according to an embodiment of the present invention. The multimedia system comprises a live client 101 and a live server 102, wherein the live client 101 is used for receiving at least two groups of multimedia data sent by the live server, and acquiring the total sending duration spent by the live server when the live server sends and the total receiving duration spent by the live client when the live client receives; the live client 101 determines a time delay gradient generated when each group of multimedia data is transmitted in a communication network between the live server 102 and the live client 101 according to the total sending time length and the total receiving time length of each group of multimedia data; the live client 101 performs smoothing processing on the time delay gradients to obtain a plurality of processed time delay gradients; the live client 101 determines congestion degree information of the communication network based on the change condition between the processed time delay gradients, and sends the congestion degree information to the live server 102; the live server 102 adjusts the size of the multimedia data transmission rate for the live client 101 according to the congestion degree information.
The example of fig. 1 is merely an example of a system architecture for implementing an embodiment of the present invention, and the embodiment of the present invention is not limited to the system architecture shown in fig. 1, and various embodiments of the present invention are proposed based on the system architecture of fig. 1.
The following will describe in detail. The following description of the embodiments is not intended to limit the preferred embodiments.
The embodiment of the invention provides a multimedia data transmission rate control method, which is applied to a multimedia system, wherein the multimedia system comprises a live broadcast server and a live broadcast client, the method can be executed by a processor of the live broadcast client and a processor of the live broadcast server, as shown in fig. 2, the specific flow of the multimedia data transmission rate control method can be as follows:
step 201, the live client receives at least two groups of multimedia data sent by the live server, and obtains a total sending duration spent when the live server sends and a total receiving duration spent when the live client receives in each group of multimedia data.
In the embodiment of the present invention, in a live scene, the multimedia data may be live data, for example, audio data, video data, live activity data, etc. forming a live picture. When a user performs live broadcast on the live broadcast client, the live broadcast server sends corresponding multimedia data to the live broadcast client, so that the live broadcast client receives and analyzes the multimedia data to form a live broadcast picture.
In some embodiments of the present invention, the multimedia data received by the live client may be multimedia data that has been divided into groups before being received, and the live client directly receives multiple groups of multimedia data for further processing. The multimedia data received by the live client may also be single multimedia data which does not form a multimedia data group, and then the received multimedia data is divided into a plurality of groups of multimedia data after the live client receives the multimedia data. The rule of dividing each multimedia data into multiple groups of multimedia data by the live broadcast client is not limited, and can be flexibly set according to practical situations, for example, the multimedia data groups can be divided according to the receiving sequence of the multimedia data, the multimedia data groups can be divided according to the types of the multimedia data, and the like.
In one embodiment of the present invention, the amount of multimedia data included in each set of multimedia data may be the same, may be partially the same, or may be different from each other. In addition, the multimedia data may be multimedia data transmitted in an arbitrary scene, for example, the multimedia data may be video stream data, audio stream data, or the like.
In some embodiments of the present invention, if the multimedia data included in each set of multimedia data are sequentially transmitted and received according to a time sequence, the method for obtaining the total transmission duration of each set of multimedia data in the live broadcast server may be: determining first transmission time of first multimedia data transmitted at a live broadcast server in each group of multimedia data, determining last transmission time of last multimedia data transmitted at the live broadcast server, calculating a first difference value between the last transmission time and the first transmission time, and taking the difference value as total transmission duration corresponding to each group of multimedia data. In addition, the method for obtaining the total time length of receiving the multimedia data of each group at the live client side can be as follows: determining first receiving time of first multimedia data received at the live client in each group of multimedia data, determining last receiving time of last multimedia data received at the live client, calculating second difference value between the last receiving time and the first receiving time, and taking the second difference value as total receiving duration corresponding to each group of multimedia data.
For example, in a transmission scenario diagram of a set of multimedia data as shown in fig. 3, a live broadcast server is in a TS i Starting to send a group of multimedia data to a live client at moment, and enabling a live server to send a group of multimedia data to a TS (transport stream) i+k The group of multimedia data is sent out all the time, and the live client is at TR i Starting to receive the group of multimedia data at moment, and enabling the live client to be at TR i+k The group of multimedia data is received all the time, and the total time length of the transmission of the group of multimedia data is TS i+k —TS i The total time length of the reception of the group of multimedia data is TR i+k —TR i 。
Step 202, the live client determines a time delay gradient generated when each group of multimedia data is transmitted in a communication network between the live server and the live client according to the total sending time length and the total receiving time length of each group of multimedia data.
In one embodiment of the invention, network congestion (network congestion) refers to a situation where network transmission performance is degraded due to limited resources of store-and-forward nodes when the number of packets transmitted in a packet-switched network is too large. Network congestion is a continuously overloaded network condition where users' demands for network resources (including link bandwidth, memory space, and processor processing power, etc.) exceed the inherent processing power and capacity.
In some embodiments of the present invention, if the transmission time of each multimedia data is the same in the transmission process of a group of multimedia data, it may indicate that the communication network is not congested in the transmission process of the multimedia data, and if the transmission time of each multimedia data is different, it may indicate that a portion of the multimedia data is delayed in the transmission process, it may indicate that the communication network is congested, and that the delay is serious, which may indicate that the congestion degree is serious.
In the embodiment of the present invention, the delay gradient is a criterion for indicating whether the communication network between the live server and the live client generates network congestion, and in the step 202, "the live client determines, according to the total sending duration and the total receiving duration of each group of multimedia data, the delay gradient generated when each group of multimedia data is transmitted in the communication network between the live server and the live client" may be:
and the live broadcast client calculates the difference value between the receiving total time length and the sending total time length corresponding to each group of multimedia data, and the difference value is used as a time delay gradient generated when each group of multimedia data is transmitted in a communication network.
The delay gradient generated by each group of multimedia data in the transmission process can represent the delay condition generated by the group of multimedia data in the transmission process outside the normal transmission time because of the network congestion degree. The congestion level information can be determined using the delay gradient. For example, as shown in FIG. 3, the calculation formula of the delay gradient may be ΔT i+k ＝(TR i+k -R i )-(TS i+k -S i )。
And 203, performing smooth processing on the time delay gradient by the live client to obtain a plurality of processed time delay gradients.
In the embodiment of the invention, since the multimedia data may shake during the transmission process of the communication network, when the delay gradients generated by each group of multimedia data are arranged in sequence based on the receiving time of each group of multimedia data, the situation that the delay gradients have larger phase differences with the delay gradients adjacent to the front and the back may occur, the delay gradient with larger phase differences may be used as abnormal multimedia data generated by shake, in order to more accurately judge the change situation of the delay gradients, avoid the interference of the abnormal multimedia data to the change situation, smooth each delay gradient by using a denoising filter, and generally, the smoothing process may be realized by using a low-pass filter. The denoising filter is a filter circuit consisting of a capacitor, an inductor and a resistor. The denoising filter can effectively filter the frequency points of the specific frequency or the frequencies outside the frequency points in the power line to obtain a power signal of the specific frequency or eliminate the power signal of the specific frequency. Specifically, the step of performing smoothing on the time delay gradient by the live client to obtain a plurality of processed time delay gradients may be:
The method comprises the steps that a live broadcast client determines a plurality of groups of to-be-processed time delay gradients from a plurality of time delay gradients, wherein one group of to-be-processed time delay gradients comprises at least two time delay gradients;
the live client acquires weights corresponding to the time delay gradients included in each group of time delay gradients to be processed;
the live client forms weighted time delay gradients corresponding to each group of time delay gradients to be processed according to the time delay gradients included in each group of time delay gradients to be processed and the weight corresponding to each time delay gradient;
the live client obtains a plurality of post-processing delay gradients based on each weighted delay gradient.
In the embodiment of the invention, each weighted time delay gradient can be used as a post-processing time delay gradient, each weighted time delay gradient and a part of time delay gradient can be used as a post-processing time delay gradient, and in addition, a time delay gradient which is not greatly different from the weighted time delay gradient can be selected as the post-processing time delay gradient.
In the embodiment of the invention, the number of the time delay gradients included in the group of time delay gradients to be processed is not limited. In addition, in order to make each weighted delay gradient generated after the smoothing process more accurately represent an unprocessed delay gradient, when the unprocessed delay gradients are arranged based on the receiving time of each group of multimedia data, the delay gradients included in one group of delay gradients to be processed may be adjacent delay gradients.
For example, the time delay gradients formed based on the sequential arrangement of the reception of the sets of multimedia data are T1, T2, T3, T4, T5, T6, the adjacent two time delay gradients are divided into a set of time delay gradients to be processed, T1, T2 is divided into one time delay gradient to be processed, T2, T3 is divided into one time delay gradient to be processed, T3, T4 is divided into one time delay gradient to be processed, T5, T6 is divided into one time delay gradient to be processed, the weight of the time delay gradients included in each time delay gradient to form the weighted time delay gradient is determined respectively, and the weighted time delay gradients formed are t1=a+t1+ (1-a), t2=b+t2+ (1-b), t3=c+t3+ (1-c) t4, t4=d+t4+ (1-d) and t5=t 5-e+1-6).
And 204, determining congestion degree information of the communication network based on the change condition between the processed time delay gradients by the live client, and sending the congestion degree information to the live server.
In one embodiment of the present invention, the congestion degree of the communication network may be determined according to the change condition between the plurality of post-processing delay gradients, specifically, the step 204 "the live client determines congestion degree information of the communication network based on the change condition between the post-processing delay gradients, and sends the congestion degree information to the live server" may be:
The method comprises the steps that a live broadcast client determines a first arrangement sequence of each time delay gradient based on a receiving sequence of each group of multimedia data;
the live client acquires the corresponding relation between each processed time delay gradient and each time delay gradient;
the live broadcast client determines a second arrangement sequence of the processed time delay gradient based on the first arrangement sequence and the corresponding relation;
the live client determines the change condition of the time delay gradient after each treatment under the second arrangement sequence;
and the live client side determines congestion degree information of the communication network based on the change condition and sends the congestion degree information to the live server.
For example, the order of receiving each group of multimedia data by the live client is data1, data2, data3, data4, the delay gradient formed according to data1 is m1, the delay gradient formed according to data2 is m2, the delay gradient formed according to data3 is m3, the delay gradient formed according to data4 is m4, the first arrangement sequence of each delay gradient is m1, m2, m3, m4, the post-processing delay gradient n1 is formed according to the delay gradients m1 and m2, the post-processing delay gradient n2 is formed according to the delay gradients m2 and m3, the post-processing delay gradient n3 is formed according to the delay gradients m3 and m4, the second arrangement sequence of the post-processing delay gradients is n1, n2, n3 is determined according to the receiving sequence of the multimedia data, and congestion degree information of the communication network is determined according to the change condition of n1 to n2 to n 3.
In the embodiment of the invention, in order to determine the congestion degree of the communication network based on the change condition of the processed time delay gradient more accurately, the change condition of the processed time delay gradient can be quantified, and as the processed time delay gradient is a scattered numerical value, a linear regression mode can be adopted to fit all the processed time delay gradients to form a straight line, so that the sum of Euclidean distances from all the processed time delay gradients to the straight line is minimum. Specifically, the step of determining, by the live client, a change condition of the delay gradient after each process in the second arrangement sequence may be:
the live client performs linear regression on the processed time delay gradients based on the second arrangement sequence to obtain a linear regression equation;
the direct broadcast client determines the change trend and the change rate of the time delay gradient after each treatment based on the slope of the linear regression equation;
and the live client takes the change trend and the change rate as the change condition of the time delay gradient after each treatment under the second arrangement sequence.
In one embodiment of the present invention, the change condition of the post-processing delay gradient, that is, whether the post-processing delay gradient is continuously increasing or continuously decreasing, may be determined according to the positive and negative of the slope of the linear regression equation, if the slope is greater than 0, it indicates that the post-processing delay gradient is in an increasing trend, and if the slope is less than 0, it indicates that the post-processing delay gradient is in a decreasing trend. The change rate refers to the increasing rate or decreasing rate of the post-treatment time delay gradient, if the slope is greater than 0 and the value of the slope is greater, the increasing rate of the post-treatment time delay gradient is indicated to be faster, and if the slope is smaller than 0 and the absolute value of the slope is greater, the decreasing rate of the post-treatment time delay gradient is indicated to be faster. For example, in the linear regression diagram shown in fig. 4, the post-processing delay gradients Δt formed by the arrangement are based on the reception times T of the respective sets of multimedia data t ' based on the delay after each processingGradient DeltaT t ' a linear regression line as shown in fig. 4 is formed.
In the embodiment of the present invention, the step 204 "determining, by the live client, congestion degree information of the communication network based on the change condition between the processed delay gradients" may be:
if the change trend is an increase trend and the last processed time delay gradient in the second arrangement sequence is larger than the time delay gradient threshold, the live client determines that the communication network is in a congestion state, and determines the congestion degree of the communication network when the communication network is in the congestion state according to the change rate;
the live client takes the congestion state and the congestion degree as congestion degree information of the communication network;
if the change trend is a decreasing trend and/or the last processed time delay gradient in the second arrangement sequence is not more than a time delay gradient threshold, the live client determines that the communication network is in an uncongested state, and determines the uncongested degree of the communication network when the communication network is in the uncongested state according to the change rate;
the live client takes the uncongested state and the uncongested degree as the congestion degree information of the communication network.
In one embodiment of the invention, the delay gradient threshold is a post-processing delay gradient that indicates that the communication network is in a congested state. If the change condition is a growing trend and the last processed time delay gradient is larger than the time delay gradient threshold value, the processed time delay gradient is larger and larger, namely the time delay which is increased in the transmission process of the multimedia data except for the normal transmission time delay is larger and larger, the communication network can be indicated to be in a congestion state, and if the processed time delay gradient is increased faster and quicker, the congestion degree of the communication network can be indicated to be larger and larger.
In one embodiment of the present invention, as long as the change condition is a decreasing trend, it may indicate that the post-processing delay gradient is smaller and smaller, that is, the delay added in the transmission process of the multimedia data is smaller and smaller except for the normal transmission delay, and it may indicate that the transmission of the multimedia data is faster and faster, at this time, whether the last post-processing delay gradient is greater than the delay gradient threshold value or not, it may be determined that the communication network is not in a congestion state.
In some embodiments of the present invention, if the change situation is not a decreasing trend, although the time delay added in the transmission process of the multimedia data is more and more except for the normal transmission time delay, the last processed time delay gradient is not greater than the time delay gradient threshold value, which indicates that the increase of the processed time delay gradient is still in the normal range, and although the bandwidth utilization of the communication network is increasing, it can be determined that the communication network is not in a congestion state yet.
In addition, if the change condition is a decreasing trend and the last processed delay gradient is not greater than the delay gradient threshold, the delay added in the transmission process of the multimedia data except the normal transmission delay is smaller and smaller, and the communication network can be judged not to be in a congestion state.
In the embodiment of the invention, the congestion degree information can only include the result that the live client judges whether the communication network is congested, so that the live server determines to reduce or increase the current multimedia data sending rate according to the result that the communication network is congested. In addition, in order to enable the live broadcast server to update the current multimedia data transmission rate more accurately according to the congestion degree of the communication network, the congestion degree information can further comprise a slope of a linear regression equation, so that the live broadcast server can update the current multimedia data transmission rate according to the size of the slope.
In the embodiment of the invention, after each multimedia data set is acquired to form the time delay gradient, congestion degree information can also be determined directly according to the change condition of the time delay gradient under the first arrangement sequence. And performing linear regression on the time delay gradient to generate a corresponding linear regression equation, and determining congestion degree information according to the slope of the linear regression equation corresponding to the time delay gradient.
Step 205, the live server adjusts the multimedia data sending rate of the live client according to the congestion degree information.
In one embodiment of the invention, after the live client forms the congestion degree information of the communication network, the congestion degree information of the communication network is fed back to the live server, and the way of feeding back the congestion degree information to the live server by the live client is not limited and can be flexibly determined according to actual conditions. For example, the live client may also send congestion level information to the live server via the communication network, and the live client may also feed back congestion level information to the live server via other networks.
In the embodiment of the present invention, after the live client receives the congestion degree information, it is required to obtain a result of whether the network state is congested, so as to adjust the multimedia data sending rate, if the communication network is in a congestion state, the live client may reduce the current multimedia data sending rate, and specifically, the step 205 "the live server adjusts the size of the multimedia data sending rate for the live client according to the congestion degree information" may be:
if the communication network is determined to be in a congestion state according to the congestion degree information, the live broadcast server reduces the multimedia data sending rate based on the congestion degree;
if the communication network is determined to be in an uncongested state according to the congestion degree information, the live broadcast server improves the multimedia data sending rate based on the uncongested state.
In order to enable the updated multimedia data transmission rate to alleviate network congestion to a greater extent, the live broadcast server needs to determine the congestion level, for example, according to the change rate in the congestion level information. If the communication network is not in a congestion state, the live client may directly promote the current multimedia data sending rate, but in order to make the updated multimedia data sending rate more widely utilize the bandwidth of the communication network, the live server needs to determine the degree of congestion (i.e. the degree of bandwidth unutilization) of the communication network, for example, may determine according to the change rate in the congestion degree information.
Specifically, the congestion degree information includes a change rate for determining the congestion degree, at this time, the change rate may be determined based on a slope of a linear regression equation formed by a delay gradient after the linear regression process, and specifically, the step of "the live broadcast server reduces the multimedia data transmission rate based on the congestion degree" may be: the live broadcast server acquires an inverse proportion relation between a change rate corresponding to the congestion degree and the first coefficient, and determines the first coefficient based on the inverse proportion relation and the change rate corresponding to the congestion degree; the live server decreases the multimedia data transmission rate based on the first coefficient.
Likewise, the step of "the live server increasing the multimedia data transmission rate based on the degree of uncongestion" may be: the direct broadcast server acquires a proportional relation between a change rate corresponding to the uncongested degree and a second coefficient, and determines the second coefficient based on the proportional relation and the change rate corresponding to the uncongested degree; the live broadcast server promotes the multimedia data sending rate based on the second coefficient.
In one embodiment of the present invention, if the communication network is congested, the slope of the linear regression equation is larger than 0, and if the slope is larger, the congestion degree when the communication network is congested is higher, the degree of reducing the current multimedia data transmission rate is larger. If the communication network is not congested and the slope of the linear regression equation is greater than 0, it indicates that the bandwidth utilization rate of the communication network is also increasing, and at this time, in order to avoid congestion of the communication network after updating the current multimedia data transmission rate, the current multimedia data transmission rate can be increased according to a smaller slope. If the communication network is not congested and the slope of the linear regression equation is smaller than 0, the network utilization rate of the communication network is reduced, and if the absolute value of the slope is larger, the uncongested degree of the communication network is larger, and the current multimedia data sending rate is increased to a larger degree.
In one embodiment of the present invention, before updating the multimedia data transmission rate, the current multimedia data transmission rate needs to be acquired, that is, the multimedia data transmission rate includes a code rate of transmitting multimedia data that has not been transmitted, and a code rate of retransmitting multimedia data packets, specifically, before the step 205 "the live server adjusts the size of the multimedia data transmission rate for the live client according to the congestion degree information", the method further includes:
the method comprises the steps that a live broadcast server obtains an input code rate of multimedia data sent to a live broadcast client by a current application layer;
the method comprises the steps that a live broadcast server determines a retransmission rate when retransmitting multimedia data to a live broadcast client;
the live server determines the size of the multimedia data transmission rate currently targeted to the live client based on the input code rate and the retransmission rate.
Wherein, the code rate refers to the number of bits of multimedia data transmitted in a unit time. For example, if the input rate is input_rate and the retransmission rate is retransmission_rate, the multimedia data transmission rate is send_rate=input_rate (1+retransmission_rate).
In one embodiment of the present invention, the specific procedure for updating the multimedia data transmission rate based on the congestion level may be: when it is determined that the communication network is not congested, the current send_rate value (i.e., send_rate=send_rate×α) is increased by a coefficient α. Wherein the coefficient α is proportional to the absolute value of the slope k value of the linear regression equation contained in the congestion degree information, i.e., α=ηxk. When it is determined that congestion occurs in the communication network, the send_rate is reduced by a factor β (send_rate=send_rate×β) until congestion is no longer checked. Where the coefficient β is inversely proportional to the absolute value of the k value, i.e., β=λ/k. The greater the absolute value of k, the more severe the network congestion or redundancy, and the faster the rate control can be converged. Wherein the values of eta and lambda can be set in the actual process.
The embodiment of the application provides a method, a device, equipment and a storage medium for controlling a multimedia data sending rate, wherein a live client can determine whether a communication network between the live client and a live server is congested and the congestion degree based on the sending total time length and the receiving total time length of at least two groups of multimedia data received from the live server, so that whether the congestion occurs or not is avoided through packet loss of the communication network, then the live client informs the live server of the congestion degree of the communication network, and further the live server can adjust the multimedia data sending rate according to the congestion degree, so that the problem that the multimedia data sending rate is reduced when the network bandwidth is enough but the packet loss rate is higher is avoided, the multimedia data can be stably sent when the network bandwidth is enough but the packet loss rate is higher, the multimedia data sending rate can be timely adjusted when the congestion scene is judged, and serious congestion is avoided.
Another multimedia data transmission rate control method according to the method described in the previous embodiment will be described in further detail below.
The embodiment of the application provides a multimedia data sending rate control method, as shown in fig. 5, the specific flow of the multimedia data sending rate control method can be as follows:
Step 501, the live client determines at least two groups of multimedia data received from the live server, wherein one group of multimedia data contains at least two multimedia data.
In the embodiment of the invention, the multimedia data received by the live client can be multimedia data which are already divided into groups before being received, and the live client directly receives a plurality of groups of multimedia data for further processing. The multimedia data received by the live client may also be single multimedia data which does not form a multimedia data group, and then the received multimedia data is divided into a plurality of groups of multimedia data after the live client receives the multimedia data.
Step 502, the live server obtains a total transmission duration spent by each group of multimedia data when the live server transmits, and a total reception duration spent when the live client receives.
In the embodiment of the present invention, the method for acquiring the total transmission time length of each group of multimedia data at the live server and the total reception time length at the live client by the live server are consistent with the method for acquiring the live client in the above embodiment, and are also determined based on the first difference value between the last transmission time and the first transmission time of each group of multimedia data and the second difference value between the last reception time and the first reception time of each group of multimedia data. And will not be described in detail herein.
Step 503, the live broadcast server determines a time delay gradient generated when each group of multimedia data is transmitted in a communication network between the live broadcast server and the live broadcast client according to the total sending time length and the total receiving time length of each group of multimedia data.
In one embodiment of the present invention, the live client also calculates a difference between the total received duration and the total transmitted duration corresponding to each group of multimedia data, as a delay gradient generated when each group of multimedia data is transmitted in the communication network.
And 504, performing smooth processing on the time delay gradient by the live broadcast server to obtain a plurality of processed time delay gradients.
And then based on the receiving time of each group of multimedia data, determining the change condition of the processed time delay gradient related to each group of multimedia data, if the change condition is an increasing trend, and the processed time delay gradient generated by the latest received group of multimedia data is larger than a time delay gradient threshold value, determining that the communication network between the live broadcast server and the live broadcast client is in a congestion state, and if the change condition is a decreasing trend and/or the processed time delay gradient generated by the latest received group of multimedia data is not larger than the time delay gradient threshold value, determining that the communication network between the live broadcast server and the live broadcast client is not in the congestion state.
Step 505, the live broadcast server determines congestion degree information of the communication network based on the change condition between the processed time delay gradients.
In the embodiment of the invention, if the communication network is not in a congestion state, the larger the absolute value of the change rate is, the higher the redundancy degree of the communication network is, and the lower the network resource utilization rate is. If the communication network is in a congestion state, the larger the absolute value of the change rate is, the higher the congestion degree of the communication network is.
And step 506, the live server adjusts the multimedia data sending rate of the live client according to the congestion degree information.
In the embodiment of the invention, after the live client receives the congestion degree information, the result of whether the network state is congested needs to be obtained, if the communication network is in the congestion state, the live client can reduce the current multimedia data sending rate, but in order to enable the updated multimedia data sending rate to relieve network congestion to a greater degree, the live server needs to judge the congestion degree, for example, can judge according to the slope in the congestion degree information. If the communication network is not in a congestion state, the live client may directly promote the current multimedia data sending rate, but in order to make the adjusted multimedia data sending rate capable of using the bandwidth of the communication network to a greater extent, the live server needs to determine the degree to which the communication network is not congested (i.e. the degree to which the bandwidth is not used), for example, may determine according to the slope in the congestion degree information.
In the structure diagram of the multimedia data transmission rate control method shown in fig. 6, after the live client receives multiple groups of multimedia data, the delay calculator calculates the total time spent by all multimedia data included in each group of multimedia data when the live server transmits, as the total transmission time spent by all multimedia data included in each group of multimedia data when the live client receives, and the delay calculator calculates the total time spent by all multimedia data included in each group of multimedia data when the live client receives, as the total reception time spent, the delay calculator makes a difference between the calculated total transmission time spent and the total reception time spent of each group of multimedia data, so as to obtain a delay gradient generated when each group of multimedia data is transmitted in a communication network, then the delay calculator transmits the obtained delay gradient to the denoising filter, the denoising filter removes abnormal multimedia data generated due to jitter in the received delay gradient, so as to obtain a weighted delay gradient, then the denoising filter transmits the weighted delay gradient to the congestion checker, and the congestion checker determines the congestion degree of the communication network according to the change condition of the weighted delay gradient. And finally, the live client transmits the congestion degree determined by the congestion checker to the live server, and the live server carries out rate control on the multimedia data transmission rate according to the received congestion degree.
As can be seen from the foregoing, according to the method for controlling a multimedia data transmission rate provided by the embodiment of the present application, a live client can determine whether congestion occurs in a communication network between the live client and a live server and the congestion degree based on a total transmission duration and a total reception duration of at least two sets of multimedia data received from the live server, so as to avoid determining whether congestion occurs by packet loss in the communication network, and then the live client informs the live server of the congestion degree of the communication network, so that the live server can adjust the multimedia data transmission rate according to the congestion degree, and the problem that the multimedia data transmission rate is reduced when the network bandwidth is sufficient but the packet loss rate is high is avoided, so that the multimedia data is stably transmitted when the network bandwidth is sufficient but the packet loss rate is high, and the multimedia data transmission rate is timely adjusted when judging a congestion occurrence scene is also avoided, so as to avoid serious congestion.
In order to better implement the above method, the embodiment of the present application further provides a multimedia data transmission rate control device, which is applied to a multimedia system, where the multimedia system includes a live broadcast server and a live broadcast client, and as shown in a schematic structural diagram of the multimedia data transmission rate control device in fig. 7, the multimedia data transmission rate control device may include a duration determining unit 701, a delay determining unit 702, a smoothing unit 703, a congestion determining unit 704, and a rate adjusting unit 705, as follows:
A time length determining unit 701, configured to receive, by a live client, at least two sets of multimedia data sent by a live server, and obtain a total sending time length spent by the live server in each set of multimedia data, and a total receiving time length spent by the live client when the live client receives the multimedia data;
the delay determining unit 702 is configured to determine, by the live client, a delay gradient generated when each group of multimedia data is transmitted in a communication network between the live server and the live client according to a total transmission duration and a total reception duration of each group of multimedia data;
a smoothing processing unit 703, configured to perform smoothing processing on the delay gradient through the live client, to obtain a plurality of processed delay gradients;
a congestion determining unit 704, configured to determine congestion degree information of the communication network based on the change condition between the processed delay gradients through the live client, and send the congestion degree information to the live server;
and the rate adjusting unit 705 is configured to adjust, by the live server, the size of the multimedia data sending rate for the live client according to the congestion degree information.
Optionally, the smoothing processing unit 703 is further configured to:
the method comprises the steps that a live broadcast client determines a plurality of groups of to-be-processed time delay gradients from a plurality of time delay gradients, wherein one group of to-be-processed time delay gradients comprises at least two time delay gradients;
The live client acquires weights corresponding to the time delay gradients included in each group of time delay gradients to be processed;
the live client forms weighted time delay gradients corresponding to each group of time delay gradients to be processed according to the time delay gradients included in each group of time delay gradients to be processed and the weight corresponding to each time delay gradient;
the live client obtains a plurality of post-processing delay gradients based on each weighted delay gradient.
Optionally, the congestion determining unit 704 is further configured to:
the method comprises the steps that a live broadcast client determines a first arrangement sequence of each time delay gradient based on a receiving sequence of each group of multimedia data;
the live client acquires the corresponding relation between each processed time delay gradient and each time delay gradient;
the live broadcast client determines a second arrangement sequence of the processed time delay gradient based on the first arrangement sequence and the corresponding relation;
the live client determines the change condition of the time delay gradient after each treatment under the second arrangement sequence;
and the live client side determines congestion degree information of the communication network based on the change condition and sends the congestion degree information to the live server.
Optionally, the congestion determining unit 704 is further configured to:
the live client performs linear regression on the processed time delay gradients based on the second arrangement sequence to obtain a linear regression equation;
The direct broadcast client determines the change trend and the change rate of the time delay gradient after each treatment based on the slope of the linear regression equation;
and the live client takes the change trend and the change rate as the change condition of the time delay gradient after each treatment under the second arrangement sequence.
Optionally, the congestion determining unit 704 is further configured to:
if the change trend is an increase trend and the last processed time delay gradient in the second arrangement sequence is larger than the time delay gradient threshold, the live client determines that the communication network is in a congestion state, and determines the congestion degree of the communication network when the communication network is in the congestion state according to the change rate;
the live client takes the congestion state and the congestion degree as congestion degree information of the communication network;
if the change trend is a decreasing trend and/or the last processed time delay gradient in the second arrangement sequence is not more than a time delay gradient threshold, the live client determines that the communication network is in an uncongested state, and determines the uncongested degree of the communication network when the communication network is in the uncongested state according to the change rate;
the live client takes the uncongested state and the uncongested degree as the congestion degree information of the communication network.
Optionally, the rate adjustment unit 705 is further configured to:
if the communication network is determined to be in a congestion state according to the congestion degree information, the live broadcast server reduces the multimedia data sending rate based on the congestion degree;
If the communication network is determined to be in an uncongested state according to the congestion degree information, the live broadcast server improves the multimedia data sending rate based on the uncongested state.
Optionally, the rate adjustment unit 705 is further configured to:
the live broadcast server acquires an inverse proportion relation between a change rate corresponding to the congestion degree and the first coefficient, and determines the first coefficient based on the inverse proportion relation and the change rate corresponding to the congestion degree;
the live broadcast server reduces the multimedia data sending rate based on the first coefficient;
the direct broadcast server acquires a proportional relation between a change rate corresponding to the uncongested degree and a second coefficient, and determines the second coefficient based on the proportional relation and the change rate corresponding to the uncongested degree;
the live broadcast server promotes the multimedia data sending rate based on the second coefficient.
Optionally, the rate adjustment unit 705 is further configured to:
the method comprises the steps that a live broadcast server obtains an input code rate of multimedia data sent to a live broadcast client by a current application layer;
the method comprises the steps that a live broadcast server determines a retransmission rate when retransmitting multimedia data to a live broadcast client;
the live server determines the size of the multimedia data transmission rate currently targeted to the live client based on the input code rate and the retransmission rate.
Optionally, the duration determining unit 701 is further configured to:
And the live broadcast client calculates the difference value between the receiving total time length and the sending total time length corresponding to each group of multimedia data, and the difference value is used as a time delay gradient generated when each group of multimedia data is transmitted in a communication network.
As can be seen from the foregoing, according to the multimedia data transmission rate control device provided by the embodiment of the present invention, the live client may determine whether congestion occurs in a communication network between the live client and the live server and the congestion degree based on the total transmission duration and the total reception duration of at least two sets of multimedia data received from the live server, so as to avoid determining whether congestion occurs by packet loss in the communication network, and then the live client may inform the live server of the congestion degree of the communication network, so that the live server may adjust the multimedia data transmission rate according to the congestion degree, and thus the problem that the multimedia data transmission rate is reduced when the network bandwidth is sufficient but the packet loss rate is high is avoided, and thus the multimedia data is stably transmitted when the network bandwidth is sufficient but the packet loss rate is high, and the multimedia data transmission rate is timely adjusted when judging that a congestion scene occurs, so as to avoid serious congestion.
In addition, the embodiment of the present invention further provides a computer device, which may be a terminal or a server, as shown in fig. 8, which shows a schematic structural diagram of the computer device according to the embodiment of the present invention, specifically:
the computer device may include one or more processors 901 of a processing core, one or more memories 902 of a computer readable storage medium, a power supply 903, and an input unit 904, among other components. Those skilled in the art will appreciate that the computer device structure shown in FIG. 8 is not limiting of the computer device and may include more or fewer components than shown, or may be combined with certain components, or a different arrangement of components. Wherein:
the processor 901 is a control center of the computer device, connects respective parts of the entire computer device using various interfaces and lines, and performs various functions of the computer device and processes multimedia data by running or executing software programs and/or modules stored in the memory 902 and calling the multimedia data stored in the memory 902, thereby performing overall detection of the computer device. Optionally, processor 901 may include one or more processing cores; preferably, the processor 901 may integrate an application processor and a modem processor, wherein the application processor primarily handles operating systems, user interfaces, applications, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 901.
The memory 902 may be used to store software programs and modules, and the processor 901 performs various functional applications and multimedia data processing by executing the software programs and modules stored in the memory 902. The memory 902 may mainly include a storage program area and a storage multimedia data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage multimedia data area may store multimedia data created according to the use of the computer device, etc. In addition, the memory 902 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 902 may also include a memory controller to provide access to the memory 902 by the processor 901.
The computer device further comprises a power supply 903 for powering the various components, preferably the power supply 903 is logically connected to the processor 901 via a power management system, whereby the functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 903 may also include one or more of any components, such as a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The computer device may also include radio frequency circuitry operable to transceive radio frequency signals to establish wireless communication with a network device or other computer device via wireless communication.
The computer device may also include an input unit 904, which input unit 904 may be used to receive input numeric or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the computer device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 901 in the computer device loads executable files corresponding to the processes of one or more application programs into the memory 902 according to the following instructions, and the processor 901 executes the application programs stored in the memory 902, so as to implement various functions as follows:
the method comprises the steps that a live client receives at least two groups of multimedia data sent by a live server, and obtains the total sending duration spent when the live server sends and the total receiving duration spent when the live client receives the multimedia data in each group;
The method comprises the steps that a live broadcast client determines a time delay gradient generated when each group of multimedia data is transmitted in a communication network between a live broadcast server and the live broadcast client according to the total sending time length and the total receiving time length of each group of multimedia data;
the live client performs smoothing treatment on the time delay gradients to obtain a plurality of treated time delay gradients;
the live client determines congestion degree information of the communication network based on the change condition between the processed time delay gradients, and sends the congestion degree information to the live server;
and the live broadcast server adjusts the multimedia data sending rate of the live broadcast client according to the congestion degree information.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
As can be seen from the foregoing, in this embodiment, the live client may determine whether congestion occurs in a communication network between the live client and the live server and the congestion degree based on the total transmission duration and the total reception duration of at least two sets of multimedia data received from the live server, so as to avoid determining whether congestion occurs by whether packet loss occurs in the communication network, and then the live client may inform the live server of the congestion degree of the communication network, so that the live server may adjust the multimedia data transmission rate according to the congestion degree, and the problem that the multimedia data transmission rate is reduced when the network bandwidth is sufficient but the packet loss rate is high is avoided, so that the multimedia data is stably transmitted when the network bandwidth is sufficient but the packet loss rate is high, and also when judging a congestion scene, the multimedia data transmission rate is timely adjusted to avoid serious congestion.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application provides a storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform the steps of any one of the multimedia data transmission rate control methods provided by the embodiment of the present application. For example, the instructions may perform the steps of:
the method comprises the steps that a live client receives at least two groups of multimedia data sent by a live server, and obtains the total sending duration spent when the live server sends and the total receiving duration spent when the live client receives the multimedia data in each group;
the method comprises the steps that a live broadcast client determines a time delay gradient generated when each group of multimedia data is transmitted in a communication network between a live broadcast server and the live broadcast client according to the total sending time length and the total receiving time length of each group of multimedia data;
the live client performs smoothing treatment on the time delay gradients to obtain a plurality of treated time delay gradients;
The live client determines congestion degree information of the communication network based on the change condition between the processed time delay gradients, and sends the congestion degree information to the live server;
and the live broadcast server adjusts the multimedia data sending rate of the live broadcast client according to the congestion degree information.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
Wherein the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The instructions stored in the storage medium can execute the steps in any of the methods for controlling the sending rate of multimedia data provided by the embodiments of the present application, so that the beneficial effects that any of the methods for controlling the sending rate of multimedia data provided by the embodiments of the present application can be achieved, which are detailed in the previous embodiments and are not described herein.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in various alternative implementations of the data downloading aspects described above.
The foregoing describes in detail a method, apparatus, device and storage medium for controlling a multimedia data transmission rate according to the embodiments of the present application, and specific examples are applied to illustrate the principles and embodiments of the present application, where the foregoing description of the embodiments is only for helping to understand the method and core idea of the present application; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present application, the present description should not be construed as limiting the present application in summary.

Claims (12)

1. The multimedia data transmission rate control method is applied to a multimedia system, and the multimedia system comprises a live broadcast server and a live broadcast client, and is characterized by comprising the following steps:
the live client receives at least two groups of multimedia data sent by the live server, and obtains the total sending duration spent by the live server in each group of multimedia data and the total receiving duration spent by the live client when the live client receives the live multimedia data;
the live broadcast client determines a time delay gradient generated when each group of multimedia data is transmitted in a communication network between the live broadcast server and the live broadcast client according to the total sending time length and the total receiving time length of each group of multimedia data;
The live broadcast client performs smooth processing on the time delay gradients to obtain a plurality of processed time delay gradients;
the live client determines congestion degree information of the communication network based on the change condition between the processed time delay gradients, and sends the congestion degree information to the live server;
and the live broadcast server adjusts the size of the multimedia data sending rate aiming at the live broadcast client according to the congestion degree information.
2. The method of claim 1, wherein the smoothing the delay gradient by the live client obtains a plurality of processed delay gradients, including:
the live broadcast client determines a plurality of groups of to-be-processed time delay gradients from a plurality of time delay gradients, wherein one group of to-be-processed time delay gradients comprises at least two time delay gradients;
the live client acquires weights corresponding to the time delay gradients included in each group of time delay gradients to be processed;
the live broadcast client forms weighted time delay gradients corresponding to each group of time delay gradients to be processed according to the time delay gradients included in each group of time delay gradients to be processed and the weight corresponding to each time delay gradient;
The live client obtains a plurality of processed time delay gradients based on each weighted time delay gradient.
3. The method of claim 1, wherein the live client determining congestion level information for the communication network based on the change between the post-processing latency gradients and transmitting the congestion level information to the live server comprises:
the live broadcast client determines a first arrangement sequence of the time delay gradients based on the receiving sequence of each group of multimedia data;
the live broadcast client acquires the corresponding relation between each processed time delay gradient and each time delay gradient;
the live broadcast client determines a second arrangement sequence of the processed time delay gradient based on the first arrangement sequence and the corresponding relation;
the live broadcast client determines the change condition of each processed time delay gradient under the second arrangement sequence;
and the live client determines congestion degree information of the communication network based on the change condition and sends the congestion degree information to the live server.
4. The method of claim 3, wherein the live client determining a change in each of the processed delay gradients in the second ordering comprises:
The live broadcast client performs linear regression on each processed time delay gradient based on the second arrangement sequence to obtain a linear regression equation;
the live client determines the change trend and the change rate of each processed time delay gradient based on the slope of the linear regression equation;
and the live client takes the change trend and the change rate as the change condition of each processed time delay gradient under the second arrangement sequence.
5. The method of claim 4, wherein the live client determining congestion level information for the communication network based on the change between the post-processing latency gradients comprises:
if the change trend is an increasing trend and the last processed time delay gradient in the second arrangement sequence is larger than a time delay gradient threshold, the live client determines that the communication network is in a congestion state, and determines the congestion degree of the communication network in the congestion state according to the change rate;
the live client takes the congestion state and the congestion degree as congestion degree information of the communication network;
if the change trend is a decreasing trend and/or the last processed time delay gradient in the second arrangement sequence is not greater than the time delay gradient threshold, the live client determines that the communication network is in an uncongested state, and determines the uncongested degree of the communication network when the communication network is in the uncongested state according to the change rate;
The live client takes the uncongested state and the uncongested degree as congestion degree information of the communication network.
6. The method of claim 5, wherein the live server adjusting the size of the multimedia data transmission rate for the live client according to the congestion degree information comprises:
if the communication network is determined to be in a congestion state according to the congestion degree information, the live broadcast server reduces the multimedia data sending rate based on the congestion degree;
and if the communication network is determined to be in an uncongested state according to the congestion degree information, the live broadcast server promotes the multimedia data sending rate based on the uncongested state.
7. The method of claim 6, wherein the live server reducing the multimedia data transmission rate based on the congestion level comprises:
the live broadcast server acquires an inverse proportion relation between the change rate corresponding to the congestion degree and a first coefficient, and determines the first coefficient based on the inverse proportion relation and the change rate corresponding to the congestion degree;
the live broadcast server reduces the multimedia data transmission rate based on the first coefficient;
The live broadcast server increases the multimedia data sending rate based on the uncongested degree, including:
the direct broadcast server obtains a proportional relation between the change rate corresponding to the uncongested degree and a second coefficient, and determines the second coefficient based on the proportional relation and the change rate corresponding to the uncongested degree;
the live broadcast server increases the multimedia data transmission rate based on the second coefficient.
8. The method of claim 1, wherein before the live server adjusts the size of the multimedia data transmission rate for the live client according to the congestion degree information, further comprising:
the live broadcast server obtains an input code rate of the multimedia data sent to the live broadcast client by the current application layer;
the live broadcast server determines the retransmission rate when retransmitting the multimedia data to the live broadcast client;
the live broadcast server determines the size of the multimedia data transmission rate currently aiming at the live broadcast client based on the input code rate and the retransmission rate.
9. The method of claim 1, wherein the determining, by the live client, a delay gradient generated when each group of multimedia data is transmitted in the communication network between the live server and the live client according to the total transmission duration and the total reception duration of each group of multimedia data comprises:
And the live broadcast client calculates the difference value between the receiving total time length and the sending total time length corresponding to each group of multimedia data, and the difference value is used as a time delay gradient generated when each group of multimedia data is transmitted in the communication network.
10. A multimedia data transmission rate control device applied to a multimedia system, wherein the multimedia system comprises a live broadcast server and a live broadcast client, and the multimedia data transmission rate control device is characterized by comprising:
the time length determining unit is used for receiving at least two groups of multimedia data transmitted by the live broadcast server through the live broadcast client, and acquiring the total transmission time length spent by the live broadcast server in each group of multimedia data and the total reception time length spent by the live broadcast client when the live broadcast client is received;
the time delay determining unit is used for determining a time delay gradient generated when each group of multimedia data is transmitted in a communication network between the live broadcast server and the live broadcast client according to the total sending time length and the total receiving time length of each group of multimedia data through the live broadcast client;
the smoothing processing unit is used for carrying out smoothing processing on the time delay gradient through the live broadcast client to obtain a plurality of processed time delay gradients;
The congestion determining unit is used for determining congestion degree information of the communication network based on the change condition between the processed time delay gradients through the live client and sending the congestion degree information to the live server;
and the rate adjusting unit is used for adjusting the size of the multimedia data sending rate aiming at the live client according to the congestion degree information through the live server.
11. A computer device, comprising:
a memory for storing a computer program;
a processor for implementing the steps in the multimedia data transmission rate control method according to any one of claims 1 to 9 when executing the computer program.
12. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps in the multimedia data transmission rate control method according to any one of claims 1 to 9.
CN202210312241.4A 2022-03-28 2022-03-28 Method, device, equipment and storage medium for controlling multimedia data transmission rate Pending CN116866622A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210312241.4A CN116866622A (en) 2022-03-28 2022-03-28 Method, device, equipment and storage medium for controlling multimedia data transmission rate

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210312241.4A CN116866622A (en) 2022-03-28 2022-03-28 Method, device, equipment and storage medium for controlling multimedia data transmission rate

Publications (1)

Publication Number Publication Date
CN116866622A true CN116866622A (en) 2023-10-10

Family

ID=88227313

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210312241.4A Pending CN116866622A (en) 2022-03-28 2022-03-28 Method, device, equipment and storage medium for controlling multimedia data transmission rate

Country Status (1)

Country Link
CN (1) CN116866622A (en)

Similar Documents

Publication Publication Date Title
US10271112B2 (en) System and method for dynamic adaptive video streaming using model predictive control
US9866791B2 (en) Efficiently distributing video content using a combination of a peer-to-peer network and a content distribution network
CN108307198B (en) Flow service node scheduling method and device and scheduling node
US9178929B2 (en) Client-side class-of-service-based bandwidth management in over-the-top video delivery
JP6442507B2 (en) How to allocate available network bandwidth during ongoing traffic sessions performed by devices on the network, corresponding devices
WO2013052255A1 (en) Improving adaptive streaming video quality by optimizing resource allocation
US8713097B2 (en) Adaptive period network session reservation
CN111225209A (en) Video data plug flow method, device, terminal and storage medium
Liu et al. Minimizing age-of-information with throughput requirements in multi-path network communication
CN107454136B (en) Calculation unloading method and device based on end-to-end P2P and control equipment
Chen et al. Edge-assisted short video sharing with guaranteed quality-of-experience
US20240106891A1 (en) Peer-to-peer network scheduling method and system
CN111897659B (en) Method, system and device for controlling service processing frequency and electronic equipment
CN109815204A (en) A kind of metadata request distribution method and equipment based on congestion aware
CN116866622A (en) Method, device, equipment and storage medium for controlling multimedia data transmission rate
CN114666318B (en) Streaming media data downloading method and device, electronic equipment and storage medium
CN114979091A (en) Data transmission method, related device, equipment and storage medium
CN110636104B (en) Resource request method, electronic device and storage medium
CN114416013A (en) Data transmission method, data transmission device, electronic equipment and computer-readable storage medium
WO2022218516A1 (en) Devices and methods for collaborative learning of a transmission policy in wireless networks
CN110505280B (en) P2P transmission control method and P2P node
CN109120480B (en) Link optimization method and device based on data center
Darwich et al. Adaptive Video Streaming: An AI-Driven Approach Leveraging Cloud and Edge Computing
WO2012158161A1 (en) Efficiently distributing video content using a combination of a peer-to-peer network and a content distribution network
US8495238B1 (en) Facilitating self-tuning traffic shaping without a central traffic manager

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination