CN114793299A - Streaming media transmission control method, system, device and medium - Google Patents

Streaming media transmission control method, system, device and medium Download PDF

Info

Publication number
CN114793299A
CN114793299A CN202210450904.9A CN202210450904A CN114793299A CN 114793299 A CN114793299 A CN 114793299A CN 202210450904 A CN202210450904 A CN 202210450904A CN 114793299 A CN114793299 A CN 114793299A
Authority
CN
China
Prior art keywords
streaming media
rendering delay
media data
average
delay
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210450904.9A
Other languages
Chinese (zh)
Other versions
CN114793299B (en
Inventor
雷鹏
付娆
陈民
张螣英
程宝平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
China Mobile Hangzhou Information Technology Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
China Mobile Hangzhou Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, China Mobile Hangzhou Information Technology Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN202210450904.9A priority Critical patent/CN114793299B/en
Publication of CN114793299A publication Critical patent/CN114793299A/en
Application granted granted Critical
Publication of CN114793299B publication Critical patent/CN114793299B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
    • H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention relates to the technical field of streaming media transmission, and discloses a streaming media transmission control method, a system, equipment and a medium. The streaming media transmission control method is applied to a sending end of a streaming media transmission control system. The invention obtains the coded streaming media data by obtaining the streaming media data and coding the streaming media data; counting the average encoding time length and the average media frame length of the encoded streaming media data, performing rendering delay estimation, and determining corresponding expected rendering delay; the encoded streaming media data and the expected rendering delay are sent to a receiving end, so that the receiving end can determine the corresponding actual rendering delay; receiving actual rendering delay, determining corresponding actual network bandwidth according to the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling the code rate, and controlling the streaming media congestion according to the regulated and controlled code rate; therefore, the congestion is controlled, and the dynamic change of the network is adapted.

Description

Streaming media transmission control method, system, device and medium
Technical Field
The present invention relates to the field of streaming media transmission technologies, and in particular, to a method, a system, a device, and a medium for controlling streaming media transmission.
Background
With the rapid growth of internet video traffic, streaming media transmission technology is also changing day by day. Streaming media video streaming is the main traffic in the current network, and during the transmission of streaming media, network congestion often occurs, and when congestion occurs, the most intuitive expression is that the quality of streaming media video streaming deteriorates. Therefore, there is a need to control streaming media transmission to reduce network congestion.
At present, the existing streaming media congestion control method is generally based on bandwidth estimation performed by a receiving end, and then the bandwidth estimation value is transmitted back to the sending end to perform code rate regulation. However, the existing bandwidth estimation method based on packet loss can effectively estimate the bandwidth when the network is already congested, and then congestion control is performed, which cannot prevent the congestion in the bud; when the network fluctuation is large, the network state cannot be estimated well.
Disclosure of Invention
The invention mainly aims to provide a streaming media transmission control method, a streaming media transmission control system, streaming media transmission control equipment and a streaming media transmission control medium, and aims to control congestion and adapt to dynamic changes of a network.
In order to achieve the above object, the present invention provides a streaming media transmission control method, which is applied to a sending end of a streaming media transmission control system, and the streaming media transmission control method includes the following steps:
acquiring streaming media data, and coding the streaming media data to obtain coded streaming media data;
counting the average coding duration and the average media frame length of the coded streaming media data, performing rendering delay estimation, and determining a corresponding expected rendering delay;
sending the streaming media data and the expected rendering delay to the receiving end so that the receiving end can determine the corresponding actual rendering delay;
and receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling a code rate, and controlling the congestion of the streaming media according to the regulated and controlled code rate.
Preferably, the step of counting the average encoding duration and the average media frame length of the encoded streaming media, performing rendering delay estimation, and determining the corresponding expected rendering delay includes:
counting the average coding duration and the average media frame length of the coded streaming media data;
if the expected rendering delay is calculated for the first time, performing rendering delay pre-estimation according to the average coding duration and the average media frame length, and determining the corresponding expected rendering delay;
if the expected rendering delay is not calculated for the first time, performing rendering delay pre-estimation according to the average coding time length, the average media frame length and the actual rendering delay of the last sampling period to determine the corresponding expected rendering delay.
Preferably, the step of receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average encoding duration, the average media frame length, and the actual rendering delay, and performing control on a code rate according to the controlled code rate includes:
receiving the actual rendering delay, and updating the expected rendering delay according to the actual rendering delay;
determining the actual network bandwidth corresponding to the streaming media data according to the average encoding duration, the average media frame length and the actual rendering delay;
and regulating and controlling the code rate of the streaming media data according to the actual network bandwidth, and controlling the congestion of the streaming media according to the regulated and controlled code rate.
Preferably, after the steps of receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average encoding duration, the average media frame length, and the actual rendering delay, and adjusting a code rate, and performing streaming media congestion control according to the adjusted code rate, the method for controlling streaming media transmission further includes:
and sending the actual network bandwidth to the receiving end so that the receiving end can decode and render the encoded streaming media data.
Preferably, after the step of sending the actual network bandwidth to the receiving end for the receiving end to decode and render the encoded streaming media data, the streaming media transmission control method further includes:
entering the next sampling period, and returning to the step: and acquiring streaming media data, and coding the streaming media data to obtain coded streaming media data.
The invention also provides a streaming media transmission control method, which is applied to a receiving end of a streaming media transmission control system and comprises the following steps:
receiving encoded streaming media data and expected rendering delay sent by a sending end, determining corresponding actual rendering delay, counting the average encoding duration and the average media frame length of the encoded streaming media data by the sending end for the expected rendering delay, and performing rendering delay pre-estimation determination;
sending the actual rendering delay to a sending end, so that the sending end determines a corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulates and controls a code rate, and controls the streaming media congestion according to the regulated code rate;
and receiving the actual network bandwidth sent by the sending end, and decoding and rendering the encoded streaming media data.
Preferably, the step of receiving the encoded streaming media data and the expected rendering delay sent by the sending end, and determining the corresponding actual rendering delay includes:
receiving coded streaming media data and expected rendering delay sent by a sending end, and acquiring a corresponding media sampling timestamp and receiving time according to the coded streaming media data;
determining corresponding rendering time according to the expected rendering delay and the media sampling time stamp;
and determining corresponding actual rendering delay according to the receiving time and the rendering time.
In addition, to achieve the above object, the present invention further provides a streaming media transmission control system, which is characterized in that the streaming media transmission control system includes:
the system comprises a sending end and a receiving end, wherein the sending end is used for acquiring streaming media data and coding the streaming media data to obtain coded streaming media data;
the sending end is used for counting the average coding duration and the average media frame length of the coded streaming media data, performing rendering delay estimation and determining the corresponding expected rendering delay;
a sending end, configured to send the streaming media data and the expected rendering delay to the receiving end;
a receiving end, configured to receive the streaming media data and the expected rendering delay, determine a corresponding actual rendering delay, and send the actual rendering delay to a sending end;
and the sending end is used for receiving the actual rendering delay, determining the corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling the code rate, and controlling the streaming media congestion according to the regulated and controlled code rate.
In addition, to achieve the above object, the present invention further provides a streaming media transmission control device, including:
the acquisition module is used for acquiring streaming media data and coding the streaming media data to obtain coded streaming media data;
the prediction module is used for counting the average coding duration and the average media frame length of the coded streaming media data, performing rendering delay prediction and determining a corresponding expected rendering delay;
a determining module, configured to send the average encoding duration, the average media frame length, and the streaming media data and the expected rendering delay to the receiving end, so that the receiving end determines a corresponding actual rendering delay;
and the control module is used for receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling a code rate, and controlling the congestion of the streaming media according to the regulated and controlled code rate.
In addition, to achieve the above object, the present invention further provides a streaming media transmission control device, including:
the receiving module is used for receiving the coded streaming media data and the expected rendering delay sent by the sending end and determining the corresponding actual rendering delay, wherein the expected rendering delay is determined by counting the average coding duration and the average media frame length of the coded streaming media data by the sending end and performing rendering delay pre-estimation;
a determining module, configured to send the actual rendering delay to a sending end, so that the sending end determines a corresponding actual network bandwidth based on the average coding duration, the average media frame length, and the actual rendering delay, regulates a code rate, and performs streaming media congestion control according to the regulated code rate;
and the decoding module is used for receiving the actual network bandwidth sent by the sending end, and decoding and rendering the encoded streaming media data.
In addition, to achieve the above object, the present invention further provides a device, where the device is a streaming media transmission control device, and the streaming media transmission control device includes: the streaming media transmission control method comprises the steps of a memory, a processor and a streaming media transmission control program which is stored on the memory and can run on the processor, wherein the steps of the streaming media transmission control method are realized when the streaming media transmission control program is executed by the processor.
In addition, to achieve the above object, the present invention further provides a medium, which is a computer-readable storage medium, wherein a streaming media transmission control program is stored on the computer-readable storage medium, and when being executed by a processor, the streaming media transmission control program implements the steps of the streaming media transmission control method as described above.
The invention provides a streaming media transmission control method, a system, equipment and a medium; the streaming media transmission control method is applied to a sending end of a streaming media transmission control system, the streaming media transmission control system comprises the sending end and a receiving end, and the streaming media transmission control method comprises the following steps: acquiring streaming media data, and encoding the streaming media data to obtain encoded streaming media data; counting the average encoding time length and the average media frame length of the encoded streaming media data, performing rendering delay estimation, and determining corresponding expected rendering delay; sending the streaming media data and the expected rendering delay to the receiving end so that the receiving end can determine the corresponding actual rendering delay; and receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling a code rate, and controlling the congestion of the streaming media according to the regulated and controlled code rate. The invention obtains the coded streaming media data by obtaining the streaming media data and coding the streaming media data; counting the average encoding time length and the average media frame length of the encoded streaming media data, performing rendering delay estimation, and determining corresponding expected rendering delay; the encoded streaming media data and the expected rendering delay are sent to a receiving end, so that the receiving end can determine the corresponding actual rendering delay; receiving actual rendering delay, determining corresponding actual network bandwidth according to the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling the code rate, and controlling the streaming media congestion according to the regulated and controlled code rate; therefore, the congestion is controlled, and the dynamic change of the network is adapted.
Drawings
Fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a streaming media transmission control method according to a first embodiment of the present invention;
fig. 3 is a schematic diagram of a work flow of a sending end of a streaming media transmission control system of the streaming media transmission control method of the present invention;
fig. 4 is a schematic diagram of a working flow of a receiving end of a streaming media transmission control system of the streaming media transmission control method according to the present invention;
fig. 5 is a schematic diagram of a framework of a streaming media transmission control system corresponding to the streaming media transmission control method of the present invention;
fig. 6 is a flowchart illustrating a streaming media transmission control method according to a second embodiment of the present invention;
fig. 7 is a flowchart illustrating a streaming media transmission control method according to a third embodiment of the invention;
fig. 8 is a flowchart illustrating a streaming media transmission control method according to a fourth embodiment of the present invention;
fig. 9 is a flowchart illustrating a streaming media transmission control method according to a fifth embodiment of the invention;
fig. 10 is a flowchart illustrating a streaming media transmission control method according to a sixth embodiment of the invention;
fig. 11 is a flowchart illustrating a streaming media transmission control method according to a seventh embodiment of the invention;
fig. 12 is a functional block diagram of a streaming media transmission control device according to a first embodiment of the invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
The device of the embodiment of the invention can be a mobile terminal or a server device.
As shown in fig. 1, the apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory such as a disk memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration of the apparatus shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a streaming control program.
The operating system is a program for managing and controlling the streaming media transmission control equipment and software resources, and supports the running of a network communication module, a user interface module, a streaming media transmission control program and other programs or software; the network communication module is used for managing and controlling the network interface 1002; the user interface module is used to manage and control the user interface 1003.
In the streaming media transmission control apparatus shown in fig. 1, the streaming media transmission control apparatus calls a streaming media transmission control program stored in a memory 1005 by a processor 1001 and performs operations in various embodiments of a streaming media transmission control method described below.
Based on the above hardware structure, the embodiment of the streaming media transmission control method of the present invention is provided.
Referring to fig. 2, fig. 2 is a flowchart illustrating a streaming media transmission control method according to a first embodiment of the present invention, where the streaming media transmission control method includes:
step S10, acquiring stream media data, and coding the stream media data to obtain coded stream media data;
step S20, counting the average coding duration and the average media frame length of the coded streaming media data, and performing rendering delay estimation to determine the corresponding expected rendering delay;
step S30, sending the streaming media data and the expected rendering delay to the receiving end, so that the receiving end determines a corresponding actual rendering delay;
step S40, receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average encoding duration, the average media frame length, and the actual rendering delay, adjusting and controlling a code rate, and performing streaming media congestion control according to the adjusted and controlled code rate.
The embodiment obtains the coded streaming media data by obtaining the streaming media data and coding the streaming media data; counting the average encoding time length and the average media frame length of the encoded streaming media data, performing rendering delay estimation, and determining corresponding expected rendering delay; transmitting the encoded streaming media data and the expected rendering delay to a receiving end so that the receiving end can determine the corresponding actual rendering delay; receiving actual rendering delay, determining corresponding actual network bandwidth according to the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling the code rate, and controlling the streaming media congestion according to the regulated and controlled code rate; therefore, the congestion is controlled, and the dynamic change of the network is adapted.
The respective steps will be described in detail below:
step S10, acquiring streaming media data, and encoding and transmitting the streaming media data.
In this embodiment, the streaming media transmission control method is applied to a sending end of a streaming media transmission control system, where the streaming media transmission control system includes the sending end and a receiving end.
The sending end comprises a media processing branch and a congestion control branch, and media processing and congestion control are performed in parallel; referring to fig. 3, fig. 3 is a schematic diagram illustrating a workflow of a transmitting end of the streaming media transmission control system, in fig. 3, a left side of fig. 3 is a workflow of a media processing branch of the transmitting end, and a right side of fig. 3 is a workflow of a congestion control branch of the transmitting end.
And the media processing branch cycle samples the streaming media data, and codes and transmits the streaming media data according to the current bandwidth estimation value.
The congestion control branch monitors the actual rendering delay fed back by the receiving end in real time, takes the actual rendering delay as input, and updates the expected rendering delay, namely, the updated expected rendering delay is the actual rendering delay.
Furthermore, updating the expected rendering delay by taking the actual rendering delay as an observed value, and calculating the corresponding actual network bandwidth bw by combining the average frame length and the average coding time; after the expected rendering delay and the actual network bandwidth bw are updated by the sending end, on one hand, the actual network bandwidth bw is set to the encoder to perform code rate control of the sending end, and on the other hand, the updated expected rendering delay is sent to the receiving end to perform rendering regulation and control.
The receiving end comprises a rendering delay receiving and processing branch and a media processing branch; referring to fig. 4, fig. 4 is a schematic diagram illustrating a workflow of a receiving end of a streaming media transmission control system, in fig. 4, a left side of fig. 4 is a workflow of a media processing branch of the receiving end, and a right side of fig. 4 is a workflow of a rendering delay receiving processing branch of the receiving end.
The rendering delay receiving and processing branch receives and stores the expected rendering delay set by the sending end in real time.
When the media processing branch receives each frame of streaming media data, the rendering time of the streaming media data is calculated according to the media sampling timestamp and the expected rendering delay, then the actual rendering delay is calculated according to the rendering time, and the actual rendering delay is fed back to the sending end.
Referring to fig. 4, fig. 4 is a schematic diagram of a framework of a streaming media transmission control system; the sending and receiving model of the small byte detection packet p is as follows:
Figure BDA0003618584270000081
wherein, TS cap,p For the transmission timestamp of probe packet p, rtt is the round trip time of the transceiver network, t offset Is the systematic time offset of the receiver relative to the sender. At the receiving side, a transmission and reception pattern based on the small byte detection packet p can be calculated by using one detection packet
Figure BDA0003618584270000082
Wherein, the clock deviation sum of the receiving end and the transmitting end is calculated by utilizing the small byte detection packet p of the transmitting end
Figure BDA0003618584270000083
And the sum, and further calculates the rendering delay, thereby avoiding the clock synchronization and rtt calculation errors of the transmitting and receiving ends.
The transmission and reception model for any streaming media data frame m is as follows:
Figure BDA0003618584270000084
the above formula represents the time relationship between any stream media data frame m from the collection to the receiving process of the receiving end; TS (transport stream) cap,m Is the sending time stamp of the streaming media data frame m, rtt is the network round-trip time of the receiving and sending party, t offset Is the systematic time offset of the receiver from the sender,/ m Average media frame length, t, for streaming media data frame m enc,m And bw is the average coding duration of the streaming media data frame m and the estimated value of the network bandwidth.
Wherein, under normal condition, the receiving end expects the time TS of decoding and rendering render,m Comprises the following steps:
TS render,m ＝TS rev,m +t dec
wherein, TS rev,m The time, t, from the acquisition to the receiving end of the streaming media data frame m dec Is the decoding time of the streaming media data frame m.
According to the above formula, the receiving end expects the time TS of decoding and rendering render,m The expression is as follows:
Figure BDA0003618584270000085
wherein, TS rev,m The time from the collection of the streaming media data frame m to the receiving process of the receiving end is rtt, the network round-trip time of the receiving end is rtt offset Is the systematic time offset of the receiver from the sender,/ m Average media frame length, t, for streaming media data frame m enc,m Is the average coding duration of the streaming media data frame m, bw is the estimated value of the network bandwidth, t dec Is the decoding time of the streaming media data frame m.
The method comprises the steps of obtaining streaming media data from different channels, and coding the streaming media data to obtain coded streaming media data. Referring to the right side of fig. 3, the media processing branch cycle at the transmitting end samples the streaming media data, and encodes the streaming media data according to the current bandwidth estimation value to obtain encoded streaming media data. The channel for acquiring the streaming media data is not limited in this embodiment.
Step S20, counting the average encoding duration and the average media frame length of the encoded streaming media data, performing rendering delay estimation, and determining the corresponding expected rendering delay.
In this embodiment, the average encoding duration and the average media frame length of the encoded streaming media data are counted through a congestion control branch of the transmitting end; the congestion control branch of the sending end comprises a Kalman filtering rendering delay estimation module; namely, according to the average encoding duration and the average media frame length of the encoded streaming media data, the Kalman filtering rendering delay estimation module is utilized to perform rendering delay estimation, and the expected rendering delay corresponding to the streaming media data is determined.
Step S30, sending the streaming media data and the expected rendering delay to the receiving end, so that the receiving end determines a corresponding actual rendering delay.
In an embodiment, the encoded streaming media data is sent to the receiving end through a media processing branch at the sending end. After receiving the encoded streaming media data at the receiving end, the encoded streaming media data comprises a sampling time stamp and a receiving time of the streaming media data; the sampling timestamp is the time when the sending end acquires the streaming media data; the receiving time is the time when the receiving end receives the coded streaming media data.
And sending the expected rendering delay to the receiving end through a congestion control branch of the sending end.
After the receiving end receives the encoded streaming media data and the expected rendering delay corresponding to the encoded streaming media data, the receiving end determines the actual rendering delay corresponding to the streaming media data according to the encoded streaming media data and the expected rendering delay. And determining the corresponding actual rendering delay through the media sampling time stamp and the receiving time of the streaming media data and the expected rendering delay.
And feeding back the actual rendering delay to a congestion control branch of the sending end, namely feeding back the actual rendering delay to a Kalman filtering rendering delay estimation module of the sending end.
Step S40, receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average encoding duration, the average media frame length, and the actual rendering delay, adjusting and controlling a code rate, and performing streaming media congestion control according to the adjusted and controlled code rate.
In this embodiment, the receiving end receives the actual rendering delay of the sending end, determines the actual network bandwidth corresponding to the encoded streaming media data according to the average encoding duration of the streaming media data, the average media frame length of the streaming media data, and the actual rendering delay, and regulates and controls the code rate according to the actual network bandwidth; controlling the congestion of the streaming media according to the regulated code rate; network fluctuation is sensed in advance by using the delay change information, and congestion is controlled.
In this embodiment, the encoded streaming media data is obtained by acquiring the streaming media data and encoding the streaming media data; counting the average encoding time length and the average media frame length of the encoded streaming media data, performing rendering delay estimation, and determining corresponding expected rendering delay; the encoded streaming media data and the expected rendering delay are sent to a receiving end, so that the receiving end can determine the corresponding actual rendering delay; receiving actual rendering delay, determining corresponding actual network bandwidth according to the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling the code rate, and controlling the streaming media congestion according to the regulated and controlled code rate; therefore, the congestion is controlled, and the dynamic change of the network is adapted.
Further, a second embodiment of the streaming media transmission control method is provided based on the first embodiment of the streaming media transmission control method of the present invention.
The difference between the first embodiment of the streaming media transmission control method and the second embodiment of the streaming media transmission control method is that, in this embodiment, for step S20, the average encoding duration and the average media frame length of the encoded streaming media data are counted, rendering delay prediction is performed, and refinement of a corresponding expected rendering delay is determined, and referring to fig. 6, this step specifically includes:
step S21, counting the average coding duration and the average media frame length of the coded streaming media data;
step S22, if the expected rendering delay is calculated for the first time, performing rendering delay pre-estimation according to the average coding duration and the average media frame length, and determining the corresponding expected rendering delay;
step S23, if the expected rendering delay is not calculated for the first time, performing rendering delay pre-estimation according to the average encoding duration, the average media frame length, and the actual rendering delay of the previous sampling period, and determining the corresponding expected rendering delay.
In this embodiment, the average encoding duration and the average media frame length of the encoded streaming media data are counted; when the expected rendering delay is calculated for the first time, performing rendering delay estimation according to the average coding duration and the average media frame length of the streaming media data, and determining the corresponding expected rendering delay; when the expected rendering delay is calculated for the non-first time, performing rendering delay pre-estimation according to the average coding duration of the streaming media data, the average media frame length and the actual rendering delay of the previous sampling period to determine the corresponding expected rendering delay; the expected rendering delay of the receiving end is calculated by using a Kalman filter at the transmitting end, and the rendering time of the receiving end is regulated, so that the feedback period of the streaming media data congestion control is shortened.
The respective steps will be described in detail below:
step S21, counting the average encoding duration and the average media frame length of the encoded streaming media data.
In this embodiment, the sending end includes a media processing branch and a congestion control branch, and performs media data processing and congestion control in parallel.
The congestion control branch of the sending end comprises a Kalman filtering rendering delay estimation module, and the Kalman filtering rendering delay estimation module comprises a Kalman filter. Counting the average coding time t of the coded streaming media data by a Kalman filtering rendering delay estimation module enc And the average media frame length l. Wherein, when the expected rendering delay is calculated for the first time, the average encoding duration t of the encoded streaming media data can be passed enc Determining a corresponding expected rendering delay t according to the average media frame length l delay 。
Step S22, if the expected rendering delay is calculated for the first time, the rendering delay is pre-estimated according to the average coding duration and the average media frame length, and the corresponding expected rendering delay is determined.
In this embodiment, when the expected rendering delay is calculated for the first time, the kalman filter rendering delay estimation module uses the average encoding duration of the encoded streaming media datat enc And averaging the media frame length l, performing rendering delay pre-estimation, and determining the corresponding expected rendering delay t delay 。
Desired rendering latency t delay The receiving end plays the streaming media data based on the time delay, and the streaming media data can be expected to be played smoothly; ideally this value is a fixed value. When the network changes, the desired rendering delay t delay With the change, it reflects the fluctuation of the network; and the gradient of this value can detect network changes in advance.
Further, by utilizing the desired rendering delay t delay Average media frame length l and average coding time length t enc An estimate of the network bandwidth bw may be calculated. Desired rendering latency t delay The expression is as follows:
Figure BDA0003618584270000111
wherein, t enc For the average encoding duration, l is the average media frame length, and bw is the estimated value of the network bandwidth.
Step S23, if the expected rendering delay is not calculated for the first time, performing rendering delay pre-estimation according to the average encoding duration, the average media frame length, and the actual rendering delay of the previous sampling period, and determining the corresponding expected rendering delay.
In this embodiment, when the expected rendering delay is not calculated for the first time, the kalman filter rendering delay estimation module uses the average encoding duration t of the encoded streaming media data enc Average media frame length l and actual rendering delay of last sampling period, performing rendering delay pre-estimation, and determining corresponding expected rendering delay t delay . Wherein, the actual rendering delay t of any ith frame delay,i The expression is as follows:
Figure BDA0003618584270000112
wherein, TS render,i For reception of any ith frameTime, TS, at which decoding and rendering is desired cap,i The transmission timestamp of the time probe packet p for any ith frame, rtt is the round trip time of the network of the transmitting and receiving party, t offset Is the systematic time offset, t, of the receiver from the sender dec Is the decoding time.
In this embodiment, the average encoding duration and the average media frame length of the encoded streaming media data are counted; when the expected rendering delay is calculated for the first time, performing rendering delay estimation according to the average coding duration and the average media frame length of the streaming media data, and determining the corresponding expected rendering delay; when the expected rendering delay is calculated for the non-first time, performing rendering delay pre-estimation according to the average coding duration of the streaming media data, the average media frame length and the actual rendering delay of the previous sampling period to determine the corresponding expected rendering delay; the expected rendering delay of the receiving end is calculated by using a Kalman filter at the transmitting end, and the rendering time of the receiving end is regulated, so that the feedback period of the streaming media data congestion control is shortened.
Further, based on the first and second embodiments of the streaming media transmission control method of the present invention, a third embodiment of the streaming media transmission control method of the present invention is provided.
The third embodiment of the streaming media transmission control method is different from the first and second embodiments of the streaming media transmission control method in that, in this embodiment, step S40 is performed, based on the average encoding duration, the average media frame length, and the actual rendering delay, a corresponding actual network bandwidth is determined, a code rate is regulated, and streaming media congestion control is refined according to the regulated code rate, with reference to fig. 7, the step specifically includes:
step S41, receiving the actual rendering delay and updating the expected rendering delay according to the actual rendering delay;
step S42, determining the actual network bandwidth corresponding to the streaming media data according to the average encoding duration, the average media frame length and the actual rendering delay;
step S43, according to the actual network bandwidth, regulating and controlling the code rate of the streaming media data, and according to the regulated and controlled code rate, performing streaming media congestion control.
In this embodiment, the actual rendering delay is received, and the expected rendering delay is updated according to the actual rendering delay; determining the actual network bandwidth corresponding to the streaming media data according to the average encoding duration, the average media frame length and the actual rendering delay; regulating and controlling the code rate of the streaming media data according to the actual network bandwidth, and controlling the congestion of the streaming media according to the regulated and controlled code rate; the actual network bandwidth estimation is carried out at the sending end, so that the code rate regulation and control can be flexibly and quickly carried out, and the method can be better suitable for scenes such as multiple receiving parties.
The respective steps will be described in detail below:
and step S41, receiving the actual rendering delay, and updating the expected rendering delay according to the actual rendering delay.
In this embodiment, in a rendering time control module at a receiving end, a corresponding actual rendering delay is obtained through a media sampling timestamp, a receiving time, and an expected rendering delay corresponding to streaming media data; and feeding back the actual rendering delay to a Kalman filtering rendering delay estimation module of the sending end through a rendering time control module.
In a Kalman filtering rendering delay estimation module of a sending end, actual rendering delay fed back by a rendering time control module of a receiving end is monitored in real time, and the actual rendering delay is delayed by t delay,i And the average coding time length t of the statistical streaming media data enc,i And average media frame length l i And as the input of a Kalman filtering rendering delay estimation module, updating the expected rendering delay to obtain the expected rendering delay of the next sampling period.
Step S42, determining an actual network bandwidth corresponding to the streaming media data according to the average encoding duration, the average media frame length, and the actual rendering delay.
In this embodiment, in the kalman filtering rendering delay estimation module at the transmitting end, the kalman filter delays the actual rendering delay t by the time t delay,i As an observationUpdating the value to expect the rendering delay; then, according to the average coding time t enc,i Average media frame length l i And the actual rendering delay t delay,i Calculating to obtain the actual network bandwidth corresponding to the streaming media data; actual network bandwidth bw i Is expressed as follows:
Figure BDA0003618584270000131
wherein, t enc,i To average code duration, l i To average media frame length, t delay,i Delay is actually rendered.
Step S43, according to the actual network bandwidth, regulating and controlling the code rate of the streaming media data, and according to the regulated and controlled code rate, performing streaming media congestion control.
In this embodiment, the actual network bandwidth bw is obtained according to the above calculation i Regulating and controlling the code rate of the streaming media data, and controlling the streaming media congestion according to the regulated and controlled code rate; network fluctuation can be sensed in advance through delay variation, and congestion can be controlled.
In this embodiment, the actual rendering delay is received, and the expected rendering delay is updated according to the actual rendering delay; determining the actual network bandwidth corresponding to the streaming media data according to the average encoding duration, the average media frame length and the actual rendering delay; regulating and controlling the code rate of the streaming media data according to the actual network bandwidth, and controlling the congestion of the streaming media according to the regulated and controlled code rate; the actual network bandwidth estimation is carried out at the sending end, so that the code rate regulation and control can be flexibly and rapidly carried out, and the method and the device can be better suitable for scenes such as multiple receiving parties.
Further, based on the first, second, and third embodiments of the streaming media transmission control method of the present invention, a fourth embodiment of the streaming media transmission control method of the present invention is provided.
The fourth embodiment of the streaming media transmission control method is different from the first, second, and third embodiments of the streaming media transmission control method in that, in this embodiment, after the step of receiving the actual rendering delay in step S40, determining a corresponding actual network bandwidth based on the actual rendering delay, and regulating and controlling a code rate, and performing streaming media congestion control according to the regulated code rate, with reference to fig. 8, the streaming media transmission control method further includes:
step a10, sending the actual network bandwidth to the receiving end, so that the receiving end decodes and renders the encoded streaming media data.
In this embodiment, the actual network bandwidth is sent to the receiving end, so that the receiving end decodes and renders the encoded streaming media data; therefore, the current actual congestion state is responded, and the optimization of the streaming media data transmission is realized.
The respective steps will be described in detail below:
step a10, sending the actual network bandwidth to the receiving end, so that the receiving end decodes and renders the encoded streaming media data.
In this embodiment, after the sending end determines the corresponding actual network bandwidth, regulates and controls the code rate according to the actual network bandwidth, and performs streaming media congestion control according to the regulated and controlled code rate, the actual network bandwidth is sent to the receiving end, so that the receiving end decodes and renders the encoded streaming media data by using the actual rendering delay and the actual network bandwidth; therefore, the current actual congestion state is responded, the code rate of the streaming media data is dynamically adjusted, and the smooth playing of the streaming media data is ensured.
In this embodiment, the actual network bandwidth is sent to the receiving end, so that the receiving end decodes and renders the encoded streaming media data; therefore, the current actual congestion state is responded, and the optimization of the streaming media data transmission is realized.
Further, a fifth embodiment of the streaming media transmission control method of the present invention is provided based on the first, second, third, and fourth embodiments of the streaming media transmission control method of the present invention.
The fifth embodiment of the streaming media transmission control method is different from the first, second, third and fourth embodiments of the streaming media transmission control method in that after the step a10 of sending the actual network bandwidth to the receiving end, so that the receiving end decodes and renders the encoded streaming media data, referring to fig. 9, the streaming media transmission control method further includes:
step B10, entering the next sampling period, and returning to the step: and acquiring streaming media data, and encoding the streaming media data to acquire encoded streaming media data.
In this embodiment, after the encoded streaming media data is decoded and rendered by the receiving end, the next sampling period is entered, and the step is returned: acquiring streaming media data, and encoding the streaming media data to obtain encoded streaming media data; thereby improving the efficiency of congestion control of streaming media data.
The respective steps will be described in detail below:
step B10, entering the next sampling period, and returning to the step: and acquiring streaming media data, and coding the streaming media data to obtain coded streaming media data.
In this embodiment, after the receiving end decodes and renders the encoded streaming media data according to the actual rendering delay and the actual network bandwidth, then enters the next sampling period, and returns to the step: acquiring streaming media data, and encoding the streaming media data to obtain encoded streaming media data; thereby improving the efficiency of congestion control of streaming media data.
In this embodiment, after the encoded streaming media data is decoded and rendered by the receiving end, the next sampling period is entered, and the step is returned: acquiring streaming media data, and encoding the streaming media data to obtain encoded streaming media data; thereby improving the efficiency of congestion control of streaming media data.
Referring to fig. 10, fig. 10 is a flowchart illustrating a streaming media transmission control method according to a sixth embodiment of the present invention, where the streaming media transmission control method is applied to a receiving end of a streaming media transmission control system, the streaming media transmission control system includes a sending end and a receiving end, and the streaming media transmission control method includes:
step C10, receiving the coded streaming media data and the expected rendering delay sent by the sending end, and determining the corresponding actual rendering delay, wherein the expected rendering delay is determined by the sending end counting the average coding duration and the average media frame length of the coded streaming media data, and performing rendering delay pre-estimation;
step C20, sending the actual rendering delay to a sending end, so that the sending end determines a corresponding actual network bandwidth based on the average encoding duration, the average media frame length and the actual rendering delay, regulates and controls a code rate, and controls the streaming media congestion according to the regulated code rate;
and step C30, receiving the actual network bandwidth sent by the sending end, and decoding and rendering the encoded streaming media data.
In this embodiment, a corresponding actual rendering delay is determined by receiving encoded streaming media data and an expected rendering delay sent by a sending end, and according to the encoded streaming media data and the expected rendering delay; sending the actual rendering delay to a sending end, so that the sending end determines the corresponding actual network bandwidth according to the average coding duration, the average media frame length and the actual rendering delay, regulates and controls the code rate, and controls the streaming media congestion according to the regulated and controlled code rate; receiving the actual network bandwidth sent by a sending end, and decoding and rendering the encoded streaming media data; therefore, the congestion is controlled, and the dynamic change of the network is adapted.
The respective steps will be described in detail below:
and step C10, receiving the encoded streaming media data and the expected rendering delay sent by the sending end, and determining the corresponding actual rendering delay, wherein the expected rendering delay is determined by counting the average encoding duration and the average media frame length of the encoded streaming media data by the sending end, and performing the prediction and determination of the rendering delay.
In this embodiment, receiving encoded streaming media data and an expected rendering delay sent by a sending end; after a receiving end receives the coded streaming media data and the expected rendering delay, wherein the coded streaming media data comprises a media sampling timestamp and receiving time corresponding to the coded streaming media data; and determining the corresponding actual rendering delay according to the expected rendering delay, the media sampling time stamp and the receiving time corresponding to the coded streaming media data.
The expected rendering delay is determined by counting the average encoding duration and the average media frame length of the encoded streaming media data by the transmitting end and performing rendering delay estimation according to the average encoding duration and the average media frame length.
And step C20, sending the actual rendering delay to a sending end, so that the sending end determines a corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulates and controls a code rate, and controls the streaming media congestion according to the regulated and controlled code rate.
In this embodiment, the actual rendering delay is sent to the sending end; the sending end determines the corresponding actual network bandwidth according to the average coding duration, the average media frame length and the actual rendering delay; furthermore, the sending end regulates and controls the code rate according to the actual network bandwidth, and controls the streaming media congestion according to the regulated and controlled code rate.
And the sending end sends the actual network bandwidth to the receiving end.
And step C30, receiving the actual network bandwidth sent by the sending end, and decoding and rendering the encoded streaming media data.
In this embodiment, the receiving end receives the actual network bandwidth sent by the sending end, and decodes and renders the encoded streaming media data according to the actual network bandwidth.
In this embodiment, the corresponding actual rendering delay is determined by receiving the encoded streaming media data and the expected rendering delay sent by the sending end, and according to the encoded streaming media data and the expected rendering delay; the actual rendering delay is sent to the sending end, so that the sending end determines the corresponding actual network bandwidth according to the average coding duration, the average media frame length and the actual rendering delay, regulates and controls the code rate, and controls the streaming media congestion according to the regulated and controlled code rate; receiving the actual network bandwidth sent by a sending end, and decoding and rendering the encoded streaming media data; therefore, the congestion is controlled, and the dynamic change of the network is adapted.
Further, a seventh embodiment of the streaming media transmission control method of the present invention is provided based on the first, second, third, fourth, fifth, and sixth embodiments of the streaming media transmission control method of the present invention.
The seventh embodiment of the streaming media transmission control method is different from the first, second, third, fourth, fifth, and sixth embodiments of the streaming media transmission control method in that the seventh embodiment of the streaming media transmission control method specifically includes, for step C10, receiving encoded streaming media data and an expected rendering delay sent by a sending end, and determining refinement of a corresponding actual rendering delay, referring to fig. 11:
step C11, receiving the coded streaming media data and the expected rendering delay sent by the sending end, and acquiring a corresponding media sampling timestamp and receiving time according to the coded streaming media data;
step C12, determining a corresponding rendering time according to the expected rendering delay and the media sampling time stamp;
and step C13, determining the corresponding actual rendering delay according to the receiving time and the rendering time.
In this embodiment, the encoded streaming media data and the expected rendering delay sent by the sending end are received, and a corresponding media sampling timestamp and a corresponding receiving time are obtained according to the encoded streaming media data; determining corresponding rendering time according to the expected rendering delay and the media sampling timestamp; determining corresponding actual rendering delay according to the rendering time and the receiving time; setting rendering time of the media frame according to the expected rendering delay through the receiving end, and feeding back the actual rendering delay to the sending end; therefore, the method can better adapt to the dynamic change of the network.
The respective steps will be described in detail below:
and step C11, receiving the encoded streaming media data and the expected rendering delay sent by the sending end, and obtaining a corresponding media sampling timestamp and receiving time according to the encoded streaming media data.
In this embodiment, the receiving end includes a rendering delay reception processing branch and a media processing branch, and performs media data processing and expected rendering delay reception processing in parallel. The receiving end obtains the coded streaming media data through the media processing branch, and the receiving end obtains the corresponding expected rendering delay through the rendering delay receiving processing branch.
After a receiving end receives the coded streaming media data and the expected rendering delay, wherein the coded streaming media data comprises a media sampling timestamp and receiving time corresponding to the coded streaming media data; the receiving end acquires the corresponding media sampling time stamp and the receiving time from the coded streaming media data.
And step C12, determining the corresponding rendering time according to the expected rendering delay and the media sampling time stamp.
In this embodiment, the receiving end includes a rendering delay receiving processing branch and a media processing branch, and performs media data processing and expected rendering delay receiving processing in parallel; the rendering delay receiving processing branch comprises a rendering time control module. The rendering time control module determines rendering time corresponding to the streaming media data according to the expected rendering delay and the media sampling time stamp; wherein the rendering time is the sum of the desired rendering delay and the media capture timestamp.
And step C13, determining the corresponding actual rendering delay according to the receiving time and the rendering time.
In this embodiment, the rendering time control module determines an actual rendering delay corresponding to the streaming media data according to the receiving time and the rendering time; wherein, the actual rendering delay is the difference between the rendering time and the receiving time.
In this embodiment, the encoded streaming media data and the expected rendering delay sent by the sending end are received, and a corresponding media sampling timestamp and a corresponding receiving time are obtained according to the encoded streaming media data; determining corresponding rendering time according to the expected rendering delay and the media sampling timestamp; determining corresponding actual rendering delay according to the rendering time and the receiving time; setting rendering time of the media frame according to the expected rendering delay through the receiving end, and feeding back the actual rendering delay to the sending end; therefore, the method can better adapt to the dynamic change of the network.
The invention also provides a streaming media transmission control device. Referring to fig. 12, the streaming media transmission control apparatus of the present invention includes:
an obtaining module 10, configured to obtain streaming media data, and encode the streaming media data to obtain encoded streaming media data;
the prediction module 20 is configured to count an average encoding duration and an average media frame length of the encoded streaming media data, perform rendering delay prediction, and determine a corresponding expected rendering delay;
a determining module 30, configured to send the streaming media data and the expected rendering delay to the receiving end, so that the receiving end determines a corresponding actual rendering delay;
and the control module 40 is configured to receive the actual rendering delay, determine a corresponding actual network bandwidth based on the average encoding duration, the average media frame length, and the actual rendering delay, regulate and control a code rate, and perform streaming media congestion control according to the regulated and controlled code rate.
Furthermore, the present invention also provides a medium, which is a computer-readable storage medium, on which a streaming media transmission control program is stored, and the streaming media transmission control program, when executed by a processor, implements the steps of the streaming media transmission control method as described above.
The method implemented when the streaming media transmission control program running on the processor is executed may refer to various embodiments of the streaming media transmission control method of the present invention, and details are not described here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or system in which the element is included.
The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solution of the present invention or the portions contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above and includes several instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present specification and the attached drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (10)

1. A streaming media transmission control method is characterized in that the streaming media transmission control method is applied to a sending end of a streaming media transmission control system, and the streaming media transmission control method comprises the following steps:
acquiring streaming media data, and encoding the streaming media data to obtain encoded streaming media data;
counting the average coding duration and the average media frame length of the coded streaming media data, performing rendering delay estimation, and determining a corresponding expected rendering delay;
sending the encoded streaming media data and the expected rendering delay to a receiving end so that the receiving end can determine the corresponding actual rendering delay;
and receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling a code rate, and controlling the congestion of the streaming media according to the regulated and controlled code rate.
2. The streaming media transmission control method according to claim 1, wherein the step of counting an average encoding duration and an average media frame length of the encoded streaming media data, performing rendering delay estimation, and determining a corresponding expected rendering delay comprises:
counting the average coding duration and the average media frame length of the coded streaming media data;
if the expected rendering delay is calculated for the first time, performing rendering delay pre-estimation according to the average coding duration and the average media frame length, and determining the corresponding expected rendering delay;
if the expected rendering delay is not calculated for the first time, performing rendering delay pre-estimation according to the average coding time length, the average media frame length and the actual rendering delay of the last sampling period, and determining the corresponding expected rendering delay.
3. The streaming media transmission control method according to claim 1, wherein the step of receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the average encoding duration, the average media frame length, and the actual rendering delay, and performing control on a code rate, and performing streaming media congestion control according to the controlled code rate includes:
receiving the actual rendering delay, and updating the expected rendering delay according to the actual rendering delay;
determining the actual network bandwidth corresponding to the streaming media data according to the average encoding duration, the average media frame length and the actual rendering delay;
and regulating and controlling the code rate of the streaming media data according to the actual network bandwidth, and controlling the congestion of the streaming media according to the regulated and controlled code rate.
4. The streaming media transmission control method according to claim 1, wherein after the step of receiving the actual rendering delay, determining a corresponding actual network bandwidth based on the actual rendering delay, and adjusting a code rate, and performing streaming media congestion control according to the adjusted code rate, the streaming media transmission control method further comprises:
and sending the actual network bandwidth to the receiving end so that the receiving end can decode and render the encoded streaming media data.
5. The streaming media transmission control method according to claim 1, wherein after the step of sending the actual network bandwidth to the receiving end for the receiving end to decode and render the encoded streaming media data, the streaming media transmission control method further comprises:
entering the next sampling period, and returning to the step: and acquiring streaming media data, and encoding the streaming media data to acquire encoded streaming media data.
6. A stream media transmission control method is characterized in that the stream media transmission control method is applied to a receiving end of a stream media transmission control system, and the stream media transmission control method comprises the following steps:
receiving encoded streaming media data and expected rendering delay sent by a sending end, determining corresponding actual rendering delay, counting the average encoding duration and the average media frame length of the encoded streaming media data by the sending end for the expected rendering delay, and performing rendering delay pre-estimation determination;
sending the actual rendering delay to a sending end, so that the sending end determines a corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulates and controls a code rate, and controls the streaming media congestion according to the regulated and controlled code rate;
and receiving the actual network bandwidth sent by the sending end, and decoding and rendering the encoded streaming media data.
7. The streaming media transmission control method of claim 6, wherein the step of receiving encoded streaming media data and an expected rendering delay sent by a sending end, and determining a corresponding actual rendering delay comprises:
receiving coded streaming media data and expected rendering delay sent by a sending end, and acquiring a corresponding media sampling timestamp and receiving time according to the coded streaming media data;
determining corresponding rendering time according to the expected rendering delay and the media sampling timestamp;
and determining corresponding actual rendering delay according to the receiving time and the rendering time.
8. A streaming media transmission control system, characterized in that the streaming media transmission control system comprises:
the system comprises a sending end and a receiving end, wherein the sending end is used for acquiring streaming media data and coding the streaming media data to obtain coded streaming media data;
the sending end is used for counting the average coding duration and the average media frame length of the coded streaming media data, performing rendering delay estimation and determining the corresponding expected rendering delay;
a sending end, configured to send the streaming media data and the expected rendering delay to the receiving end;
a receiving end, configured to receive the streaming media data and the expected rendering delay, determine a corresponding actual rendering delay, and send the actual rendering delay to a sending end;
and the sending end is used for receiving the actual rendering delay, determining the corresponding actual network bandwidth based on the average coding duration, the average media frame length and the actual rendering delay, regulating and controlling the code rate, and controlling the congestion of the streaming media according to the regulated and controlled code rate.
9. An apparatus, which is a streaming media transmission control apparatus, characterized by comprising: a memory, a processor and a streaming media transmission control program stored on the memory and executable on the processor, the streaming media transmission control program when executed by the processor implementing the steps of the streaming media transmission control method according to any one of claims 1 to 7.
10. A medium which is a computer-readable storage medium, characterized in that a streaming media transmission control program is stored on the computer-readable storage medium, and when executed by a processor, the steps of the streaming media transmission control method according to any one of claims 1 to 7 are implemented.
CN202210450904.9A 2022-04-27 2022-04-27 Streaming media transmission control method, system, equipment and medium Active CN114793299B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210450904.9A CN114793299B (en) 2022-04-27 2022-04-27 Streaming media transmission control method, system, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210450904.9A CN114793299B (en) 2022-04-27 2022-04-27 Streaming media transmission control method, system, equipment and medium

Publications (2)

Publication Number Publication Date
CN114793299A true CN114793299A (en) 2022-07-26
CN114793299B CN114793299B (en) 2024-05-24

Family

ID=82461232

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210450904.9A Active CN114793299B (en) 2022-04-27 2022-04-27 Streaming media transmission control method, system, equipment and medium

Country Status (1)

Country Link
CN (1) CN114793299B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115550713A (en) * 2022-11-29 2022-12-30 杭州星犀科技有限公司 Audio and video live broadcast rendering method, device, equipment and medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1571516A (en) * 2004-05-10 2005-01-26 牛广锋 Video coding
CN103686221A (en) * 2013-12-31 2014-03-26 北京大洋润博科技有限公司 Audio/video smooth broadcasting method adaptive to public wireless network
WO2017084277A1 (en) * 2015-11-16 2017-05-26 乐视控股（北京）有限公司 Code stream self-adaption method and system for online media service
CN109361494A (en) * 2018-11-29 2019-02-19 广州市百果园信息技术有限公司 A kind of processing method of audio data, device, equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1571516A (en) * 2004-05-10 2005-01-26 牛广锋 Video coding
CN103686221A (en) * 2013-12-31 2014-03-26 北京大洋润博科技有限公司 Audio/video smooth broadcasting method adaptive to public wireless network
WO2017084277A1 (en) * 2015-11-16 2017-05-26 乐视控股（北京）有限公司 Code stream self-adaption method and system for online media service
CN109361494A (en) * 2018-11-29 2019-02-19 广州市百果园信息技术有限公司 A kind of processing method of audio data, device, equipment and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115550713A (en) * 2022-11-29 2022-12-30 杭州星犀科技有限公司 Audio and video live broadcast rendering method, device, equipment and medium

Also Published As

Publication number Publication date
CN114793299B (en) 2024-05-24

Similar Documents

Publication Publication Date Title
US9191664B2 (en) Adaptive bitrate management for streaming media over packet networks
JP4504429B2 (en) Method and apparatus for managing media latency of voice over internet protocol between terminals
CN104735470B (en) A kind of streaming media data transmission method and device
US20080062322A1 (en) Digital video content customization
EP2612495B1 (en) Adaptive streaming of video at different quality levels
US9826260B2 (en) Video encoding device and video encoding method
US9781474B2 (en) Content playback information estimation apparatus and method and program
CN108494454A (en) Link-aware steaming transfer is adaptive
US20150271231A1 (en) Transport accelerator implementing enhanced signaling
KR20180109890A (en) Information processing apparatus and information processing method
CN106982371B (en) Code stream control method and system, code stream management method and system, and terminal equipment
CN110913245A (en) Method and device for controlling video transcoding code rate
CN111093083B (en) Data transmission method and device
US20110235709A1 (en) Frame dropping algorithm for fast adaptation of buffered compressed video to network condition changes
US20120303833A1 (en) Methods for transmitting and receiving a digital signal, transmitter and receiver
JP2005513876A (en) System and method for modifying a video stream based on a client or network environment
JP2010119098A (en) Communication apparatus, communication method for communication apparatus, and communication control program for communication apparatus
CN114793299A (en) Streaming media transmission control method, system, device and medium
CN111617466A (en) Method and device for determining coding format and method for realizing cloud game
US9641445B2 (en) Communication apparatus, communication method, and program
JP2022091710A (en) Multiple-protocol prediction and in-session adaptation in video streaming
CN113473125B (en) Code rate control method, equipment, storage medium and product
KR101632012B1 (en) Communication system, server apparatus, server apparatus controlling method and computer readable storage medium storing program
JP2010063004A (en) Video data transmission apparatus, video distribution system, video quality evaluation method and program
CN106936808B (en) HTTP streaming media transmission method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant