CN114244778A - QoE-aware WebRTC congestion control method - Google Patents
QoE-aware WebRTC congestion control method Download PDFInfo
- Publication number
- CN114244778A CN114244778A CN202210016796.4A CN202210016796A CN114244778A CN 114244778 A CN114244778 A CN 114244778A CN 202210016796 A CN202210016796 A CN 202210016796A CN 114244778 A CN114244778 A CN 114244778A
- Authority
- CN
- China
- Prior art keywords
- frame
- information
- data packet
- receiving
- sending
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000005540 biological transmission Effects 0.000 claims abstract description 59
- 238000013139 quantization Methods 0.000 claims abstract description 49
- 230000008569 process Effects 0.000 claims description 13
- 230000006798 recombination Effects 0.000 claims description 13
- 238000005215 recombination Methods 0.000 claims description 13
- 238000012790 confirmation Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 108091026890 Coding region Proteins 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0268—Traffic management, e.g. flow control or congestion control using specific QoS parameters for wireless networks, e.g. QoS class identifier [QCI] or guaranteed bit rate [GBR]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a QoE-aware WebRTC congestion control method, which comprises the following steps: acquiring basic information of a sending frame in a network layer, recombining the basic information of the sending frame, recording the recombined information of the sending frame, and determining a sending frame record; receiving a data packet sent by a sending end, recombining the information of a receiving frame based on a sending frame record, and determining a receiving frame rate and a quantization parameter of a corresponding frame according to the information of the recombined receiving frame; confirming data packet information received by a receiving end, acquiring the corresponding sending time and receiving time of each data packet, calculating queuing delay according to the corresponding sending time and receiving time of each data packet, and acquiring queuing delay data; and updating the network transmission rate in real time based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and the queuing delay data corresponding to the received data packet. The invention can control the network transmission rate in real time according to the acquired quantization parameter and the received frame rate information.
Description
Technical Field
The invention relates to the technical field of Web real-time communication, in particular to a QoE-aware WebRTC congestion control method.
Background
At present, with the improvement of network infrastructure performances such as the internet and the like and the increasing popularization of 5G networks, more and more funds, enterprises and research and development personnel are invested in industries based on RTC technology such as video conferences, Cloud games, Cloud XR and the like, and as the RTC field integrating technologies such as graphic rendering, audio and video coding and decoding, network transmission and the like has higher technical threshold, a set of complete RTC system from zero research and development needs to invest a large amount of manpower, financial resources and time resources, so many enterprises and individuals can rely on or directly use the open-source WebRTC solution of Google.
The open-source WebRTC system of Google is a complete solution in accordance with the WebRTC standard, and a GCC (Google Congestion control) Congestion control method of Google is adopted at the bottom layer, models the change of queuing delay in transmission, predicts the change of the queuing delay on a network transmission path, and provides a corresponding rate control scheme, and feeds back the rate control scheme to the output of an encoder control encoder, so that the operation of the whole RTC system is driven. It can be said that a suitable congestion control scheme is a driving engine of the entire RTC system, which can efficiently drive the operation of the entire system. The GCC is based on the data packet sending and receiving information of the network layer during decision making, and the transmission delay of the data packet in the network has certain randomness due to the influence of noise, so that the network state of the bottom layer is difficult to be accurately sensed. The WebRTC system bears audio and video delay-sensitive applications, and if the real-time experience of a receiving end user can be acquired in real time at a network layer of a transmitting end, reference information of a congestion control decision can be enriched, so that the decision of congestion control is more scientific and reasonable. On the premise of not changing the whole frame of the WebRTC, the decision of acquiring the application layer information at the network layer and carrying out congestion control based on the application layer information has practical application value.
Disclosure of Invention
The invention provides a QoE-aware WebRTC congestion control method, which is used for solving the problem that a network layer cannot directly acquire related information of an application layer.
A QoE-aware WebRTC congestion control method, comprising:
acquiring basic information of a sending frame in a network layer, recombining the basic information of the sending frame, acquiring sending frame recombination information and a sent data packet, recording the sending frame recombination information, and determining a sending frame record;
receiving a data packet sent by a sending end, recombining the information of a receiving frame based on the sending frame record, acquiring the recombined information of the receiving frame, and determining a receiving frame rate and a quantization parameter of a corresponding frame according to the recombined information of the receiving frame;
confirming data packet information received by a receiving end, acquiring the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet;
and updating the network transmission rate in real time based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and the queuing delay data corresponding to the received data packet.
As an embodiment of the present invention: the acquiring basic information of a transmission frame in a network layer, recombining the basic information of the transmission frame, acquiring recombination information of the transmission frame and a transmitted data packet, recording the recombination information of the transmission frame, and determining a transmission frame marker includes:
splitting the sending frame according to the basic information of the sending frame in the network layer to obtain the information of the sent data packet, and recording the information of the sent data packet according to the content record item to determine the information record content; wherein the content record item comprises: the data Packet corresponds to an RTP timestamp, a Packet type and a coding quantization parameter corresponding to the data Packet;
defining the coding frame structure according to the information recording content and preset definition content, and determining coding frame definition information; wherein the definition content comprises: dividing the coding frame to obtain the sequence information of a first data packet and the sequence information of a last data packet;
and acquiring the sending time corresponding to the sent data packet according to the sent data packet information, and recombining the sending frame data.
As an embodiment of the present invention: the sending time corresponding to the sent data packet is obtained according to the sent data packet information, and the sending frame data is recombined, wherein the executing process comprises the following steps:
acquiring transmitted data packet information corresponding to a transmitting end, traversing video coding frames according to the transmitted data packet information, and acquiring video coding frames matched with data packets;
acquiring the current timestamp of the sent data packet based on the matching information of the sent data packet and the video coding frame, judging whether the current timestamp is consistent with the recorded RTP timestamp, and determining a judgment result;
when the judgment result shows that the current timestamp is inconsistent with the recorded RTP timestamp, reconstructing a coding frame structure;
and when the judgment result shows that the current timestamp is consistent with the recorded RTP timestamp, setting the serial number of the data packet as the serial number of the last data packet of the corresponding sending frame.
As an embodiment of the present invention: when the judgment result shows that the current timestamp is inconsistent with the recorded RTP timestamp, reconstructing a coding frame structure, including:
acquiring serial number information corresponding to a first data packet of a sending frame according to the serial number information corresponding to the sending frame;
and acquiring the time stamp of the current data packet, and setting the time stamp of the current data packet as the RTP time stamp corresponding to the current data packet.
As an embodiment of the present invention: the receiving of the data packet sent by the sending end and the reconstruction of the receiving frame information based on the sending frame record, the obtaining of the receiving frame reconstruction information, and the determination of the receiving frame rate and the quantization parameter of the corresponding frame according to the receiving frame reconstruction information, include:
acquiring data packet confirmation information received by a receiving end, traversing video coding frames according to the data packet confirmation information, and acquiring the video coding frames matched with the data packets;
searching a first serial number, a last serial number and a corresponding quantization parameter of a data packet of a corresponding frame according to the sending frame record and calculating the number of the data packets of the corresponding frame according to the sending frame record;
traversing the receiving record of the receiving end, judging whether the receiving record contains the receiving information of the frame,
if so,
the current frame reception time is updated to the reception time of the corresponding data packet,
updating the receiving quantity of the data packets corresponding to the current frame;
judging whether the current frame is completely received, if so, marking the current frame and determining to finish receiving;
if the sending frame in the sending frame record is not received, reconstructing the received frame data, and updating the receiving time and the receiving quantity corresponding to the received frame;
obtaining the receiving time T of the last frame, and counting the 500ms interval [ T-500, T]The number of complete frames received within frame _ num determines the frame rate fps receivedrecv＝2×frame_num；
And counting the quantization parameters of the received frame in a 500ms interval [ T-500, T ] according to a formula smooth _ qp of 0.5 × smooth _ qp +0.5 × qp, wherein qp is the quantization parameter of the current frame in the traversal process, and smooth _ qp is the quantization parameter after smoothing.
As an embodiment of the present invention: the method comprises the following steps of confirming data packet information received by a receiving end, acquiring the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet, wherein the execution steps comprise:
obtaining the confirmation information of the data Packet received by the receiving end, and judging B for each confirmed data Packets,BrIf the value is null, when Bs,BrIf the value is empty, set Bs＝Ps,Br＝PrWherein B iss,BrRespectively representing the basic transmission and reception times, P, of a Packets,PrRespectively representing the sending time and the receiving time of a Packet;
when B is presents,BrIf the value is not null, for each Packet received, according to the formula qdelay ═ Pr-Br)-(Ps-Bs) Calculating queuing delay, and adding the calculated queuing delay qdelay into a container qdelay, wherein qdelay represents the result of the calculation of the queuing delay of each Packet;
calculating a smooth delay smoothjdelay of 0.875 × smoothjdelay +0.125 × delay for the queuing delay in the last 100ms in the container qdelay, wherein delay is each delay in the traversal process;
for each Packet confirmed this time, if the calculation result qdelay is less than 0, updating Bs＝Ps,Br＝Pr。
As an embodiment of the present invention: the method comprises the following steps of obtaining a QoE index based on the one-way queuing delay information, and updating a network transmission rate in real time according to the QoE index, wherein the execution steps comprise:
step 1: presetting parameters, namely a highest-quality target quantization parameter high _ target _ qp, a target receiving frame rate high _ target _ fps, a lowest-quality target quantization parameter low _ target _ qp and a target receiving frame rate low _ target _ fps;
step 2: judging whether smooth _ delay is greater than 100 and is in a slow starting state currently, and if smooth _ delay is greater than 100 and is in the slow starting state currently, exiting the slow starting state;
and step 3: if the current state is a slow start state, acquiring the last rate updating time, and if the last rate updating time exceeds 100ms, setting the affected _ bitrate to be 1.5 multiplied affected _ bitrate, wherein the affected _ bitrate represents the rate to be transmitted at this time;
and 4, step 4: if the current state is not in the slow start state and smooth _ delay is less than 100, if smooth _ qp is less than high _ target _ qp and fpsrecvWhen the data transmission rate is higher than high _ target _ fps, the transmission rate is kept unchanged; when smooth _ qp < low _ target _ qp and fpsrecvIf the time exceeds 100ms from the last update time, setting the affected _ bitrate to be 1.05 × affected _ bitrate; in other cases, every time the last update time exceeds 100ms, set the affected _ bitrate equal to 1.08 × affected _ bitrate;
and 5: if the current state is not in the slow start state and smooth _ delay is larger than 250, setting an acknowledged _ bitrate-ack _ bitrate, wherein the ack _ bitrate takes the minimum value of the sending rate and the receiving rate in the latest 500 ms;
step 6: if the current state is not in the slow start state and the smooth _ delay is more than or equal to 100 and less than or equal to 250, if smooth _ qp is less than low _ target _ qp and fpsrecvIf the transmission speed is higher than low _ target _ fps, keeping the transmission speed unchanged; otherwise, the transmission rate is set to 1.05 × affected _ bitrate every time when the last update time exceeds 100 ms.
The beneficial effects of the invention are as follows: the invention recombines information such as quantization parameter of video coding, receiving frame rate of the receiver, etc. in the network layer, as the important basis of the congestion control decision; congestion control methods that meet various user experience criteria can be designed based on these application layer metrics; a QoE-aware congestion control algorithm is designed based on the acquired information such as the receiving frame rate, the quantization parameter of video frame coding and the like, and in combination with real-time queuing delay, decision can be made in real time based on the information such as the receiving frame rate and the quantization parameter of the current receiving end and the queuing delay in the current network, so that the congestion control has pertinence, and the user experience requirements can be better met.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention.
In the drawings:
fig. 1 is a flow chart of a QoE-aware WebRTC congestion control method structure in an embodiment of the present invention;
fig. 2 is a flowchart of sending frame reassembly in a QoE-aware WebRTC congestion control method according to an embodiment of the present invention;
fig. 3 is a flowchart of received frame reassembly in a QoE-aware WebRTC congestion control method according to an embodiment of the present invention.
Fig. 4 is a flow chart of congestion control in a QoE-aware WebRTC congestion control method in an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions, and "plurality" means two or more unless specifically limited otherwise. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Example 1:
the embodiment of the invention provides a QoE-aware WebRTC congestion control method, which comprises the following steps:
acquiring basic information of a sending frame in a network layer, recombining the basic information of the sending frame, acquiring sending frame recombination information and a sent data packet, recording the sending frame recombination information, and determining a sending frame record;
receiving a data packet sent by a sending end, recombining the information of a receiving frame based on the sending frame record, acquiring the recombined information of the receiving frame, and determining a receiving frame rate and a quantization parameter of a corresponding frame according to the recombined information of the receiving frame;
confirming data packet information received by a receiving end, acquiring the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet;
updating the network transmission rate in real time based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and the queuing delay data corresponding to the received data packet;
the implementation principle of the invention is as follows: in the prior art, the data packet is obtained to send and receive information in the network layer in the data transmission process, but the transmission delay of the data packet in the network layer has certain randomness due to the influence of noise, so that the network state of the bottom layer is difficult to be accurately sensed; the invention reproduces the quantization parameter of the video coding of the sending end and the receiving frame rate of the receiving end as the reference information of the congestion control;
the beneficial effects of the above technical scheme are: the invention reproduces the quantization parameter of the video coding of the sending end and the receiving frame rate of the receiving end and other information at the network layer under the condition of not changing the Google open source WebRTC frame structure and only slightly changing the source code, enriches the reference information of the congestion control decision, can be used as the important reference of the congestion control decision, and is beneficial to effectively controlling the network transmission rate in real time according to the user experience of the receiving end based on the QoE information of the obtained coding video quantization parameter, the receiving frame rate and the like, is beneficial to preventing the congestion from occurring in the data transmission, can make a decision in real time based on the information of the receiving frame rate and the quantization parameter of the current receiving end, the queuing delay and the like in the current network, so that the congestion control has pertinence, and the user experience requirements can be better met.
Example 2:
in one embodiment of the invention: the acquiring basic information of a transmission frame in a network layer, recombining the basic information of the transmission frame, acquiring recombination information of the transmission frame and a transmitted data packet, recording the recombination information of the transmission frame, and determining a transmission frame marker includes:
splitting the sending frame according to the basic information of the sending frame in the network layer to obtain the information of the sent data packet, and recording the information of the sent data packet according to the content record item to determine the information record content; wherein the content record item comprises: the data Packet corresponds to an RTP timestamp, a Packet type and a coding quantization parameter corresponding to the data Packet;
defining the coding frame structure according to the information recording content and preset definition content, and determining coding frame definition information; wherein the definition content comprises: dividing the coding frame to obtain the sequence information of a first data packet and the sequence information of a last data packet;
acquiring sending time corresponding to the sent data packet according to the sent data packet information, and recombining sending frame data;
the implementation principle of the invention is as follows: according to the principle that a sending end can send data packets according to a coding sequence, data recombination is carried out on basic information of each frame of a network layer, and data records of the sent frames are compared according to the received data packet information, so that received frame information is recombined;
the beneficial effects of the above technical scheme are: the invention recombines the data of the sending frame, which is beneficial to updating the information of quantization parameter, sending time, data packet and the like of each coding frame in real time, is beneficial to more pertinence of congestion control and improves the efficiency of data transmission.
Example 3:
in one embodiment of the invention: the sending time corresponding to the sent data packet is obtained according to the sent data packet information, and the sending frame data is recombined, wherein the executing process comprises the following steps:
acquiring transmitted data packet information corresponding to a transmitting end, traversing video coding frames according to the transmitted data packet information, and acquiring video coding frames matched with data packets;
acquiring the current timestamp of the sent data packet based on the matching information of the sent data packet and the video coding frame, judging whether the current timestamp is consistent with the recorded RTP timestamp, and determining a judgment result;
when the judgment result shows that the current timestamp is inconsistent with the recorded RTP timestamp, reconstructing a coding frame structure;
when the judgment result shows that the current timestamp is consistent with the recorded RTP timestamp, setting the serial number of the data packet as the serial number of the last data packet of the corresponding transmission frame;
the implementation principle of the invention is as follows: the invention recombines and sends frame information according to sending time when a sending end sends a packet by defining a coding frame structure, wherein the coding frame structure comprises an RTP time stamp of a frame, a first packet serial number sent after the frame is split into data packets and a serial number of a last packet;
the beneficial effects of the above technical scheme are: the invention is beneficial to recording the updated data content in real time by recombining the sending frame information according to the sending time, wherein the information comprises the quantization parameter, the sending time and the composition information of a data packet of each coding frame, the congestion problem in the data transmission process is solved, the network rate is controlled, and the real-time transmission of data is facilitated.
Example 4:
in one embodiment of the invention: when the judgment result shows that the current timestamp is inconsistent with the recorded RTP timestamp, reconstructing a coding frame structure, including:
acquiring serial number information corresponding to a first data packet of a sending frame according to the serial number information corresponding to the sending frame;
acquiring a timestamp of a current data packet, and setting the timestamp of the current data packet as an RTP timestamp corresponding to the current data packet;
the implementation principle of the invention is as follows: the invention recombines the sending frame aiming at the sending time, wherein the recombination process comprises the following steps: firstly, judging whether a data packet belongs to a video coding frame, when the data packet belongs to the video coding frame, comparing whether a current time stamp is consistent with an RTP time stamp of the data packet, if so, constructing a new coding frame structure, recording a first packet sending serial number aiming at the frame, taking the current packet sending serial number as the current RTP time stamp, and if not, setting the serial number of the packet as the serial number of the last sending packet of the frame;
the beneficial effects of the above technical scheme are: the invention is beneficial to recording the updated data content in real time by recombining the sending frame information according to the sending time, wherein the information comprises the quantization parameter, the sending time and the composition information of a data packet of each coding frame, the congestion problem in the data transmission process is solved, the network rate is controlled, and the real-time transmission of data is facilitated.
Example 5:
in one embodiment of the invention: the receiving of the data packet sent by the sending end and the reconstruction of the receiving frame information based on the sending frame record, the obtaining of the receiving frame reconstruction information, and the determination of the receiving frame rate and the quantization parameter of the corresponding frame according to the receiving frame reconstruction information, include:
acquiring data packet confirmation information received by a receiving end, traversing video coding frames according to the data packet confirmation information, and acquiring the video coding frames matched with the data packets;
searching a first serial number, a last serial number and a corresponding quantization parameter of a data packet of a corresponding frame according to the sending frame record and calculating the number of the data packets of the corresponding frame according to the sending frame record;
traversing the receiving record of the receiving end, judging whether the receiving record contains the receiving information of the frame,
if so,
the current frame reception time is updated to the reception time of the corresponding data packet,
updating the receiving quantity of the data packets corresponding to the current frame;
judging whether the current frame is completely received, if so, marking the current frame and determining to finish receiving;
if the sending frame in the sending frame record is not received, reconstructing the received frame data, and updating the receiving time and the receiving quantity corresponding to the received frame;
obtaining the receiving time T of the last frame, and counting the 500ms interval [ T-500, T]The number of complete frames received within frame _ num determines the frame rate fps receivedrecv＝2×frame_num；
Counting the quantization parameters of the received frame in a 500ms interval [ T-500, T ] according to a formula smooth _ qp of 0.5 × smooth _ qp +0.5 × qp, wherein qp is the quantization parameter of the current frame in the traversal process, and smooth _ qp is the quantization parameter after smoothing;
the implementation principle of the invention is as follows: the invention recombines the received frame information according to the receiving time by the acknowledgement information received by the data packet sent by the receiving end and received by the sending end, the recombination step includes: firstly, judging whether a received packet belongs to a certain video coding frame, searching a first serial number, a last serial number and a quantization parameter of a sending data packet of a corresponding frame in a sending frame record, calculating the number of data packets contained in the frame and recording the quantization parameter of the corresponding frame, when the receiving record of the frame is recorded, updating the receiving time of a current frame into the receiving time of the packet, if the receiving time of the frame is complete, marking the receiving time as complete receiving, otherwise, constructing a new receiving frame structure, updating the receiving time and the number of the frame, and if the receiving time of the frame is complete, marking the receiving time as complete state;
the beneficial effects of the above technical scheme are: the invention is beneficial to recording the relevant information of the receiving frame of the receiving end in real time at the sending end by recombining the receiving frame information according to the receiving time and the confirmation information received by the data packet sent by the receiving end and received by the sending end, and is beneficial to improving the details and the fineness of the image and improving the effectiveness and the quality of the data by conveniently acquiring the quantization parameter of the current frame.
Example 6:
in one embodiment of the invention: the method comprises the following steps of confirming data packet information received by a receiving end, acquiring the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet, wherein the execution steps comprise:
obtaining the confirmation information of the data Packet received by the receiving end, and judging B for each confirmed data Packets,BrIf the value is null, when Bs,BrIf the value is empty, set Bs＝Ps,Br＝PrWherein B iss,BrRespectively representing the basic transmission and reception times, P, of a Packets,PrRespectively representing the sending time and the receiving time of a Packet;
when B is presents,BrIf the value is not null, for each Packet received, according to the formula qdelay ═ Pr-Br)-(Ps-Bs) Calculating queuing delay, and adding the calculated queuing delay qdelay into a container qdelay, wherein qdelay represents the result of the calculation of the queuing delay of each Packet;
calculating a smooth delay smoothjdelay of 0.875 × smoothjdelay +0.125 × delay for the queuing delay in the last 100ms in the container qdelay, wherein delay is each delay in the traversal process;
for each Packet confirmed this time, if the calculation result qdelay is less than 0, updating Bs＝Ps,Br＝Pr；
The implementation principle of the invention is as follows: in the invention, the received data is received and confirmed, the sending time and the receiving time corresponding to the data packet are obtained, and the one-way queuing delay time is calculated;
the beneficial effects of the above technical scheme are: the invention receives and confirms the received data, which is beneficial to improving the transmission efficiency of the data, reducing the delay and improving the throughput in the data transmission process.
Example 7:
in one embodiment of the invention: the method comprises the following steps of obtaining a QoE index based on the one-way queuing delay information, and updating a network transmission rate in real time according to the QoE index, wherein the execution steps comprise:
step 1: presetting parameters, namely a highest-quality target quantization parameter high _ target _ qp, a target receiving frame rate high _ target _ fps, a lowest-quality target quantization parameter low _ target _ qp and a target receiving frame rate low _ target _ fps;
step 2: judging whether smooth _ delay is greater than 100 and is in a slow starting state currently, and if smooth _ delay is greater than 100 and is in the slow starting state currently, exiting the slow starting state;
and step 3: if the current state is a slow start state, acquiring the last rate updating time, and if the last rate updating time exceeds 100ms, setting the affected _ bitrate to be 1.5 multiplied affected _ bitrate, wherein the affected _ bitrate represents the rate to be transmitted at this time;
and 4, step 4: if the current state is not in the slow start state and smooth _ delay is less than 100, if smooth _ qp is less than high _ target _ qp and fpsrecvWhen the data transmission rate is higher than high _ target _ fps, the transmission rate is kept unchanged; when smooth _ qp < low _ target _ qp and fpsrecv> low _ target _ fps, if distanceSetting the updated _ bitrate to be 1.05 multiplied by the updated _ bitrate when the last updating time exceeds 100 ms; in other cases, every time the last update time exceeds 100ms, set the affected _ bitrate equal to 1.08 × affected _ bitrate;
and 5: if the current state is not in the slow start state and smooth _ delay is larger than 250, setting an acknowledged _ bitrate-ack _ bitrate, wherein the ack _ bitrate takes the minimum value of the sending rate and the receiving rate in the latest 500 ms;
step 6: if the current state is not in the slow start state and the smooth _ delay is more than or equal to 100 and less than or equal to 250, if smooth _ qp is less than low _ target _ qp and fpsrecvIf the transmission speed is higher than low _ target _ fps, keeping the transmission speed unchanged; otherwise, setting the transmission rate to be 1.05 multiplied by the updated _ bitrate when the last updating time exceeds 100ms each distance;
the implementation principle of the invention is as follows: the method is based on QoE indexes, updates are carried out on the current network transmission rate, and the network transmission rate is updated by presetting the highest-quality target quantization parameter, the target receiving frame rate, the lowest-quality target quantization parameter and the target receiving frame rate;
the beneficial effects of the above technical scheme are: according to the invention, based on the acquired information such as the received frame rate, the quantization parameter of video frame coding and the like, the real-time queuing delay is combined, the decision efficiency is improved, so that the congestion control is more targeted, and the experience requirement of a user can be met more effectively.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (7)
1. A QoE-aware WebRTC congestion control method is characterized by comprising the following steps:
acquiring basic information of a sending frame in a network layer, recombining the basic information of the sending frame, acquiring sending frame recombination information and a sent data packet, recording the sending frame recombination information, and determining a sending frame record;
receiving a data packet sent by a sending end, recombining the information of a receiving frame based on the sending frame record, acquiring the recombined information of the receiving frame, and determining a receiving frame rate and a quantization parameter of a corresponding frame according to the recombined information of the receiving frame;
confirming data packet information received by a receiving end, acquiring the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet;
and updating the network transmission rate in real time based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and the queuing delay data corresponding to the received data packet.
2. The method for controlling QoE-aware WebRTC congestion as claimed in claim 1, wherein the acquiring basic information of a transmission frame in a network layer, performing reassembly with respect to the basic information of the transmission frame, acquiring transmission frame reassembly information and a transmitted data packet, recording the transmission frame reassembly information, and determining a transmission frame record comprises:
splitting the sending frame according to the basic information of the sending frame in the network layer to obtain the information of the sent data packet, and recording the information of the sent data packet according to the content record item to determine the information record content; wherein the content record item comprises: the data Packet corresponds to an RTP timestamp, a Packet type and a coding quantization parameter corresponding to the data Packet;
defining the coding frame structure according to the information recording content and preset definition content, and determining coding frame definition information; wherein the definition content comprises: dividing the coding frame to obtain the sequence information of a first data packet and the sequence information of a last data packet;
and acquiring the sending time corresponding to the sent data packet according to the sent data packet information, and recombining the sending frame data.
3. The QoE-aware WebRTC congestion control method according to claim 2, wherein the obtaining, according to the information of the sent data packet, the sending time corresponding to the sent data packet, and performing reassembly on the sent frame data, the performing process includes:
acquiring transmitted data packet information corresponding to a transmitting end, traversing video coding frames according to the transmitted data packet information, and acquiring video coding frames matched with data packets;
acquiring the current timestamp of the sent data packet based on the matching information of the sent data packet and the video coding frame, judging whether the current timestamp is consistent with the recorded RTP timestamp, and determining a judgment result;
when the judgment result shows that the current timestamp is inconsistent with the recorded RTP timestamp, reconstructing a coding frame structure;
and when the judgment result shows that the current timestamp is consistent with the recorded RTP timestamp, setting the serial number of the data packet as the serial number of the last data packet of the corresponding sending frame.
4. The method of claim 3, wherein the reconstructing the encoding frame structure when the determining result indicates that the current timestamp is inconsistent with the recorded RTP timestamp comprises:
acquiring serial number information corresponding to a first data packet of a sending frame according to the serial number information corresponding to the sending frame;
and acquiring the time stamp of the current data packet, and setting the time stamp of the current data packet as the RTP time stamp corresponding to the current data packet.
5. The QoE-aware WebRTC congestion control method according to claim 1, wherein the receiving a data packet sent by a sending end, reconstructing information of a received frame based on the sent frame record, obtaining received frame reconstruction information, and determining a received frame rate and a quantization parameter of a corresponding frame according to the received frame reconstruction information, comprises:
acquiring data packet confirmation information received by a receiving end, traversing video coding frames according to the data packet confirmation information, and acquiring the video coding frames matched with the data packets;
searching a first serial number, a last serial number and a corresponding quantization parameter of a data packet of a corresponding frame according to the sending frame record and calculating the number of the data packets of the corresponding frame according to the sending frame record;
traversing the receiving record of the receiving end, judging whether the receiving record contains the receiving information of the frame,
if so,
the current frame reception time is updated to the reception time of the corresponding data packet,
updating the receiving quantity of the data packets corresponding to the current frame;
judging whether the current frame is completely received, if so, marking the current frame and determining to finish receiving;
if the sending frame in the sending frame record is not received, reconstructing the received frame data, and updating the receiving time and the receiving quantity corresponding to the received frame;
obtaining the receiving time T of the last frame, and counting the 500ms interval [ T-500, T]The number of complete frames received within frame _ num determines the frame rate fps receivedrecv＝2×frame_num；
And counting the quantization parameters of the received frame in a 500ms interval [ T-500, T ] according to a formula smooth _ qp of 0.5 × smooth _ qp +0.5 × qp, wherein qp is the quantization parameter of the current frame in the traversal process, and smooth _ qp is the quantization parameter after smoothing.
6. The QoE-aware WebRTC congestion control method according to claim 1, wherein the determining of the packet information received by the receiving end, obtaining the sending time and the receiving time corresponding to each packet, calculating the queuing delay according to the sending time and the receiving time corresponding to each packet, and obtaining the queuing delay data corresponding to each packet, the executing step includes:
obtaining the confirmation information of the data Packet received by the receiving end, and judging B for each confirmed data Packets,BrIf the value is null, when Bs,BrIf the value is empty, set Bs＝Ps,Br＝PrWherein B iss,BrRespectively representing the basic transmission and reception times, P, of a Packets,PrRespectively representing the sending time and the receiving time of a Packet;
when B is presents,BrIf the value is not null, for each Packet received, according to the formula qdelay ═ Pr-Br)-(Ps-Bs) Calculating queuing delay, and adding the calculated queuing delay qdelay into a container qdelay, wherein qdelay represents the result of the calculation of the queuing delay of each Packet;
calculating a smooth delay smoothjdelay of 0.875 × smoothjdelay +0.125 × delay for the queuing delay in the last 100ms in the container qdelay, wherein delay is each delay in the traversal process;
for each Packet confirmed this time, if the calculation result qdelay is less than 0, updating Bs＝Ps,Br＝Pr。
7. The method of claim 6, wherein the step of obtaining the QoE indicator based on the one-way queuing delay information and updating the QoE indicator in real time for the network transmission rate according to the QoE indicator comprises:
step 1: presetting parameters, namely a highest-quality target quantization parameter high _ target _ qp, a target receiving frame rate high _ target _ fps, a lowest-quality target quantization parameter low _ target _ qp and a target receiving frame rate low _ target _ fps;
step 2: judging whether smooth _ delay is greater than 100 and is in a slow starting state currently, and if smooth _ delay is greater than 100 and is in the slow starting state currently, exiting the slow starting state;
and step 3: if the current state is a slow start state, acquiring the last rate updating time, and if the last rate updating time exceeds 100ms, setting the affected _ bitrate to be 1.5 multiplied affected _ bitrate, wherein the affected _ bitrate represents the rate to be transmitted at this time;
and 4, step 4: if the current state is not in the slow start state and smooth _ delay is less than 100, if smooth _ qp is less than high _ target _ qp and fpsrecvWhen the data transmission rate is higher than high _ target _ fps, the transmission rate is kept unchanged; when smooth _ qp < low _ target _ qp and fpsrecvIf the time exceeds 100ms from the last update time, setting the affected _ bitrate to be 1.05 × affected _ bitrate; in other cases, every time the last update time exceeds 100ms, set the affected _ bitrate equal to 1.08 × affected _ bitrate;
and 5: if the current state is not in the slow start state and smooth _ delay is larger than 250, setting an acknowledged _ bitrate-ack _ bitrate, wherein the ack _ bitrate takes the minimum value of the sending rate and the receiving rate in the latest 500 ms;
step 6: if the current state is not in the slow start state and the smooth _ delay is more than or equal to 100 and less than or equal to 250, if smooth _ qp is less than low _ target _ qp and fpsrecvIf the transmission speed is higher than low _ target _ fps, keeping the transmission speed unchanged; otherwise, the transmission rate is set to 1.05 × affected _ bitrate every time when the last update time exceeds 100 ms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210016796.4A CN114244778B (en) | 2022-01-07 | 2022-01-07 | QoE-aware WebRTC congestion control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210016796.4A CN114244778B (en) | 2022-01-07 | 2022-01-07 | QoE-aware WebRTC congestion control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114244778A true CN114244778A (en) | 2022-03-25 |
| CN114244778B CN114244778B (en) | 2023-11-21 |
Family
ID=80746007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210016796.4A Active CN114244778B (en) | 2022-01-07 | 2022-01-07 | QoE-aware WebRTC congestion control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114244778B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396370A (en) * | 2022-07-04 | 2022-11-25 | 北京百度网讯科技有限公司 | Exit judgment mode acquisition and slow start exit method, device and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102711172A (en) * | 2012-05-25 | 2012-10-03 | 浙江工业大学 | Modified TCPW congestion control method in wireless network |
| CN103457790A (en) * | 2013-08-16 | 2013-12-18 | 北京华为数字技术有限公司 | Marking method for timestamp and detection method, device and system of timestamp |
| US20160337241A1 (en) * | 2015-05-11 | 2016-11-17 | Oracle International Corporation | Congestion control for tunneled real-time communications |
| CN109743600A (en) * | 2019-01-15 | 2019-05-10 | 国网河南省电力公司 | Based on wearable live O&M adaptive video stream transmission rate control |
| US20190200368A1 (en) * | 2017-12-22 | 2019-06-27 | Aviwest | Transmission rate regulation method |
| CN111263102A (en) * | 2020-05-07 | 2020-06-09 | 翱捷科技(上海)有限公司 | ViLTE video call congestion control method and system based on delay gradient accumulation |
| CN113347114A (en) * | 2021-06-03 | 2021-09-03 | 清华大学 | Real-time streaming media transmission control method and device facing deadline sensing |
-
2022
- 2022-01-07 CN CN202210016796.4A patent/CN114244778B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102711172A (en) * | 2012-05-25 | 2012-10-03 | 浙江工业大学 | Modified TCPW congestion control method in wireless network |
| CN103457790A (en) * | 2013-08-16 | 2013-12-18 | 北京华为数字技术有限公司 | Marking method for timestamp and detection method, device and system of timestamp |
| US20160337241A1 (en) * | 2015-05-11 | 2016-11-17 | Oracle International Corporation | Congestion control for tunneled real-time communications |
| US20190200368A1 (en) * | 2017-12-22 | 2019-06-27 | Aviwest | Transmission rate regulation method |
| CN109743600A (en) * | 2019-01-15 | 2019-05-10 | 国网河南省电力公司 | Based on wearable live O&M adaptive video stream transmission rate control |
| CN111263102A (en) * | 2020-05-07 | 2020-06-09 | 翱捷科技(上海)有限公司 | ViLTE video call congestion control method and system based on delay gradient accumulation |
| CN113347114A (en) * | 2021-06-03 | 2021-09-03 | 清华大学 | Real-time streaming media transmission control method and device facing deadline sensing |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396370A (en) * | 2022-07-04 | 2022-11-25 | 北京百度网讯科技有限公司 | Exit judgment mode acquisition and slow start exit method, device and storage medium |
| CN115396370B (en) * | 2022-07-04 | 2023-08-18 | 北京百度网讯科技有限公司 | Exit discrimination mode acquisition and slow start exit method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114244778B (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Qian et al. | Flare: Practical viewport-adaptive 360-degree video streaming for mobile devices | |
| US8924476B1 (en) | Recovery and fault-tolerance of a real time in-memory index | |
| JP5780684B2 (en) | Content reproduction information estimation apparatus, method, and program | |
| CN101188477B (en) | A data packet sequence receiving method and device | |
| JP2016516323A (en) | Method, device, and system for obtaining streaming media data | |
| CN108155999B (en) | Intelligent UDP multicast file distribution system and method with TCP (transmission control protocol) packet supplementing mechanism | |
| CN112954300B (en) | Performance testing device and method for video monitoring platform | |
| CN108235149B (en) | Method and device for optimizing video playing fluency | |
| CN113891175B (en) | Live broadcast push flow method, device and system | |
| CN113014968B (en) | Multi-user dynamic code rate video transmission method and system based on reinforcement learning | |
| García et al. | Quality-control algorithm for adaptive streaming services over wireless channels | |
| CN112968959B (en) | Resource request method and terminal | |
| CN114244778A (en) | QoE-aware WebRTC congestion control method | |
| CN113783944B (en) | Video data processing method, device, system and equipment based on cloud edge cooperation | |
| CN112019905A (en) | Live broadcast playback method, computer equipment and readable storage medium | |
| CN110662017A (en) | Video playing quality detection method and device | |
| CN103607395A (en) | Touch sense data transmission method and apparatus | |
| CN114401253B (en) | Video transmission method in analog network based on WebRTC | |
| Sun et al. | Optimal strategies for live video streaming in the low-latency regime | |
| CN106453270A (en) | Adaptive transmission algorithm based on PI control streaming media | |
| CN104753811B (en) | A kind of streaming media service optimization method, equipment and system | |
| CN110139128A (en) | A kind of information processing method, blocker, electronic equipment and storage medium | |
| CN108063746A (en) | Processing method, client, server and the system of data | |
| CN114040257A (en) | Self-adaptive video stream transmission playing method, device, equipment and storage medium | |
| CN109769125A (en) | Dynamic adjusting method, media server and the transcoding server of streaming media bit rate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |