CN114244778B - QoE-aware WebRTC congestion control method - Google Patents
QoE-aware WebRTC congestion control method Download PDFInfo
- Publication number
- CN114244778B CN114244778B CN202210016796.4A CN202210016796A CN114244778B CN 114244778 B CN114244778 B CN 114244778B CN 202210016796 A CN202210016796 A CN 202210016796A CN 114244778 B CN114244778 B CN 114244778B
- Authority
- CN
- China
- Prior art keywords
- frame
- information
- data packet
- time
- receiving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000005540 biological transmission Effects 0.000 claims abstract description 101
- 238000013139 quantization Methods 0.000 claims abstract description 49
- 230000008521 reorganization Effects 0.000 claims abstract description 13
- 230000008569 process Effects 0.000 claims description 13
- 230000006798 recombination Effects 0.000 claims description 11
- 238000005215 recombination Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012790 confirmation Methods 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 6
- 230000001934 delay Effects 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 description 13
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 108091026890 Coding region Proteins 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0268—Traffic management, e.g. flow control or congestion control using specific QoS parameters for wireless networks, e.g. QoS class identifier [QCI] or guaranteed bit rate [GBR]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a QoE perceived WebRTC congestion control method, which comprises the following steps: acquiring basic information of a transmission frame in a network layer, reorganizing the basic information of the transmission frame, recording the reorganization information of the transmission frame, and determining a transmission frame record; receiving a data packet sent by a sending end, reorganizing the received frame information based on a sending frame record, and determining a received frame rate and quantization parameters of a corresponding frame according to the received frame reorganization information; confirming data packet information received by a receiving end, acquiring the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data; based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and the queuing delay data corresponding to the received data packet, the real-time updating is carried out for the network transmission rate. The invention can control the network transmission rate in real time according to the acquired quantization parameter and the received frame rate information.
Description
Technical Field
The invention relates to the technical field of Web real-time communication, in particular to a QoE-aware WebRTC congestion control method.
Background
At present, with the increasing performance of network infrastructures such as the internet, more and more funds, enterprises and research personnel are put into the industry based on RTC technology such as video conferences, cloud games and Cloud XR, and since the RTC field integrating technologies such as graphic rendering, audio and video encoding and decoding, network transmission has a higher technical threshold, a complete RTC system needs to be developed from zero to put into a great deal of manpower, financial resources and time resources, so many enterprises and individuals can rely on or directly use the open source WebRTC solution of Google.
The open source WebRTC system of Google is a complete solution meeting WebRTC standards, and the bottom layer adopts a GCC (Google Congestion Control) congestion control method of Google, which models the change of queuing delay in transmission, predicts the change of queuing delay on a network transmission path, thereby giving a corresponding rate control scheme, and feeds back to an encoder to control the output of the encoder, thereby driving the operation of the whole RTC system. A suitable congestion control scheme can be said to be a drive engine for the whole RTC system, which is capable of efficiently driving the operation of the whole system. GCC transmits and receives information based on data packets at the network layer at the decision time, and transmission delay of the data packets in the network may have a certain randomness due to noise influence, so it is difficult to accurately perceive the network state of the bottom layer. The WebRTC system is a highly modularized system, has a good hierarchical structure, and can not directly acquire the relevant information of the application layer. On the premise of not changing the whole WebRTC framework, the decision of acquiring the application layer information at the network layer and performing congestion control based on the application layer information has practical application value.
Disclosure of Invention
The invention provides a QoE perceived WebRTC congestion control method, which is used for solving the problem that a network layer cannot directly acquire relevant information of an application layer.
A QoE-aware WebRTC congestion control method, comprising:
acquiring basic information of a transmission frame in a network layer, reorganizing the basic information of the transmission frame, acquiring transmission frame reorganization information and transmitted data packets, recording the transmission frame reorganization information, and determining a transmission frame record;
receiving a data packet sent by a sending end, recombining the received frame information based on the sending frame record, acquiring the received frame recombination information, and determining a received frame rate and quantization parameters of a corresponding frame according to the received frame recombination information;
confirming data packet information received by a receiving end, acquiring sending time and receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet;
based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and the queuing delay data corresponding to the received data packet, the real-time updating is carried out for the network transmission rate.
As an embodiment of the present invention: the obtaining the basic information of the transmission frame in the network layer, the reorganizing the basic information of the transmission frame, obtaining the reorganizing information of the transmission frame and the sent data packet, and recording the reorganizing information of the transmission frame, and determining the transmission frame record includes:
splitting the transmission frame according to the basic information of the transmission frame in the network layer, obtaining the transmitted data packet information, recording the information of the transmitted data packet information according to a content record item, and determining information record content; wherein the content record item includes: the data Packet corresponds to an RTP timestamp, a Packet type and a coding quantization parameter corresponding to the data Packet;
defining a coding frame structure according to the information record content and preset definition content, and determining coding frame definition information; wherein, the definition content includes: splitting the coded frame to obtain the sequence information of the first data packet and the sequence information of the last data packet;
and acquiring the sending time corresponding to the sent data packet according to the sent data packet information, and recombining the sending frame data.
As an embodiment of the present invention: the sending time corresponding to the sent data packet is obtained according to the sent data packet information, the sending frame data is recombined, and the execution process comprises the following steps:
acquiring transmitted data packet information corresponding to a transmitting end, traversing video coding frames according to the transmitted data packet information, and acquiring video coding frames matched with data packets;
based on the matching information of the sent data packet and the video coding frame, acquiring the current time stamp of the sent data packet, judging whether the current time stamp is consistent with the recorded RTP time stamp, and determining a judging result;
when the judging result shows that the current time stamp is inconsistent with the recorded RTP time stamp, reconstructing the coding frame structure;
and when the judgment result shows that the current time stamp is consistent with the recorded RTP time stamp, setting the sequence number of the data packet as the sequence number of the last data packet of the corresponding transmission frame.
As an embodiment of the present invention: and when the judging result shows that the current time stamp is inconsistent with the recorded RTP time stamp, reconstructing the coding frame structure, wherein the method comprises the following steps:
acquiring serial number information corresponding to a first data packet of a transmission frame according to the serial number information corresponding to the transmission frame;
and acquiring the time stamp of the current data packet, and setting the time stamp of the current data packet as the RTP time stamp corresponding to the current data packet.
As an embodiment of the present invention: the method for receiving the data packet sent by the sending end, and recombining the received frame information based on the sending frame record, obtaining the received frame recombination information, and determining the received frame rate and the quantization parameter of the corresponding frame according to the received frame recombination information comprises the following steps:
acquiring data packet confirmation information received by a receiving end, traversing a video coding frame according to the data packet confirmation information, and acquiring a video coding frame matched with the data packet;
searching a first serial number, a last serial number and corresponding quantization parameters of the data packets of the corresponding frames according to the sending frame record, and calculating the number of the data packets of the corresponding frames according to the sending frame record;
traversing the receiving record of the receiving end, judging whether the receiving record contains the receiving information of the frame,
if so, the first and second data are not identical,
updating the current frame reception time to the reception time of the corresponding data packet,
updating the data packet receiving quantity corresponding to the current frame;
judging whether the current frame is received completely, if so, marking the current frame, and determining that the receiving is completed;
if the sending frame in the sending frame record is not received, reconstructing the received frame data, and updating the receiving time and the receiving quantity corresponding to the received frame;
acquiring the receiving time T of the last frame, counting 500ms intervals [ T-500, T ]]Frame_num of the number of received complete frames, determining the received frame rate fps recv ＝2×frame_num；
The quantization parameters of the received frame in the 500ms interval [ T-500, T ] are counted according to the formula smoothqp=0.5×smoothqp+0.5×qp, where qp is the quantization parameter of the current frame in the traversal process, and smoothqp is the quantization parameter after smoothing.
As an embodiment of the present invention: the method comprises the steps of confirming the data packet information received by a receiving end, obtaining the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and obtaining queuing delay data corresponding to each data packet, wherein the execution steps comprise:
acquiring acknowledgement information of a data Packet received by a receiving end, and judging B according to each Packet of acknowledgement s ,B r Whether or not the value is null, when B s ,B r When the value is empty, setting B s ＝P s ,B r ＝P r Wherein B is s ,B r Respectively representing the basic sending and receiving time of Packet, P s ,P r Respectively representing the sending time and the receiving time of the Packet;
when B is s ,B r When the value is not null, for each received Packet, the formula qdelay= (P) r -B r )-(P s -B s ) Calculating queuing delay, and adding the calculated queuing delay qdelay into a container qdelays, wherein qdelay represents the queuing delay calculation result of each Packet;
for queuing delays within the last 100ms in the container qdelays, calculating a smoothing delay smoothjdelay = 0.875 x smoothjdelay +0.125 x delay, where delay is each delay in the traversal process;
for each Packet confirmed at this time, if the calculation result qdelay is less than 0, updating B s ＝P s ,B r ＝P r 。
As an embodiment of the present invention: the step of obtaining QoE index based on the one-way queuing delay information and updating the network transmission rate in real time according to the QoE index comprises the following steps:
step 1: presetting parameters, namely a target quantization parameter high_target_qp with the highest quality, a target quantization parameter low_target_qp with the lowest quality of a target received frame rate high_target_fps, and a target received frame rate low_target_fps;
step 2: judging whether the smoothdelay is more than 100 and is in a slow start state currently, and if the smoothdelay is more than 100 and is in the slow start state currently, exiting the slow start state;
step 3: if the current state is a slow start state, acquiring the update time of the last rate, and if the update time of the last rate exceeds 100ms, setting an estimated_bit rate=1.5×estimated_bit rate, wherein the estimated_bit rate represents the rate to be transmitted at this time;
step 4: if the current state is not in the slow start state and the smooth_delay is less than 100, if the smooth_qp is less than high_target_qp and fps recv At > high_target_fps, the transmission rate is kept unchanged; when smoth_qp < low_target_qp and fps recv If the update time exceeds 100ms from the last time, > low_target_fps, set estimed_bit=1.05×estimed_bit; in other cases, the updated_bit rate=1.08×updated_bit rate is set every time the last update time exceeds 100 ms;
step 5: if the current state is not in a slow start state and the smooth_delay is more than 250, setting an estimated_bit rate=ack_bit rate, wherein the ack_bit rate takes the minimum value of the sending rate and the receiving rate in the last 500 ms;
step 6: if the current state is not in the slow start state and 100 is less than or equal to the smooth_delay is less than or equal to 250, if the smooth_qp is less than the low_target_qp and fps recv > low_target_fps, then the transmission rate is kept unchanged; otherwise, the transmission rate is set to be estimed_bit rate=1.05×estimed_bit rate every time the last update time exceeds 100 ms.
The beneficial effects of the invention are as follows: the invention reorganizes the information such as the quantization parameter of video coding, the receiving frame rate of the receiving party and the like in the network layer, and uses the information as the important basis of congestion control decision; congestion control methods meeting various user experience criteria can be designed based on these application layer metrics; based on the obtained information such as the received frame rate and the quantization parameter of video frame coding, a QoE perceived congestion control algorithm is designed by combining real-time queuing delay, and decisions can be made in real time based on the information such as the received frame rate and the quantization parameter of the current receiving end and the queuing delay in the current network, so that congestion control is more targeted, and user experience requirements can be better met.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.
In the drawings:
fig. 1 is a structural flow chart of a QoE-aware WebRTC congestion control method according to an embodiment of the present invention;
fig. 2 is a flowchart of a sending frame reorganization in a QoE-aware WebRTC congestion control method according to an embodiment of the present invention;
fig. 3 is a flowchart of a received frame reorganization in a QoE-aware WebRTC congestion control method according to an embodiment of the present invention.
Fig. 4 is a congestion control flow chart in a QoE-aware WebRTC congestion control method according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
It should be noted that in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Example 1:
the embodiment of the invention provides a QoE perceived WebRTC congestion control method, which comprises the following steps:
acquiring basic information of a transmission frame in a network layer, reorganizing the basic information of the transmission frame, acquiring transmission frame reorganization information and transmitted data packets, recording the transmission frame reorganization information, and determining a transmission frame record;
receiving a data packet sent by a sending end, recombining the received frame information based on the sending frame record, acquiring the received frame recombination information, and determining a received frame rate and quantization parameters of a corresponding frame according to the received frame recombination information;
confirming data packet information received by a receiving end, acquiring sending time and receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet;
based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and queuing delay data corresponding to the received data packet, updating the network transmission rate in real time;
the implementation principle of the invention is as follows: in the prior art, information is sent and received by the data packet at the network layer in the data transmission process, but the transmission delay of the data packet at the network layer has certain randomness due to the influence of noise, so that the network state of the bottom layer is difficult to accurately sense; the invention uses the quantized parameter of video coding of the sending end and the receiving frame rate of the receiving end as the reference information of congestion control;
the beneficial effects of the technical scheme are as follows: according to the invention, under the condition that the Google open source WebRTC frame structure is not changed and only the source code is slightly changed, the quantization parameter of the video coding of the transmitting end, the receiving frame rate and other information of the receiving end are reproduced in the network layer, the reference information of the congestion control decision is enriched, the reference information can be used as an important reference of the congestion control decision, and secondly, based on the acquired QoE information of the quantization parameter, the receiving frame rate and the like of the coded video, the network transmission rate can be effectively controlled in real time according to the user experience of the receiving end, congestion in data transmission can be prevented, and the congestion control can be more targeted based on the receiving frame rate and the quantization parameter of the current receiving end, queuing delay and other information in the current network, so that the user experience requirement can be better met.
Example 2:
in one embodiment of the invention: the obtaining the basic information of the transmission frame in the network layer, the reorganizing the basic information of the transmission frame, obtaining the reorganizing information of the transmission frame and the sent data packet, and recording the reorganizing information of the transmission frame, and determining the transmission frame record includes:
splitting the transmission frame according to the basic information of the transmission frame in the network layer, obtaining the transmitted data packet information, recording the information of the transmitted data packet information according to a content record item, and determining information record content; wherein the content record item includes: the data Packet corresponds to an RTP timestamp, a Packet type and a coding quantization parameter corresponding to the data Packet;
defining a coding frame structure according to the information record content and preset definition content, and determining coding frame definition information; wherein, the definition content includes: splitting the coded frame to obtain the sequence information of the first data packet and the sequence information of the last data packet;
acquiring the transmission time corresponding to the transmitted data packet according to the transmitted data packet information, and recombining the transmitted frame data;
the implementation principle of the invention is as follows: according to the principle that a transmitting end transmits data packets according to a coding sequence, data recombination is carried out on basic information of each frame of a network layer, and data records of transmitting frames are compared according to received data packet information, so that received frame information is recombined;
the beneficial effects of the technical scheme are as follows: the invention reorganizes the data of the transmission frames, is favorable for updating the information of the quantization parameter, the transmission time, the data packet and the like of each coding frame in real time, is favorable for controlling congestion more pertinently, and improves the efficiency of data transmission.
Example 3:
in one embodiment of the invention: the sending time corresponding to the sent data packet is obtained according to the sent data packet information, the sending frame data is recombined, and the execution process comprises the following steps:
acquiring transmitted data packet information corresponding to a transmitting end, traversing video coding frames according to the transmitted data packet information, and acquiring video coding frames matched with data packets;
based on the matching information of the sent data packet and the video coding frame, acquiring the current time stamp of the sent data packet, judging whether the current time stamp is consistent with the recorded RTP time stamp, and determining a judging result;
when the judging result shows that the current time stamp is inconsistent with the recorded RTP time stamp, reconstructing the coding frame structure;
when the judging result shows that the current time stamp is consistent with the recorded RTP time stamp, setting the sequence number of the data packet as the sequence number of the last data packet of the corresponding transmission frame;
the implementation principle of the invention is as follows: the invention uses the definition of the coding frame structure, wherein the coding frame structure comprises RTP time stamp of the frame, the sequence number of the first packet and the sequence number of the last packet sent after the frame is split into data packets, and when the sending end sends a packet, the frame information is recombined and sent according to the sending time;
the beneficial effects of the technical scheme are as follows: the invention is beneficial to real-time recording of updated data content by reorganizing the transmission frame information according to the transmission time, wherein the data content comprises quantization parameters of each coding frame, the transmission time and the composition information of the data packet, and the congestion problem in the data transmission process is solved, the network rate is controlled, and the real-time transmission of the data is facilitated.
Example 4:
in one embodiment of the invention: and when the judging result shows that the current time stamp is inconsistent with the recorded RTP time stamp, reconstructing the coding frame structure, wherein the method comprises the following steps:
acquiring serial number information corresponding to a first data packet of a transmission frame according to the serial number information corresponding to the transmission frame;
acquiring a time stamp of a current data packet, and setting the time stamp of the current data packet as an RTP time stamp corresponding to the current data packet;
the implementation principle of the invention is as follows: the invention reorganizes the transmission frames according to the transmission time, wherein the reorganization process comprises the following steps: firstly judging whether a data packet belongs to a video coding frame, when the data packet belongs to the video coding frame, comparing whether the current time stamp is consistent with the RTP time stamp of the data packet, when the current time stamp is consistent with the RTP time stamp of the data packet, constructing a new coding frame structure, recording a first packet sending sequence number aiming at the frame, taking the current packet time stamp as the current RTP time stamp, and if the current packet time stamp is inconsistent with the RTP time stamp, setting the sequence number of the packet as the sequence number of the last sending packet of the frame;
the beneficial effects of the technical scheme are as follows: the invention is beneficial to real-time recording of updated data content by reorganizing the transmission frame information according to the transmission time, wherein the data content comprises quantization parameters of each coding frame, the transmission time and the composition information of the data packet, and the congestion problem in the data transmission process is solved, the network rate is controlled, and the real-time transmission of the data is facilitated.
Example 5:
in one embodiment of the invention: the method for receiving the data packet sent by the sending end, and recombining the received frame information based on the sending frame record, obtaining the received frame recombination information, and determining the received frame rate and the quantization parameter of the corresponding frame according to the received frame recombination information comprises the following steps:
acquiring data packet confirmation information received by a receiving end, traversing a video coding frame according to the data packet confirmation information, and acquiring a video coding frame matched with the data packet;
searching a first serial number, a last serial number and corresponding quantization parameters of the data packets of the corresponding frames according to the sending frame record, and calculating the number of the data packets of the corresponding frames according to the sending frame record;
traversing the receiving record of the receiving end, judging whether the receiving record contains the receiving information of the frame,
if so, the first and second data are not identical,
updating the current frame reception time to the reception time of the corresponding data packet,
updating the data packet receiving quantity corresponding to the current frame;
judging whether the current frame is received completely, if so, marking the current frame, and determining that the receiving is completed;
if the sending frame in the sending frame record is not received, reconstructing the received frame data, and updating the receiving time and the receiving quantity corresponding to the received frame;
acquiring the receiving time T of the last frame, counting 500ms intervals [ T-500, T ]]Frame_num of the number of received complete frames, determining the received frame rate fps recv ＝2×frame_num；
Counting the quantization parameters of the received frame in the 500ms interval [ T-500, T ] according to the formula of smoothqp=0.5×smoothqp+0.5×qp, wherein qp is the quantization parameter of the current frame in the traversal process, and smoothqp is the quantization parameter after smoothing;
the implementation principle of the invention is as follows: the invention reorganizes the received frame information according to the receiving time by receiving acknowledgement information received by the data packet sent by the receiving end by the sending end, and the reorganizing step comprises: firstly judging whether a received packet belongs to a certain video coding frame, searching a first serial number, a last serial number and a quantization parameter of a transmission data packet of a corresponding frame in a transmission frame record, calculating the quantity of the data packet contained in the frame and recording the quantization parameter of the corresponding frame, updating the current frame receiving time to the receiving time of the packet when the receiving record of the frame exists, if the frame is received completely, marking the receiving time as finished, otherwise, constructing a new receiving frame structure, updating the receiving time and the quantity of the frame, and if the frame is received completely, marking the receiving time as finished;
the beneficial effects of the technical scheme are as follows: the invention is beneficial to recording the relevant information of the receiving frame of the receiving end in real time at the sending end by reorganizing the receiving frame information according to the receiving time and the receiving acknowledgement information received by the data packet sent by the receiving end and is beneficial to improving the detail and the fineness of the image and the effectiveness and the quality of the data by conveniently acquiring the quantization parameter of the current frame.
Example 6:
in one embodiment of the invention: the method comprises the steps of confirming the data packet information received by a receiving end, obtaining the sending time and the receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and obtaining queuing delay data corresponding to each data packet, wherein the execution steps comprise:
acquiring acknowledgement information of a data Packet received by a receiving end, and judging B according to each Packet of acknowledgement s ,B r Whether or not the value is null, when B s ,B r When the value is empty, setting B s ＝P s ,B r ＝P r Wherein B is s ,B r Respectively representing the basic sending and receiving time of Packet, P s ,P r Respectively representing the sending time and the receiving time of the Packet;
when B is s ,B r When the value is not null, for each received Packet, the formula qdelay= (P) r -B r )-(P s -B s ) Calculating queuing delay, and adding the calculated queuing delay qdelay into a container qdelays, wherein qdelay represents the queuing delay calculation result of each Packet;
for queuing delays within the last 100ms in the container qdelays, calculating a smoothing delay smoothjdelay = 0.875 x smoothjdelay +0.125 x delay, where delay is each delay in the traversal process;
for each Packet confirmed at this time, if the calculation result qdelay is less than 0, updating B s ＝P s ,B r ＝P r ；
The implementation principle of the invention is as follows: the method comprises the steps of receiving and confirming received data, obtaining the sending time and the receiving time corresponding to a data packet, and calculating one-way queuing delay time;
the beneficial effects of the technical scheme are as follows: the invention receives and confirms the received data, is beneficial to improving the transmission efficiency of the data, reduces delay and improves throughput in the data transmission process.
Example 7:
in one embodiment of the invention: the step of obtaining QoE index based on the one-way queuing delay information and updating the network transmission rate in real time according to the QoE index comprises the following steps:
step 1: presetting parameters, namely a target quantization parameter high_target_qp with the highest quality, a target quantization parameter low_target_qp with the lowest quality of a target received frame rate high_target_fps, and a target received frame rate low_target_fps;
step 2: judging whether the smoothdelay is more than 100 and is in a slow start state currently, and if the smoothdelay is more than 100 and is in the slow start state currently, exiting the slow start state;
step 3: if the current state is a slow start state, acquiring the update time of the last rate, and if the update time of the last rate exceeds 100ms, setting an estimated_bit rate=1.5×estimated_bit rate, wherein the estimated_bit rate represents the rate to be transmitted at this time;
step 4: if the current state is not in the slow start state and the smooth_delay is less than 100, if the smooth_qp is less than high_target_qp and fps recv At > high_target_fps, the transmission rate is kept unchanged; when smoth_qp < low_target_qp and fps recv If the update time exceeds 100ms from the last time, > low_target_fps, set estimed_bit=1.05×estimed_bit; in other cases, the updated_bit rate=1.08×updated_bit rate is set every time the last update time exceeds 100 ms;
step 5: if the current state is not in a slow start state and the smooth_delay is more than 250, setting an estimated_bit rate=ack_bit rate, wherein the ack_bit rate takes the minimum value of the sending rate and the receiving rate in the last 500 ms;
step 6: if the current state is not in the slow start state and 100 is less than or equal to the smooth_delay is less than or equal to 250, if the smooth_qp is less than the low_target_qp and fps recv > low_target_fps, then the transmission rate is kept unchanged; otherwise, setting the transmission rate to be escripted_bit rate=1.05×escripted_bit rate when the last update time exceeds 100 ms;
the implementation principle of the invention is as follows: the invention updates the network transmission rate by presetting a target quantization parameter with highest quality, a target receiving frame rate, a target quantization parameter with lowest quality and a target receiving frame rate based on QoE index aiming at the current network transmission rate;
the beneficial effects of the technical scheme are as follows: based on the obtained information such as the received frame rate and the quantization parameter of video frame coding, the invention combines real-time queuing delay, improves decision efficiency, ensures that congestion control is more targeted, and can meet the experience requirements of users.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (5)
1. A QoE-aware WebRTC congestion control method, comprising:
acquiring basic information of a transmission frame in a network layer, reorganizing the basic information of the transmission frame, acquiring transmission frame reorganization information and transmitted data packets, recording the transmission frame reorganization information, and determining a transmission frame record;
receiving a data packet sent by a sending end, recombining the received frame information based on the sending frame record, acquiring the received frame recombination information, and determining a received frame rate and quantization parameters of a corresponding frame according to the received frame recombination information;
confirming data packet information received by a receiving end, acquiring sending time and receiving time corresponding to each data packet, calculating queuing delay according to the sending time and the receiving time corresponding to each data packet, and acquiring queuing delay data corresponding to each data packet;
based on the receiving frame rate of the receiving end, the quantization parameter of the corresponding frame and queuing delay data corresponding to the received data packet, updating the network transmission rate in real time;
the acquiring the basic information of the transmission frame in the network layer, reorganizing the basic information of the transmission frame, acquiring the reorganization information of the transmission frame and the sent data packet, recording the reorganization information of the transmission frame, and determining the transmission frame record includes:
splitting the transmission frame according to the basic information of the transmission frame in the network layer, obtaining the transmitted data packet information, recording the information of the transmitted data packet information according to a content record item, and determining information record content; wherein the content record item includes: the data Packet corresponds to an RTP timestamp, a Packet type and a coding quantization parameter corresponding to the data Packet;
defining a coding frame structure according to the information record content and preset definition content, and determining coding frame definition information; wherein, the definition content includes: splitting the coded frame to obtain the sequence information of the first data packet and the sequence information of the last data packet;
acquiring the transmission time corresponding to the transmitted data packet according to the transmitted data packet information, and recombining the transmitted frame data;
the sending time corresponding to the sent data packet is obtained according to the sent data packet information, the sending frame data is recombined, and the execution process comprises the following steps:
acquiring transmitted data packet information corresponding to a transmitting end, traversing video coding frames according to the transmitted data packet information, and acquiring video coding frames matched with data packets;
based on the matching information of the sent data packet and the video coding frame, acquiring the current time stamp of the sent data packet, judging whether the current time stamp is consistent with the recorded RTP time stamp, and determining a judging result;
when the judging result shows that the current time stamp is inconsistent with the recorded RTP time stamp, reconstructing the coding frame structure;
and when the judgment result shows that the current time stamp is consistent with the recorded RTP time stamp, setting the sequence number of the data packet as the sequence number of the last data packet of the corresponding transmission frame.
2. The QoE-aware WebRTC congestion control method of claim 1, wherein when the determination shows that the current timestamp does not coincide with the recorded RTP timestamp, reconstructing the encoded frame structure comprises:
acquiring serial number information corresponding to a first data packet of a transmission frame according to the serial number information corresponding to the transmission frame;
and acquiring the time stamp of the current data packet, and setting the time stamp of the current data packet as the RTP time stamp corresponding to the current data packet.
3. The QoE-aware WebRTC congestion control method of claim 1, wherein the receiving a packet sent by a sender and reassembling received frame information based on the sending frame record, obtaining received frame reassembling information, and determining a received frame rate and quantization parameters of a corresponding frame according to the received frame reassembling information, includes:
acquiring data packet confirmation information sent by a receiving end, traversing a video coding frame according to the data packet confirmation information, and acquiring a video coding frame matched with a data packet of the receiving frame;
searching a first serial number, a last serial number and corresponding quantization parameters of the data packets of the corresponding frames according to the sending frame record, and calculating the number of the data packets of the corresponding frames according to the sending frame record;
traversing the receiving record of the receiving end, judging whether the receiving record contains the receiving information of the frame,
if so, the first and second data are not identical,
updating the current frame reception time to the reception time of the corresponding data packet,
updating the data packet receiving quantity corresponding to the current frame;
judging whether the current frame is received completely, if so, marking the current frame, and determining that the receiving is completed;
if the transmission frame in the transmission record is not received, reconstructing the received frame data, and updating the receiving time and the receiving quantity corresponding to the received frame;
acquiring the receiving time T of the last frame, counting 500ms intervals [ T-500, T ]]Frame_num of the number of received complete frames, determining the received frame rate fps recv ＝2×frame_num；
The quantization parameters of the received frame in the 500ms interval [ T-500, T ] are counted according to the formula smoothqp=0.5×smoothqp+0.5×qp, where qp is the quantization parameter of the current frame in the traversal process, and smoothqp is the quantization parameter after smoothing.
4. The QoE-aware WebRTC congestion control method of claim 1, wherein the receiving-side acknowledges the received packet information, obtains a transmission time and a reception time corresponding to each packet, calculates a queuing delay according to the transmission time and the reception time corresponding to each packet, and obtains queuing delay data corresponding to each packet, and the performing step includes:
acquiring acknowledgement information of a data Packet received by a receiving end, and judging B according to each Packet of acknowledgement s ,B r When the value is empty, setting B s ＝P s ,B r ＝P s Wherein B is s ,B r Respectively representing the basic sending and receiving time of Packet, P s ,P r Respectively representing the sending time and the receiving time of the Packet;
when B is s ,B r When the value is not null, for each received Packet, the formula qdelay= (P) r -B r )-(P s -B s ) Calculating queuing delay, and adding the calculated queuing delay qdelay into a container qdelay s; wherein qdelay represents the queuing delay calculation result of each Packet;
for queuing delays within the last 100ms in the container qdelays, calculating a smoothing delay smoothjdelay = 0.875 x smoothjdelay +0.125 x delay, where delay is each delay in the traversal process;
for each Packet confirmed at this time, if the calculation result qdelay is less than 0, updating B s ＝P s ,B r ＝P r 。
5. The QoE-aware WebRTC congestion control method of claim 4, wherein the performing step includes, based on the received frame rate of the receiving end, the quantization parameter of the corresponding frame, and queuing delay data corresponding to the received data packet, updating the network transmission rate in real time:
step 1: presetting parameters, namely a target quantization parameter high_target_qp with the highest quality and a target received frame rate high_target_fps; target quantization parameter low_target_qp of the lowest quality, target received frame rate low_target_fps;
step 2: judging whether the smooth_delay is more than 100 and is in a slow start state currently, if so, exiting the slow start state;
step 3: if the current state is a slow start state, setting an estimated_bit rate=1.5×estimated_bit rate when the last rate update time exceeds 100ms, wherein the estimated_bit rate is the rate to be transmitted at this time;
step 4: if the smoth_delay is less than 100, when smoth_qp is less than high_target_qp and fps recv At > high_target_fps, the transmission rate is kept unchanged; when smoth_qp < low_target_qp and fps recv If the update time exceeds 100ms from the last time, > low_target_fps, set estimed_bit=1.05×estimed_bit; in other cases, the updated_bit rate=1.08×updated_bit rate is set every time the last update time exceeds 100 ms;
step 5: if the smoothdelay > 250, then set an estimated_bit rate=ack_bit rate, where ack_bit rate takes the minimum of the sending rate and the receiving rate in the last 500 ms;
step 6: in other cases, if smoth_qp < low_target_qp and fps recv > low_target_fps, then the transmission rate is kept unchanged; otherwise, the transmission rate is set to be estimed_bit rate=1.05×estimed_bit rate every time the last update time exceeds 100 ms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210016796.4A CN114244778B (en) | 2022-01-07 | 2022-01-07 | QoE-aware WebRTC congestion control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210016796.4A CN114244778B (en) | 2022-01-07 | 2022-01-07 | QoE-aware WebRTC congestion control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114244778A CN114244778A (en) | 2022-03-25 |
| CN114244778B true CN114244778B (en) | 2023-11-21 |
Family
ID=80746007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210016796.4A Active CN114244778B (en) | 2022-01-07 | 2022-01-07 | QoE-aware WebRTC congestion control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114244778B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396370B (en) * | 2022-07-04 | 2023-08-18 | 北京百度网讯科技有限公司 | Exit discrimination mode acquisition and slow start exit method, device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102711172A (en) * | 2012-05-25 | 2012-10-03 | 浙江工业大学 | Modified TCPW congestion control method in wireless network |
| CN103457790A (en) * | 2013-08-16 | 2013-12-18 | 北京华为数字技术有限公司 | Marking method for timestamp and detection method, device and system of timestamp |
| CN109743600A (en) * | 2019-01-15 | 2019-05-10 | 国网河南省电力公司 | Based on wearable live O&M adaptive video stream transmission rate control |
| CN111263102A (en) * | 2020-05-07 | 2020-06-09 | 翱捷科技(上海)有限公司 | ViLTE video call congestion control method and system based on delay gradient accumulation |
| CN113347114A (en) * | 2021-06-03 | 2021-09-03 | 清华大学 | Real-time streaming media transmission control method and device facing deadline sensing |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9660915B2 (en) * | 2015-05-11 | 2017-05-23 | Oracle International Corporation | Congestion control for tunneled real-time communications |
| FR3076158B1 (en) * | 2017-12-22 | 2020-01-10 | Aviwest | FLOW REGULATION METHOD |
-
2022
- 2022-01-07 CN CN202210016796.4A patent/CN114244778B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102711172A (en) * | 2012-05-25 | 2012-10-03 | 浙江工业大学 | Modified TCPW congestion control method in wireless network |
| CN103457790A (en) * | 2013-08-16 | 2013-12-18 | 北京华为数字技术有限公司 | Marking method for timestamp and detection method, device and system of timestamp |
| CN109743600A (en) * | 2019-01-15 | 2019-05-10 | 国网河南省电力公司 | Based on wearable live O&M adaptive video stream transmission rate control |
| CN111263102A (en) * | 2020-05-07 | 2020-06-09 | 翱捷科技(上海)有限公司 | ViLTE video call congestion control method and system based on delay gradient accumulation |
| CN113347114A (en) * | 2021-06-03 | 2021-09-03 | 清华大学 | Real-time streaming media transmission control method and device facing deadline sensing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114244778A (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Izquierdo et al. | A survey of statistical source models for variable-bit-rate compressed video | |
| US6947045B1 (en) | Coding of animated 3-D wireframe models for internet streaming applications: methods, systems and program products | |
| CN110430441B (en) | Cloud mobile phone video acquisition method, system, device and storage medium | |
| US8055974B2 (en) | Content distribution method, encoding method, reception/reproduction method and apparatus, and program | |
| KR20140098248A (en) | Dynamic modification of video properties | |
| Liubogoshchev et al. | Adaptive cloud-based extended reality: Modeling and optimization | |
| CN114244778B (en) | QoE-aware WebRTC congestion control method | |
| WO2012119459A1 (en) | Data transmission method, apparatus and system | |
| Tizon et al. | MPEG-4-based adaptive remote rendering for video games | |
| CN112104867B (en) | Video processing method, video processing device, intelligent equipment and storage medium | |
| US11089334B1 (en) | Methods and systems for maintaining quality of experience in real-time live video streaming | |
| CN113783944A (en) | Video data processing method, device, system and equipment based on cloud edge cooperation | |
| CN113014968A (en) | Multi-user dynamic code rate video transmission method and system based on reinforcement learning | |
| CN114401253B (en) | Video transmission method in analog network based on WebRTC | |
| CN115484240A (en) | Decoding method, decoding device, data transmission method, data transmission device, terminal and server | |
| CN109862386A (en) | Live data transmission method and device | |
| US20230396783A1 (en) | Data processing method and apparatus, device, and readable storage medium | |
| CN1468002A (en) | Flow media compression, transmission and storage system based on internet | |
| CN116320431B (en) | Video compression coding data dynamic wireless network transmission system | |
| CN104243412B (en) | Network data processing terminal and method in wherein network data | |
| EP1507412A1 (en) | Information processing device and method, recording medium, and program | |
| CN101651827A (en) | Code rate control method of screen coding | |
| CN116962179A (en) | Network transmission optimization method and device, computer readable medium and electronic equipment | |
| CN116156182A (en) | Data processing method, device, electronic equipment and medium | |
| CN107105265B (en) | Dynamic image prediction decoding method, dynamic image prediction decoding device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |