CN113904989B - Throughput rate improving method based on NewReno - Google Patents

Throughput rate improving method based on NewReno Download PDF

Info

Publication number
CN113904989B
CN113904989B CN202111225179.7A CN202111225179A CN113904989B CN 113904989 B CN113904989 B CN 113904989B CN 202111225179 A CN202111225179 A CN 202111225179A CN 113904989 B CN113904989 B CN 113904989B
Authority
CN
China
Prior art keywords
congestion window
newreno
congestion
label
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111225179.7A
Other languages
Chinese (zh)
Other versions
CN113904989A (en
Inventor
卢继华
殷宇航
冯立辉
傅雄军
吴编
周正阳
谢民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN202111225179.7A priority Critical patent/CN113904989B/en
Publication of CN113904989A publication Critical patent/CN113904989A/en
Application granted granted Critical
Publication of CN113904989B publication Critical patent/CN113904989B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/12—Avoiding congestion; Recovering from congestion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/27—Evaluation or update of window size, e.g. using information derived from acknowledged [ACK] packets
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention relates to a throughput rate improving method based on NewReno, and belongs to the technical field of network congestion control in wireless network transmission. The method adopts NewReno at the slow start stage; in the congestion avoidance stage, the sending end performs strategy learning according to the network state and the Sarsa method, and selects actions according to the learned Q table, namely, when receiving a new ACK, the sending end keeps the original congestion window increment constant or reselects the new congestion window increment; when packet loss occurs, whether the packet loss is caused by bad communication links or barrier blocking at the moment is judged through a trained SVM model according to the position and the speed of a data transmitting end and the received SINR, so that a congestion window is kept unchanged or reduced, and the bandwidth utilization rate is improved. Compared with the NewReno, the invention improves the network throughput rate and the bandwidth utilization rate in the wireless network transmission and makes up the short plate of the NewReno in the wireless network transmission.

Description

Throughput rate improving method based on NewReno
Technical Field
The invention relates to a throughput rate improving method based on NewReno, and belongs to the technical field of network congestion control in wireless network transmission.
Background
The transmission control protocol (Transmission Control Protocol, TCP) is a connection-oriented, reliable, byte stream based transport layer communication protocol defined by IETF RFC 793. The main function of TCP is that when the application layer sends a data stream represented by 8 bytes for inter-network transmission to the TCP layer, TCP breaks the data stream into segments of appropriate length and passes the data packets to the IP layer, which transmits the packets over the network to the TCP layer of the receiving entity. In order to ensure reliable transmission of messages, TCP gives each packet a sequence number, and the sequence number also ensures sequential reception of packets transmitted to the receiving entity. Then the receiving end entity sends back a corresponding acknowledgement to the successfully received bytes; if the sender entity does not receive an acknowledgement within a reasonable round trip delay, the corresponding data will be retransmitted.
NewReno is a classical TCP congestion control method, which mainly comprises four main phases: slow start, congestion avoidance, fast retransmission and fast recovery; adding 1 to the congestion window every time the data transmitting end receives a new ACK in the slow start phase, and entering a congestion avoiding phase when the congestion window reaches a slow start threshold value; in the congestion avoidance stage, the increasing speed of the congestion window is slowed down, and 1 is added to the congestion window every transmission round; the fast retransmission is an improvement of timeout retransmission, and when the data sending end receives three repeated acknowledgements of the same message, the message segment is directly retransmitted, so that the occurrence of timeout is avoided; fast recovery is an improvement to the loss recovery mechanism, and after fast retransmission, the congestion avoidance phase is entered directly without going through a slow start procedure.
In the wired network, when the network is congested, timeout retransmission occurs, that is, the data transmitting end cannot receive a confirmation of a certain transmitted data packet within a set timeout retransmission time, so as to retransmit the data packet, and initialize a congestion window, so that the data transmitting end transmits a small amount of data packets next time, and the network congestion is relieved. The timeout retransmission occurs because the packet stays somewhere in the network for a long time or because the packet is discarded by a node in the network due to an insufficient buffer. However, in a wireless network, the cause of packet loss may be not only caused by network congestion, but also caused by poor quality of a communication link, for example, there is an obstacle between a communication device and a base station; if the network congestion is mistakenly considered by the sending end due to overtime retransmission caused by the overtime retransmission, a congestion window is initialized, the sending rate of the data packet is reduced, and the overall throughput rate of the network is reduced; this is a disadvantage of the NewReno congestion control method in wireless networks, and the present invention is an improvement of the NewReno method.
The invention has the application scene that after a disaster occurs, the unmanned aerial vehicle performs small-range shooting on a disaster area and transmits the disaster area back to the base station at a high speed and in real time by means of a 5G millimeter wave network, and the obstacle in the disaster area can influence the communication between the unmanned aerial vehicle and the base station, so that the NewReno is improved on the basis. The main idea is that in wireless communication, a data transmitting end and a receiving end firstly establish TCP connection, and newReno is adopted in a slow start stage; in the congestion avoidance stage, the data transmitting end performs strategy learning according to network states including round trip delay, moving index weighted average of data packet transmission time intervals, moving index weighted average of ACK arrival time intervals, size of congestion window and slow start threshold value by combining with a Sarsa method, so that action selection is performed according to a learned Q table, namely, when a new ACK is received, the increase and decrease amount of the original congestion window is kept unchanged or the increase and decrease amount of the new congestion window is reselected; when packet loss occurs, judging whether the packet loss is caused by interruption caused by bad communication links or barrier blocking or not through a trained SVM model according to the position, speed and received SINR of a data sending end, so that the congestion window is kept unchanged or reduced. The congestion window initialization caused by non-network congestion reasons is avoided, the network throughput is improved, and the method effect is verified by means of ns3 network simulation software.
Disclosure of Invention
Aiming at the technical defects that the congestion window is not initialized because of network congestion in the face of packet loss caused by poor quality of a communication link in the wireless communication network by the NewReno of the traditional TCP protocol, so that the link is underutilized and the throughput rate is not high enough, the invention provides a throughput rate improving method based on the NewReno, and the NewReno is adopted in the slow start stage of the method; in the congestion avoidance stage, the sending end performs strategy learning according to the network state and combines with the Sarsa method, and performs action selection according to the learned Q table, namely, when a new ACK is received, the increase and decrease of the original congestion window are kept unchanged or the increase and decrease of the new congestion window is reselected; when packet loss occurs, judging whether the packet loss is caused by bad communication links or barrier blocking at the moment through a trained SVM model according to the position, speed and received SINR of a data sending end, so that the congestion window is kept unchanged or reduced.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
the wireless transmission system based on the NewReno throughput rate improving method comprises a transmitting end, a receiving end and a base station; the mobile model of the transmitting end is a random waypoint model;
the transmitting end is in wireless connection with the base station; the base station is connected with the receiving end in a wired way; the transmission layers of the sending end and the receiving end adopt TCP protocol;
the throughput rate improving method based on NewReno modifies NewReno in a congestion control stage and when timeout retransmission occurs, and comprises the following steps:
step 1, a sending end initiates a TCP connection request to a receiving end, and a TCP connection is established;
step 2, after entering the congestion avoidance phase, at t 0 ～t 0 During +RTT, the transmitting end collects network state, initializes the Q table, learns and updates the Q table by using the Sarsa method, and then selects actions according to the Q table, so that the congestion window is increased or reduced;
wherein t is 0 In order to simulate the operation time when the sending end receives the newly arrived ACK for the first time after entering the congestion avoidance stage, RTT is the round trip time of data transmission between the sending end and the receiving end;
step 2.1), the transmitting end collects the network state s, which is specifically:
wherein the network state s comprises: the ratio of round trip delay to minimum delay, the moving exponential weighted average of packet transmission time intervals, the moving exponential weighted average of ACK arrival time intervals, the size of congestion window, and the slow start threshold;
step 2.2), initializing a Q table by a transmitting end;
wherein, the Q table comprises a state-action pair < s, a > and Q values corresponding to each state-action pair < s, a >; s is the state, a is the action;
step 2.3) learning and updating a Q table according to the Sarsa method, selecting actions according to the Q table, and updating the size of the congestion window, wherein the steps are as follows:
step 2.3.1), the transmitting end learns by adopting a Sarsa method to obtain a reward value, and then updates the Q value in the Q table according to the reward value;
the rewarding value is determined according to the relationship between the network utilization difference value and the threshold value epsilon when the Q table is updated this time and the Q table is updated last time, and specifically comprises the following steps:
1a) When the network utilization difference value is greater than or equal to a threshold value epsilon, setting the rewarding value as R1;
1b) When the network utilization difference value is more than or equal to 0 and less than the threshold value epsilon, setting the rewarding value as R2;
1c) When the network utilization difference is smaller than 0 and larger than-E, the rewarding value is set to be-R2;
1d) When the network utilization difference is smaller than-E, the rewarding value is set to be-R1;
wherein, threshold value e >0; the prize values R1 and R2 satisfy R1> R2>0;
step 2.3.2) updating the congestion window according to the Q table selection action, specifically:
select action a according to Q table and at t 0 ～t 0 Keeping action a unchanged when a new ACK arrives during +rtt, when the new ACK arrival time is less than or equal to t 0 +RTT, the sender only updates the congestion window; otherwise, when the new ACK arrival time is greater than t 0 +RTT, jumping to step 3;
wherein, update the congestion window, specifically: updating the size of the congestion window according to the action a;
wherein, the change quantity of the updated congestion window is the product of a and the reciprocal of the congestion window at the moment; and when the value of the action a is positive, the congestion window is increased; when the value of the action a is negative, the congestion window is reduced;
step 3, in congestion avoidance stage, when new ACK arrival time is greater than t 0 +RTT, the sending end re-collects the network state, learns by adopting a Sarsa method to obtain a reward value, updates the Q value in the Q table according to the reward value, selects actions according to the Q table, and updates the congestion window;
step 4, setting the height of the transmitting end as a fixed value, wherein the moving model of the transmitting end is a random waypoint model; randomly selecting a route point in the fixed area, and randomly selecting the next route point and the speed of reaching the route point after the node reaches the route point at a certain speed;
wherein the speed at which each node chooses to reach the waypoint obeys uniform distribution within a certain range;
step 5, recording the movement condition of a transmitting end, combining a rule of adding a label, acquiring a training set and a testing set, inputting the training set into an SVM for training, storing a trained SVM model, and testing the classification capability of the trained SVM model by using the testing set, wherein the method specifically comprises the following steps:
step 5.1) recording the movement condition of the transmitting end, including the current analog running time, the position, the speed, the signal-to-interference-plus-noise ratio SINR and the time-out retransmission time t RTO ；
Step 5.2) obtaining a training set and a testing set, which specifically comprise the following steps: forming a characteristic vector by the position, the speed and the signal-to-interference-plus-noise ratio SINR of a transmitting end in the recorded movement condition; adding three types of labels Hl, hs and Hn to each feature vector, obtaining the feature vector and the corresponding label, and constructing a training set and a testing set;
wherein, the label Hl indicates that the TCP connection is interrupted for a long time when packet loss occurs; label Hs indicates a short interruption of the TCP connection when packet loss occurs; label Hn represents network congestion when packet loss occurs;
three types of labels are added to each feature vector, and a training set and a testing set are constructed, wherein the method specifically comprises the following sub-steps:
step 5.2.1) judging whether the SINR in each piece of movement condition data is larger than a certain threshold value theta, if the SINR is larger than the threshold value theta, setting Hn in the label of the corresponding feature vector to be 1, setting Hs to be 0 and setting Hl to be 0, at the moment, simulating the running time to be t, continuing to add the label to the next feature vector, otherwise, jumping to the step 5.2.2);
step 5.2.2) judging the recorded movement condition data, wherein the movement condition data is from t to t+t RTO During this period, whether or not there is SIN in a certain piece of movement condition dataR is greater than a threshold value θ, if SINR exists>θ, the label Hs of the corresponding feature vector is set to 1, the label Hn is set to 0, and the label Hl is set to 0; otherwise, setting the label Hl of the corresponding feature vector to be 1, setting the label Hs to be 0, and setting the label Hn to be 0;
step 5.2.3) randomly selecting m% from the obtained feature vectors and the corresponding tag data to serve as a training set, and taking the rest feature vectors and tags as a test set;
wherein, the proportion range of m% is more than or equal to 60% and less than or equal to 90%;
step 5.3) inputting the training set into the SVM for training; after training, storing the trained model;
step 5.4) inputting the test set into a trained SVM model, and testing the classification capability of the model;
and 6, when packet loss occurs, the transmitting end inputs the characteristic vector formed by the position, the speed, the signal to interference plus noise ratio SINR of the current transmitting end into a trained SVM model, acquires a corresponding classification result, and takes corresponding measures for a congestion window, wherein the method comprises the following specific steps:
step 6.1) if the classification result is hl=1, the congestion window remains unchanged;
step 6.2) if the classification result is hs=1, the congestion window remains unchanged;
step 6.3) if the classification result is hn=1, the congestion window is reduced to the original k%;
wherein the proportion of k% is in the range of 30% to 80%;
thus, the throughput rate improving method based on NewReno is completed from the step 1 to the step 5.
Advantageous effects
Compared with the prior NewReno, the throughput rate improving method based on NewReno has the following beneficial effects:
1. according to the throughput rate improving method based on the NewReno, the situations that the network throughput rate is low and the bandwidth cannot be fully utilized due to the fact that a TCP congestion window is initialized due to the fact that the congestion is not caused by the blocking of an obstacle under the condition of wireless communication are considered, and corresponding measures are taken when packet loss occurs, so that a short board of the NewReno under the condition of wireless is made up;
2. according to the throughput rate improving method based on NewReno, the network state is collected in time, and the congestion window is updated by using the reinforcement learning method, so that the ratio of increasing or decreasing the congestion window is decided according to the real-time congestion state of the network in the congestion avoiding stage, and the network throughput rate is improved;
3. compared with the NewReno, the throughput rate improving method based on the NewReno has the advantages that the throughput rate improving proportion is large, the time delay is improved, and the advantages brought by the method are negligible compared with the throughput rate improving.
Drawings
FIG. 1 is a flow chart of a reinforcement learning module in a throughput rate improving method based on NewReno according to the present invention;
FIG. 2 is a flow chart of an SVM training module in a throughput rate enhancement method based on NewReno in accordance with the present invention;
FIG. 3 is a flow chart of a throughput rate enhancement method based on NewReno of the present invention;
fig. 4 is a topology distribution diagram of initial positions of a base station, an obstacle, and an unmanned aerial vehicle in embodiment 1 of a throughput rate improving method based on NewReno of the present invention;
fig. 5 is a topology distribution diagram of initial positions of a base station, an obstacle, and a drone in embodiment 2 of a throughput rate improving method based on NewReno of the present invention.
Detailed Description
The following describes a throughput rate improving method based on NewReno in detail with reference to the accompanying drawings and embodiments.
Example 1
The application scene of the throughput rate improving method based on NewReno is that after a disaster occurs, an unmanned aerial vehicle carries out small-range shooting on a disaster area and transmits the disaster area back to a base station in real time at a high speed by means of a 5G millimeter wave network, an obstacle in the disaster area can influence communication between the unmanned aerial vehicle and the base station, and a traditional TCP congestion control method can cause packet loss due to obstacle blocking, so that a congestion window is initialized to influence the network throughput rate and the utilization rate, therefore, the invention provides the congestion control method for improving the throughput rate on the basis of NewReno, and the congestion control method is more suitable for being used as a congestion control method under a wireless network; in addition, under the condition of better communication link, compared with the NewReno, the invention can also improve throughput rate and network utilization rate, so that bandwidth is fully utilized.
Aiming at the problem that the existing TCP NewReno causes packet loss in a wireless communication network due to poor quality of a communication link, so that a congestion window is not initialized due to network congestion, and therefore, the link is not fully utilized and the throughput rate is not high enough, a NewReno-based throughput rate improving method is provided, and the method still adopts NewReno in a slow starting stage; in the congestion avoidance stage, policy learning is carried out by combining a Sarsa method according to network states including round trip delay, moving exponential weighted average of data packet sending time intervals, moving exponential weighted average of ACK arrival time intervals, size of a congestion window and a slow start threshold value, so that action selection is carried out according to a learned Q table, namely, the increasing and decreasing amount of the congestion window is selected when a new ACK is received; when packet loss occurs, judging whether the packet loss is caused by interruption caused by bad communication links or obstacle blocking at the moment or not through a trained SVM model according to the position, speed and received SINR of the unmanned aerial vehicle, thereby keeping the congestion window unchanged or reduced.
The embodiment is based on Network Simulator 3 simulation software, namely ns3 software, and is mainly applied to a TCP module, a random waypoint module, a building module and a 5G millimeter wave module which is expanded on the basis.
Now consider the following scenario: topology area 2000m 2 In this area, 16 0.5m wide by 3.5m high obstacles and 2 0.5m wide by 20m high obstacles are distributed, each 40m long and 50m wide. The drone connected to the 5G base station flies in the topological area at a constant altitude, and a TCP connection is established with a server at a far end through the 5G base station, and data is transmitted thereto, as shown in fig. 4, wherein a curve with an arrow indicates a flight direction.
In the embodiment, the positions of the unmanned aerial vehicle and the building are represented by (x, y and z), wherein x is more than or equal to 0 and less than or equal to 40, y is more than or equal to 0 and less than or equal to 50, and z is more than or equal to 0, and the unit is m;
initial condition setting of a base station and an unmanned aerial vehicle: the base station 10m is high, the unmanned aerial vehicle Gao Duheng is 10m, the initial position of the unmanned aerial vehicle is (0, 10), the model adopted by the unmanned aerial vehicle is a random waypoint model, in order to make the flight track more reasonable and cover the whole topological area as much as possible, the whole topology is divided into four areas, as shown in fig. 4, then the unmanned aerial vehicle randomly selects one waypoint and the speed of flying to the point in one area each time, after reaching the point, randomly selects the next waypoint and the speed in the area, after randomly selecting the next waypoint and the speed in the area for two times, the area is replaced, and the random waypoint and the speed are selected in the new area for two times, and so on until the simulation experiment is finished; the speed v-U (54,90) of the unmanned aerial vehicle, namely, the speed v-U is uniformly distributed between 54km/h and 90 km/h;
initial condition setting of TCP: setting transmission buffer and receiving buffer of TCP in ns3 to be 131072 x 50 bytes, namely modifying two parameters of SndBufSize and RcvBufSize; the initial threshold value of the congestion window is 20000, namely, the SlowStartThreshold parameter is modified;
the application of the application layer is self-defined application, the sending rate is 1000Mbps, the size of each data packet is 1400 bytes, and a strategy of continuously sending the data packets is adopted;
the link between the remote server and the 5G base station is a point-to-point channel, the propagation delay is 10ms, and the link bandwidth is 100Gbps;
the unmanned aerial vehicle starts to establish TCP to establish connection to the remote server at 0.1s of the simulation experiment, and the duration of each simulation experiment is 25s in ns 3.
In combination with the flowchart of the throughput rate improving method based on NewReno in fig. 3, the technical scheme adopted by the invention is as follows:
step 1, an unmanned aerial vehicle initiates a TCP connection request to a remote host, and establishes TCP connection; the congestion control method integrally adopts NewReno: namely, in the slow start phase, the fast retransmission phase and the fast recovery phase, the congestion control behavior adopted by the TCP is consistent with that of the NewReno; during the congestion control stage and packet loss, namely overtime retransmission, modifying the NewReno;
step 2, in combination with the flowchart of the reinforcement learning module in the throughput rate improving method based on NewReno in fig. 1, after entering the congestion avoidance stage, the unmanned aerial vehicle collects the network state, initializes the Q table, learns by applying the Sarsa method, updates the Q table, and takes greedy policy selection action according to the Q table, thereby increasing or decreasing the congestion window, specifically:
step 1, a sending end initiates a TCP connection request to a receiving end, and a TCP connection is established;
step 2, combining the flowchart of the reinforcement learning module in the throughput rate improving method based on NewReno of fig. 1, and after entering the congestion avoidance stage, at t 0 ～t 0 During +RTT, the unmanned plane collects network state, initializes the Q table, learns and updates the Q table by using the Sarsa method, and then selects actions according to the Q table, so that the congestion window is increased or reduced;
wherein t is 0 In order to simulate the operation time when the sending end receives the newly arrived ACK for the first time after entering the congestion avoidance stage, RTT is the round trip time of data transmission between the sending end and the receiving end;
the method comprises the following steps:
step 2.1), the sending end, namely the unmanned aerial vehicle, collects the network state s, specifically:
wherein the network state s comprises: the ratio of round trip delay to minimum delay, the moving exponential weighted average of packet transmission time intervals, the moving exponential weighted average of ACK arrival time intervals, the size of congestion window, and the slow start threshold;
step 2.2), initializing a Q table by a transmitting end;
wherein, the Q table comprises a state-action pair < s, a > and Q values corresponding to each state-action pair < s, a >; s is the state, a is the action;
step 2.3) learning and updating a Q table according to the Sarsa method, selecting actions according to the Q table, and updating the size of the congestion window, wherein the steps are as follows:
step 2.3.1), the unmanned aerial vehicle learns by adopting a Sarsa method to obtain a reward value, and then updates a Q value in a Q table according to the reward value, wherein a Q value updating formula is shown in a formula (2.1);
Q(s n ,a n )＝Q(s n ,a n )+α(r n+2 +γQ(s n+1 ,a n+1 )-Q(s n ,a n )) (2.1)
in the formula (2.1), s n Sum s n+1 Representing the network state at this moment and the network state at the next update of the Q-table, a) n And a n+1 Representing the action taken at this moment and the action taken next time when the Q table is updated, respectively, alpha being the learning rate, gamma being the discount factor, r n+2 As a prize value, as shown in formula (2.2):
delta in formula (2.2) n+2 ＝U n+2 -U n+1 U representing a change value of network utilization n The network utilization rate from the moment n-1 to the moment n after the TCP sender takes a certain action a is shown, and the network utilization rate adopts a definition mode as shown in the following formula (2.3):
in formula (2.3), tp is throughput, B represents the bandwidth of the wired connection between the remote host and the base station, d=rtt-RTT min For delay difference, p is packet loss rate, delta 1 、δ 2 Delta as a weight parameter 1 ＝0.05，δ 2 ＝0.01；
Select action a from the Q table, and at t 0 ～t 0 The action selection is not carried out when a new ACK arrives in the +RTT period, namely the action a selected last time is kept unchanged, wherein the action a has four values which are +3, +1, +0.5 or-1 respectively; RTT is round trip time of data transmission between a sending end and a receiving end;
step 2.2.3) updating the congestion window, the added congestion window being a multiplied by the inverse of the congestion window, as shown in equation (2.4):
wherein cwnd is a congestion window;
select action a according to Q table and at t 0 ～t 0 Keeping action a unchanged when a new ACK arrives during +rtt, when the new ACK arrival time is less than or equal to t 0 +RTT, the sender only updates the congestion window; otherwise, when the new ACK arrival time is greater than t 0 +RTT, jumping to step 3;
step 3, in congestion avoidance stage, when new ACK arrival time is greater than t 0 +RTT, the sending end re-collects the network state, learns by adopting a Sarsa method to obtain a reward value, updates the Q value in the Q table according to the reward value, selects actions according to the Q table, and updates the congestion window;
step 4, in ns3, the unmanned aerial vehicle Gao Duheng is 10m, the initial position of the unmanned aerial vehicle is (0, 10), the model adopted by the unmanned aerial vehicle is a random waypoint model, in order to make the flight track more reasonable and cover the whole topological area as much as possible, the whole topology is divided into four areas, as shown in fig. 4, the unmanned aerial vehicle randomly selects one waypoint and the speed of flying to the point in one area each time, after reaching the point, randomly selects the next waypoint and the speed in the area, after randomly selecting for two times, the area is replaced, and the random waypoint and the speed are selected for two times in the new area, and so on until the simulation experiment is finished; the speed v-U (54,90) of the unmanned aerial vehicle, namely, the speed v-U is uniformly distributed between 54km/h and 90 km/h;
step 5, recording the movement condition of the unmanned aerial vehicle, acquiring a training set and a testing set by combining the rule of adding the label, inputting the training set into the SVM for training, storing a trained SVM model, and testing the classification capability of the trained SVM model by using the testing set as shown in figure 2;
the method specifically comprises the following steps:
step 5.1) performing multiple simulation operation experiments, and recording the movement condition of the unmanned aerial vehicle at intervals of 0.1s, wherein the movement condition comprises the current simulation operation time t and the position (l) of a transmitting end x 、l y 、l z ) Speed (v) x 、v y 、v z ) Signal to interference plus noise ratio SINR, timeout retransmission time t RTO ；
Step 5.2) obtainingThe training set and the testing set are taken, and specifically comprise: the position of the sender in the case of movement to be recorded (l x 、l y 、l z ) Speed (v) x 、v y 、v z ) And the signal-to-interference-plus-noise ratio SINR constitutes a feature vector; adding three types of labels Hl, hs and Hn to each feature vector, thereby obtaining the feature vector and the corresponding label;
wherein, the label Hl indicates that the TCP connection is interrupted for a long time when packet loss occurs; label Hs indicates a short interruption of the TCP connection when packet loss occurs; label Hn represents network congestion when packet loss occurs;
when three types of labels are added, the method comprises the following steps:
step 5.2.1) judging whether the SINR in each piece of movement condition data is larger than a certain threshold value theta, if the SINR is larger than the certain threshold value theta, setting Hn in the label of the corresponding feature vector to be 1, setting Hs to be 0, setting Hl to be 0, and continuing to add the label to the next feature vector when the simulation running time is t; otherwise, executing the step 5.2.2);
step 5.2.2) judging the recorded movement condition data, wherein the movement condition data is from t to t+t RTO During the period, if the SINR in the data with a certain movement condition is greater than a certain threshold value theta, if the SINR is present>θ, the label Hs of the corresponding feature vector is set to 1, the label Hn is set to 0, and the label Hl is set to 0; otherwise, setting the label Hl of the corresponding feature vector to be 1, setting the label Hs to be 0, and setting the label Hn to be 0;
randomly selecting 70% -80% of the obtained feature vectors and corresponding label data as a training set, and taking the rest feature vectors and labels as a test set;
step 5.3) inputting the training set into the SVM for training; a linear kernel function is adopted; after training, storing the trained model;
step 5.4) inputting the test set into a trained SVM model, and testing the classification capability of the model;
step 6, when packet loss occurs, the sender sends the current sender position (l x 、l y 、l z ) Speed (v) x 、v y 、v z ) Sum signal to interference plus noise ratio SINR component feature vectors are input into a trained SVM model, corresponding classification results are obtained, and corresponding measures are taken for congestion windows, specifically as follows:
step 6.1) if the classification result is hl=1, the congestion window remains unchanged;
step 6.2) if the classification result is hs=1, the congestion window remains unchanged;
step 6.3) if the classification result is hn=1, the congestion window is reduced to half of the original congestion window;
the realization of the throughput rate improving method based on NewReno is completed from the step 1 to the step 5.
Comparison of experimental results:
in this example, the comparison of throughput and delay compared to NewReno is shown in table 1:
table 1 throughput rate comparison
As can be seen from table 1, the throughput rate of the TCP congestion control method according to the present invention is improved more than that of the NewReno topology, the ratio is about 136.83%, and the delay increment is not large; the cost of the delay increase is negligible compared to the throughput improvement.
Example 2
The present embodiment is similar to the embodiment, except that the distribution of obstacles in the topology is modified, specifically: in a region 40m long and 50m wide, 5m wide and 5m high obstacles are distributed, as shown in fig. 5, wherein the curve with the arrow indicates the flight direction.
In this example, the comparison of throughput and delay compared to NewReno is shown in table 2:
table 2 throughput rate comparison
As can be seen from table 2, the method for controlling the TCP congestion has a larger throughput rate improvement and a smaller delay increment compared with the method adopting NewReno under the topology; the cost of delay increase is negligible compared to throughput improvement;
the simulation results of the two embodiments prove that compared with the NewReno throughput rate, the throughput rate improving method based on NewReno has the advantages that the throughput rate is greatly improved, the time delay increment is negligible, and the method has quite advantages.
The invention has been described in detail with reference to two specific embodiments. While the invention is described in connection with the preferred embodiments, the invention should not be limited to the embodiments and drawings disclosed. All equivalents and modifications that come within the spirit of the disclosure are desired to be protected.

Claims (6)

1. The throughput rate improving method based on NewReno comprises a transmitting end, a receiving end and a base station; the mobile model of the transmitting end is a random waypoint model, and the transmitting end is in wireless connection with the base station; the base station is connected with the receiving end in a wired way, and is characterized in that: during the congestion control phase and when timeout retransmission occurs, the NewReno is modified, which comprises the following steps:
step 1, a sending end initiates a TCP connection request to a receiving end, and a TCP connection is established;
step 2, after entering the congestion avoidance phase, at t 0 ～t 0 During +RTT, the transmitting end collects network state, initializes the Q table, learns and updates the Q table by using the Sarsa method, and then selects actions according to the Q table, so that the congestion window is increased or reduced;
wherein t is 0 In order to simulate the operation time when the sending end receives the newly arrived ACK for the first time after entering the congestion avoidance stage, RTT is the round trip time of data transmission between the sending end and the receiving end;
step 2, comprising the following sub-steps:
step 2.1), the transmitting end collects the network state s, which is specifically:
wherein the network state s comprises: the ratio of round trip delay to minimum delay, the moving exponential weighted average of packet transmission time intervals, the moving exponential weighted average of ACK arrival time intervals, the size of congestion window, and the slow start threshold;
step 2.2), initializing a Q table by a transmitting end;
wherein, the Q table comprises a state-action pair < s, a > and Q values corresponding to each state-action pair < s, a >; s is the state, a is the action;
step 2.3) learning and updating a Q table according to the Sarsa method, selecting actions according to the Q table, and updating the size of the congestion window, wherein the steps are as follows:
step 2.3.1), the transmitting end learns by adopting a Sarsa method to obtain a reward value, and then updates the Q value in the Q table according to the reward value;
step 2.3.2) updating the congestion window according to the Q table selection action, specifically:
select action a according to Q table and at t 0 ～t 0 Keeping action a unchanged when a new ACK arrives during +rtt, when the new ACK arrival time is less than or equal to t 0 +RTT, the sender only updates the congestion window; otherwise, when the new ACK arrival time is greater than t 0 +RTT, jumping to step 3;
step 3, in congestion avoidance stage, when new ACK arrival time is greater than t 0 +RTT, the sending end re-collects the network state, learns by adopting a Sarsa method to obtain a reward value, updates the Q value in the Q table according to the reward value, selects actions according to the Q table, and updates the congestion window;
step 4, setting the height of the transmitting end as a fixed value, wherein the moving model of the transmitting end is a random waypoint model; randomly selecting a route point in the fixed area, and randomly selecting the next route point and the speed of reaching the route point after the node reaches the route point at a certain speed;
step 5, recording the movement condition of the transmitting end, combining the rule of adding the label, acquiring a training set and a testing set, inputting the training set into the SVM for training, storing a trained SVM model, and testing the classification capability of the trained SVM model by using the testing set;
step 5, specifically comprising the following steps:
step (a)5.1 Recording the movement condition of the transmitting end, including the current analog running time, the position, the speed, the signal-to-interference-plus-noise ratio SINR and the time-out retransmission time t RTO ；
Step 5.2) obtaining a training set and a testing set, which specifically comprise the following steps: forming a characteristic vector by the position, the speed and the signal-to-interference-plus-noise ratio SINR of a transmitting end in the recorded movement condition; adding three types of labels Hl, hs and Hn to each feature vector, obtaining the feature vector and the corresponding label, and constructing a training set and a testing set;
wherein, the label Hl indicates that the TCP connection is interrupted for a long time when packet loss occurs; label Hs indicates a short interruption of the TCP connection when packet loss occurs; label Hn represents network congestion when packet loss occurs;
step 5.3) inputting the training set into the SVM for training; after training, storing the trained model;
step 5.4) inputting the test set into a trained SVM model, and testing the classification capability of the model;
and 6, when packet loss occurs, the transmitting end inputs a characteristic vector formed by the position, the speed, the signal to interference plus noise ratio SINR of the current transmitting end into a trained SVM model to obtain a corresponding classification result, and when the classification result is Hl=1 and Hs=1, the congestion window is kept unchanged, and when Hn=1, the congestion window is reduced.
2. The NewReno-based throughput rate improving method according to claim 1, wherein: in step 2.3.1), the rewarding value is determined according to the relationship between the network utilization difference value and the threshold value epsilon when the Q table is updated this time and the Q table is updated last time, specifically:
1a) When the network utilization difference value is greater than or equal to a threshold value epsilon, setting the rewarding value as R1;
1b) When the network utilization difference value is more than or equal to 0 and less than the threshold value epsilon, setting the rewarding value as R2;
1c) When the network utilization difference is smaller than 0 and larger than-E, the rewarding value is set to be-R2;
1d) When the network utilization difference is smaller than-E, the rewarding value is set to be-R1;
threshold e >0; the prize values R1 and R2 satisfy R1> R2>0.
3. The NewReno-based throughput rate improving method according to claim 1, wherein: in step 2.3.2), the congestion window is updated, specifically: updating the size of the congestion window according to the action a; and updating the variation of the congestion window to be the product of a and the reciprocal of the congestion window at the moment; and when the value of the action a is positive, the congestion window is increased; when the value of the action a is negative, the congestion window is reduced.
4. The NewReno-based throughput rate improving method according to claim 1, wherein: in step 4, the speed at which each node chooses to reach the waypoint is subject to a uniform distribution over a range.
5. The NewReno-based throughput rate improving method according to claim 1, wherein: in step 5.2), three types of labels are added to each feature vector, and a training set and a testing set are constructed, specifically comprising the following sub-steps:
step 5.2.1) judging whether the SINR in each piece of movement condition data is larger than a certain threshold value theta, if the SINR is larger than the certain threshold value theta, setting Hn in the label of the corresponding feature vector to be 1, setting Hs to be 0, setting Hl to be 0, simulating the running time to be t at the moment, continuing to add the label to the next feature vector, and otherwise, executing step 5.2.2;
step 5.2.2) judging the recorded movement condition data, wherein the movement condition data is from t to t+t RTO During the period, if the SINR in the data with a certain movement condition is greater than a certain threshold value theta, if the SINR is present>θ, the label Hs of the corresponding feature vector is set to 1, the label Hn is set to 0, and the label Hl is set to 0; otherwise, setting the label Hl of the corresponding feature vector to be 1, setting the label Hs to be 0, and setting the label Hn to be 0;
step 5.2.3) randomly selecting m% from the obtained feature vectors and the corresponding tag data to serve as a training set, and taking the rest feature vectors and tags as a test set; wherein the proportion range of m% is more than or equal to 60% and less than or equal to 90%.
6. The NewReno-based throughput rate improving method according to claim 1, wherein: step 6, the concrete steps are as follows:
step 6.1) if the classification result is hl=1, the congestion window remains unchanged;
step 6.2) if the classification result is hs=1, the congestion window remains unchanged;
step 6.3) if the classification result is hn=1, the congestion window is reduced to the original k%; wherein the proportion of k% is in the range of 30% to 80%.
CN202111225179.7A 2021-10-21 2021-10-21 Throughput rate improving method based on NewReno Active CN113904989B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111225179.7A CN113904989B (en) 2021-10-21 2021-10-21 Throughput rate improving method based on NewReno

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111225179.7A CN113904989B (en) 2021-10-21 2021-10-21 Throughput rate improving method based on NewReno

Publications (2)

Publication Number Publication Date
CN113904989A CN113904989A (en) 2022-01-07
CN113904989B true CN113904989B (en) 2023-11-07

Family

ID=79193038

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111225179.7A Active CN113904989B (en) 2021-10-21 2021-10-21 Throughput rate improving method based on NewReno

Country Status (1)

Country Link
CN (1) CN113904989B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101610199A (en) * 2008-06-17 2009-12-23 北京邮电大学 Heterogeneous network packet loss differentiating method based on fuzzy comprehensive evoluation
CN104378307A (en) * 2014-10-27 2015-02-25 北京华夏创新科技有限公司 Optimizing method and system based on throughput rate and packet loss control CWND
CN104640155A (en) * 2013-11-11 2015-05-20 西南科技大学 Gray throughput forecast based wireless Ad Hoc network TCP (transmission control protocol) congestion avoidance mechanism
GB201812862D0 (en) * 2018-08-08 2018-09-19 British Telecomm Improved congestion response
US10873533B1 (en) * 2019-09-04 2020-12-22 Cisco Technology, Inc. Traffic class-specific congestion signatures for improving traffic shaping and other network operations

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101610199A (en) * 2008-06-17 2009-12-23 北京邮电大学 Heterogeneous network packet loss differentiating method based on fuzzy comprehensive evoluation
CN104640155A (en) * 2013-11-11 2015-05-20 西南科技大学 Gray throughput forecast based wireless Ad Hoc network TCP (transmission control protocol) congestion avoidance mechanism
CN104378307A (en) * 2014-10-27 2015-02-25 北京华夏创新科技有限公司 Optimizing method and system based on throughput rate and packet loss control CWND
GB201812862D0 (en) * 2018-08-08 2018-09-19 British Telecomm Improved congestion response
US10873533B1 (en) * 2019-09-04 2020-12-22 Cisco Technology, Inc. Traffic class-specific congestion signatures for improving traffic shaping and other network operations

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于认知无线电的频谱感知及TCP跨层设计;林正红等;计算机工程;第40卷(第7期);第33-37页 *

Also Published As

Publication number Publication date
CN113904989A (en) 2022-01-07

Similar Documents

Publication Publication Date Title
CN107171842B (en) Multipath transmission protocol congestion control method based on reinforcement learning
Matsuo et al. Performance analysis of WMNs by WMN-GA simulation system for two WMN architectures and different TCP congestion-avoidance algorithms and client distributions
CN111479306B (en) Q-learning-based flight ad hoc network QoS routing method
CN106059950B (en) A kind of adaptive network congestion control method based on SCPS-TP
CN101854738B (en) Transmission control protocol method for satellite network
CN101651963B (en) Transmission control method based on differentiated service in IEEE 802.11 WLAN
JPH03174848A (en) Delay base rush evading method in computer network and device
Ma et al. Performance improvements of mobile SCTP in integrated heterogeneous wireless networks
CN113329413A (en) Multipath transmission data pre-scheduling method and system for high-speed rail scene
CN105376172A (en) Method for controlling network congestion
CN109587751B (en) Self-adaptive routing method and system for unmanned aerial vehicle ad hoc network
CN113904989B (en) Throughput rate improving method based on NewReno
KR100534610B1 (en) method and system for controlling packet transmission using bind update message when mobile node performs hand-off in IPv6 based wireless network
CN115037672B (en) Multipath congestion control method and device
CN112822720A (en) Cross-layer congestion control method based on MAC (media Access control) layer link quality in unmanned aerial vehicle networking technology
Sardar et al. A novel enhancement of TCP for on-board IP networks with wireless cellular connectivity
Leung et al. Methods to improve TCP throughput in wireless networks with high delay variability [3G network example]
Rahimi et al. An improvement mechanism for low priority traffic TCP performance in Strict Priority Queueing
Mbarushimana et al. A cross-layer TCP enhancement in QoS-aware mobile ad hoc networks
CN113766561B (en) Unmanned cluster network congestion control method based on cross-layer optimization
Bai et al. MACC: Cross-layer multi-agent congestion control with deep reinforcement learning
KR100952228B1 (en) Appratus for TCP congestion control, a method thereof and recorded medium recorded with a program executing the same
Park et al. A packet scheduling scheme for seamless transmission of life media contents
Gong et al. Nuwa-RL: A Reinforcement Learning based Receiver-side Congestion Control Algorithm to Meet Applications Demands over Dynamic Wireless Networks
Poorzare Contribution to reliable end-to-end communication over 5G networks using advanced techniques

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant