CN112468405B

CN112468405B - Data center network congestion control method based on credit and reaction type

Info

Publication number: CN112468405B
Application number: CN202011395285.5A
Authority: CN
Inventors: 董德尊; 白洋; 胡鼎煌; 黄山; 廖湘科; 罗章; 欧洋
Original assignee: National University of Defense Technology
Current assignee: National University of Defense Technology
Priority date: 2020-11-30
Filing date: 2020-11-30
Publication date: 2022-05-27
Anticipated expiration: 2040-11-30
Also published as: CN112468405A

Abstract

A congestion control method for a data center network based on credit and reaction types comprises the following steps: at the switch, it separates the data path and the credit path on a physical channel; the credit packet data is transmitted by using a credit channel, and the normal packet data is transmitted by using a data channel; the receiving end sends a credit packet to the sending end after receiving the credit request, and the sending end sends data according to the received credit packet; when the network is not detected to be in a congestion state, a credit feedback control method based on an active congestion control protocol is adopted; and when the network is detected to be in a congestion state, adopting a feedback control method based on explicit congestion control to enter a low sending rate mode. The method breaks the isolation between the data queue and the credit interest rate limiter by using the ECN as a congestion signal; the novel efficient feedback control algorithm based on ECN can guarantee the high performance of CCRP without interfering with other traffic in the network.

Description

Data center network congestion control method based on credit and reaction type

Technical Field

The invention relates to the field of data center networks, in particular to a credit and reaction type-based data center network congestion control method.

Background

In recent years, the size and link speed of Data Center Networks (DCNs) has been rapidly increasing. Large data centers use shallow buffered switches consisting of Clos networks to connect 100,000 multiple computers. In the past decade, link speed has steadily increased from 10Gbps to 100 Gbps. These developments in DCNs enable low latency and high bandwidth communications in data centers, while also presenting a set of challenges to congestion control.

In order to solve the congestion control problem of DCN, many reactive congestion control methods have been proposed, which use congestion signals, such as packet loss, Explicit Congestion Notification (ECN), network delay, etc., to make an accurate response after congestion occurs, so as to maintain good average delay performance of the system under a long traffic condition. However, since the network congestion detection speed is slow, it is difficult for the reactive protocol method to obtain an appropriate rate in each flow.

In response to this problem, active congestion control methods have received much attention in recent years. The active congestion control protocol method (ExpressPass method) has the advantages of zero data loss, fast convergence, low buffer occupancy rate and high utilization rate. However, the current DCNs, such as google and amazon, still mainly deploy ECN-based reactive protocols, and therefore, the gradual deployment of the active congestion control method into the DCN is an important technical direction of future data centers. Nevertheless, deploying proactive credit-based protocols in DCNs can pose many challenges to fairness of bandwidth allocation, especially in multi-tenant DCNs, which can pose serious problems if the proactive approach is simply mixed with the reactive protocol approach deployed in the actual DCN.

Therefore, applying the proactive protocol approach in the current DCN faces many serious challenges, the root cause of which is due to the different approaches for detecting network congestion. Reactive protocols detect network congestion, such as packet loss, ECN and network delay, based on indirect and passive congestion signals used in data queues. Taking the common reactive congestion control protocol (DCTCP) as an example, when the queue length exceeds the ECN threshold in the switch, the packet will be marked using a Congestion Experience (CE) codepoint. DCTCP can then detect congestion by simply identifying whether the packet marks the ECN on the end host. However, the proactive congestion control protocol method acquires congestion information from a credit queue. Under the active congestion protocol, obvious physical isolation exists between a data queue and a credit queue, and the credit loss rate is used as a congestion index. When congestion is detected, the credit sending rate of the receiving party is reduced. Thus, if both active and reactive traffic are mixed in the network, the active credit-based congestion control mechanism cannot detect network congestion in the data queues, and packets will be transmitted at full speed even if the queue length exceeds the buffer size in the switch. In contrast, the reactive congestion control method will continuously reduce its transmission rate until its bandwidth occupancy approaches zero, since a large number of packets may be marked in the data queue using CE code points.

The congestion control method based on credit can only detect network congestion through the credit queue, which causes the network congestion to conflict with the reactive protocol method, therefore, the invention can also detect the congestion by optimizing the congestion detection mechanism of the active protocol method, and realizes the fusion of the protocol based on credit and the reactive protocol method.

Disclosure of Invention

The invention discloses a credit and reaction based congestion control method (CCRP) of a data center network, aiming at the problem that the network congestion is detected by a credit queue in the data center network to cause the network congestion to conflict with a reaction type protocol method, wherein the method comprises the following steps:

the size of the credit packet used is set at 84B, with 5% of the network link capacity being used for credit information transmission and the remaining 95% of the network link capacity being used for packet information transmission. At the switch, it separates the data path and the credit path on a physical channel. The credit packet data is transmitted using a credit path, and the normal packet data is transmitted using a data path. When a sending end wants to send data to a receiving end, a credit request is sent to the receiving end through a credit channel, and the credit request contains information such as the size and the source address of data flow needing to be sent. The receiving end sends credit packets to the sending end after receiving the credit request, the sending end sends data according to the received credit packets, and one credit packet can only schedule one data packet. When the network is not detected to be in a congestion state, a credit feedback control method based on an active congestion control protocol is adopted, namely, data packets are sent as much as possible within a range allowed by link capacity; when the network is detected to be in a congestion state, adopting a feedback control method based on explicit congestion control (ECN) to enter a low sending rate mode: the reasonable ECN threshold value is deployed on the switch, when other inactive flows exist in the network, the length of a data queue in the switch exceeds the ECN threshold value, the switch marks the data packet with the ECN, and after a receiving end receives the data packet with the ECN mark, the receiving end immediately and automatically reduces the sending rate. When the receiving end no longer receives the data packet with the ECN mark in a plurality of data packet Round Trip Times (RTTs), more credit information is sent.

The detection of the network congestion state uses the ECN to detect the network congestion of the data queue, and uses the instantaneous queue length as the judgment basis of the ECN marking. Specifically, two queue length thresholds Kmax and Kmin are set in the switch data queue, the Kmin value is set to 5KB, and the Kmax value is set to 200 KB. If the length value of the data queue in the switch is below Kmin, the data can be taken as common data to carry out forwarding operation, namely the probability of carrying out congestion marking is 0; if the length of the data queue in the switch is above Kmax, the switch considers that congestion occurs in the network, and the data packet is marked with a congestion mark, that is, the probability that the data packet is marked as congested is 1. If the length of the data queue in the switch is more than Kmin and less than Kmax, the probability of the data packet being marked with congestion is subjected to analog calculation, and the data packet is marked with congestion according to the calculation result probability.

Assuming that the probability of being marked as congested when the queue length is Kmax is Pmax, if the current instantaneous queue length is X and Kmin < X < Kmax, then the probability of the packet being marked as congested is: pmax × (X-Kmin)/(Kmax-Kmin).

The congestion marking of the data packet is specifically performed by modifying a numerical value of IP header data of the data packet, wherein the IP header data of the data packet includes a TOS field for indicating a differentiated service, and two idle numerical bits in the TOS field are used for performing congestion marking. When the sending host does not support the ECN function, the two remaining value bits in the data packet are marked as 00; when the sending host supports the ECN function, the two remaining value bits in the data packet are marked as 01 or 10; when the packet experiences congestion, the two remaining bits of the value in the packet are marked 11.

When the data traffic adopting the active protocol coexists with other data traffic adopting the reactive protocol in the network, and the queue length of the switch exceeds the congestion threshold, the switch marks the data packet with a congested ECN. And realizing the feedback congestion control method based on the ECN at the receiving end. When a data packet arrives at a receiving end, the receiving end detects congestion by detecting whether the data packet it receives is marked with a Congestion Experience (CE) code point. ECN _ ratio is the ratio of the number of packets marked ECN to the number of all packets. And when the ECN _ ratio value is larger than 0, judging that the data link is in a congestion state, otherwise, judging that the data link is in a normal state.

After the data link is judged to be in the congestion state, the receiving end starts an ECN-based feedback congestion control method, and the specific process comprises the following steps:

when the network is in a normal state, namely no data packet is marked with an ECN mark, the data packet is scheduled through a credit packet by using an active congestion control protocol method. During the first few RTTs, the network tends to send data traffic at the link rate, and after a number of RTTs, if any data packet with ECN congestion flag is not obtained, it is determined that there is no other data traffic in the network using the reactive protocol. In this stage, the packet loss rate of the credit information is detected at the same time, when the packet loss rate of the credit information is smaller than the set credit packet loss threshold value target _ loss, the network enters a speed-up stage, and the corresponding credit sending rate is calculated by adopting the following method:

w＝(w+wmax)/2，

cur_rate＝(1-w)*tmp_rate+w*max_rate，

wherein w is a calculation factor, cur _ rate is the current credit sending rate, tmp _ rate is the last round of credit sending rate, max _ rate is the maximum sending rate of credit, and target _ loss is a set credit packet loss threshold, and in order to reduce the initial deployment aggressivity of the method, the target _ loss value is set to 0; the value range of wmax is more than or equal to 0.04 and less than or equal to 0.07.

When the packet loss rate of the credit information is greater than the set credit packet loss threshold value target _ loss, the network enters a deceleration stage, and the corresponding credit sending rate is calculated in the following way:

cur_rate＝(1-credit_loss_rate)*tmp_rate，

w＝max(wmin,w/2)，

max () represents the maximum value of the two variables in the brackets, and credit _ loss _ rate is the actual packet loss rate of the credit packet.

When detecting that ECN _ ratio is larger than threshold value target _ ECN _ ratio of data volume marked with ECN congestion, judging that the network is in a congestion state, and performing the following operations:

ECN_a＝(1-g)*ECN_a+g*ECN_ratio，

tmp_rate＝(1-(target_ECN_ratio+ECN_a)/2)*cur_rate，

ECN _ a is the intermediate calculated amount, and g is a set weight value, and the value of g is 0.5. In a shallow buffered switch environment, to ensure that the average queue length in the switch is around the ECN threshold, the targetECNratio is set to 0.

The invention has the beneficial effects that: the method breaks the isolation between the data queue and the credit interest rate limiter by using the ECN as a congestion signal; the novel efficient feedback control algorithm based on ECN can guarantee the high performance of CCRP without interfering with other traffic in the network. Evaluation results show that the method can greatly reduce the competition difference between credit-based active protocols such as ExpressPass and the like and traditional passive protocols such as DCQCN and the like. Compared with credit-based active protocols such as Expresspass, the method can perfectly converge and simultaneously realize high utilization rate, fairness, small FCT and lower buffer occupancy rate. Therefore, the invention has high applicability in the deployment of the data center based on the credit congestion control protocol.

Drawings

Fig. 1 shows fairness test results of the method and the existing network congestion control method.

Fig. 2 shows the convergence test results of the present method and the existing network congestion control method.

Detailed Description

For a better understanding of the present disclosure, an example is given here.

The invention discloses a credit and reaction type-based data center network congestion control method (CCRP), which comprises the following steps:

the size of the credit packet used in this method is set to 84B, with 5% of the network link capacity being used for credit information transmission and the remaining 95% of the network link capacity being used for data packet information transmission. At the switch, it separates the data path and the credit path on a physical channel. The credit packet data is transmitted by using a credit channel, and the normal packet data is transmitted by using a data channel, wherein the credit packet data and the normal packet data are not interfered with each other. When a sending end wants to send data to a receiving end, a credit request is sent to the receiving end through a credit channel, and the credit request contains information such as the size and the source address of data flow needing to be sent. The receiving end sends credit packets to the sending end after receiving the credit request, the sending end sends data according to the received credit packets, and one credit packet can only schedule one data packet. When the network is not detected to be in a congestion state, a credit feedback control method based on an active congestion control protocol is adopted, namely, data packets are sent as much as possible within a range allowed by link capacity; when the network is detected to be in a congestion state, adopting a feedback control method based on explicit congestion control (ECN) to enter a low sending rate mode: the reasonable ECN threshold value is deployed on the switch, when other inactive flows exist in the network, the length of a data queue in the switch exceeds the ECN threshold value, the switch marks the data packet with the ECN, and after a receiving end receives the data packet with the ECN mark, the receiving end immediately and automatically reduces the sending rate. When the receiving end does not receive the data packet with the ECN mark any more in a plurality of data packet Round Trip Time (RTT), more credit information is sent.

The detection of the network congestion state uses the ECN to detect the network congestion of the data queue, and uses the instantaneous queue length as the judgment basis of the ECN marking. Specifically, two queue length thresholds Kmax and Kmin are set in the switch data queue, the Kmin value is set to 5KB, and the Kmax value is set to 200 KB. If the length value of the queuing data in the switch is below Kmin, the data can be taken as common data to carry out forwarding operation, namely the probability of carrying out congestion marking is 0; if the length of the queued data in the switch is above Kmax, the switch considers that congestion occurs in the network, and the packet is marked with a congestion flag, that is, the probability that the packet is marked as congested is 1. If the queue length in the switch is more than Kmin and less than Kmax, the probability of the data packet being marked with congestion is calculated in a simulation mode, and the data packet is marked with congestion according to the probability of the calculation result.

When the data traffic adopting the active protocol coexists with other data traffic adopting the reactive protocol in the network, and the queue length of the switch exceeds the congestion threshold, the switch marks the data packet with a congested ECN. And realizing the feedback congestion control method based on the ECN at the receiving end. When the data packet arrives at the receiving end, the receiving end detects the congestion by detecting whether the received data packet is marked with a CE code point. ECN _ ratio is the ratio of the number of packets marked ECN to the number of all packets. And when the ECN _ ratio value is larger than 0, judging that the data link is in a congestion state, otherwise, judging that the data link is in a normal state.

when the network is in a normal state, namely no data packet is marked with an ECN mark, an active congestion control protocol method is used for scheduling the data packet through a credit packet. During the first few RTTs, CCRP tends to send data traffic at the link rate, and after several RTTs, if no data packet with ECN congestion flag is obtained, it is determined that there is no other data traffic in the network using the reactive protocol. In this stage, the packet loss rate of the credit information is detected at the same time, when the packet loss rate of the credit information is smaller than the set credit packet loss threshold value target _ loss, the network enters a speed-up stage, and the corresponding credit sending rate is calculated by adopting the following method:

w＝(w+wmax)/2，

cur_rate＝(1-w)*tmp_rate+w*max_rate，

wherein w is a calculation factor, cur _ rate is the current credit sending rate, tmp _ rate is the last round of credit sending rate, max _ rate is the maximum sending rate of credit, and target _ loss is a set credit packet loss threshold, and in order to reduce the initial deployment aggressivity of the method, the target _ loss value is set to 0; the value range of wmax is more than or equal to 0.04 and less than or equal to 0.07. In the experiment here, wmax was set to 0.06.

cur_rate＝(1-credit_loss_rate)*tmp_rate，

w＝max(wmin,w/2)，

When detecting that the ECN _ ratio is larger than the threshold value target _ ECN _ ratio of the data volume marked with the ECN congestion mark, judging that the network is in a congestion state, and performing the following operations:

ECN_a＝(1-g)*ECN_a+g*ECN_ratio，

tmp_rate＝(1-(target_ECN_ratio+ECN_a)/2)*cur_rate，

ECN _ a is the intermediate calculated amount, and g is a set weight value, and the value of g is 0.5. In a shallow buffered switch environment, to ensure that the average queue length in the switch is around the ECN threshold, the targetECNratio is set to 0. An interface is provided to improve the feed forward compatibility of the DCN by defining a targetecratio.

Here Expresspass, a representative of the active congestion control protocol, is mixed with DCQCN, a representative of the reactive congestion control protocol, and evaluated.

The performance of CCRP is measured here from two perspectives: (i) fairness, (ii) convergence speed. All experiments were performed using an OMNeT + + simulator with a CCRP/ExpressPass to DCQCN ratio of 1: 1. fig. 1 shows fairness test results of the method and the existing network congestion control method. Fig. 2 shows the convergence test results of the present method and the existing network congestion control method.

First, the utilization is measured here, since one of the benefits of using ECN is high utilization. The Expresspass must provide 5% of the link bandwidth to transmit the credits, so the utilization of CCRP and Expresspass is close to 95%. The performance of the CCRP, Expresspass and DCQCN are compared in the same multi-protocol network environment, and the results show that the three protocols can fully utilize the network bandwidth. The utilization of a network where credit-based and ECN-based protocols coexist is 95%, while the utilization of ECN-based protocols alone approaches 100%.

CCRP aims at optimizing feedback control algorithms and reducing the aggressiveness of credit-based active protocols so that they can coexist with other traditional "passive" traffic in the network. Fairness is therefore the most important indicator in the evaluation here. Here mainly two experiments are done to measure the fairness of CCRP in multi-protocol networks. Here, the average bandwidth of each flow in a 100 ms interval is calculated by using the fairness index in, and the result is shown in part (a) of fig. 1. Since credit-based protocols are aggressive, fairness between Expresspass and DCQCN is poor. Furthermore, Expresspass will suffer from packet loss as more concurrent traffic is injected into the network, and fairness will be further degraded. In contrast, as shown in part (a) of FIG. 1, CCRP performs much better than Expresspass. This improvement is attributed here to ECN-based feedback control in CCRP. As shown in parts (b) and (c) of fig. 1, fairness between two flows is evaluated by a ratio of CCRP/Expresspass to DCQCN. This metric only focuses on unfairness between different flows, ignoring internal unfairness caused by packet loss. The results also show that fairness can be ensured when CCRP coexists with other "reactive" traffic.

When CCRP/ExpressPass and DCQCN coexist in the network, a series of experiments were performed to simulate their convergence. Specifically, there are four types of hosts (a to D). The type a computer (running ExpressPass or CCRP, sender) is connected to the type C computer (running ExpressPass or CCRP feedback control, receiver), and the type B computer (running DCQCN, sender) is connected to the type D computer (running DCQCN, receiver) through a 10Gbps link through an ECN-capable switch. A to C and B to D pass through the same path. The switch creates two connections to get two large flows from senders a and D at the same time. Since ω max is one of the most important variables of the ECN-based feedback control algorithm in CCRP, ω max is set to 0.03-0.07 here in the experiments herein.

As can be seen from the sections (b), (c), (d) and (e) of FIG. 2, when ω max is between 0.04 and 0.07, CCRP can greatly reduce the aggressiveness of Expresspass and perfectly converge with DCQCN. As shown in part (e) of fig. 2, when the attack factor is set to 0.07, the convergence speed is only 100 msec slower than the DCQCN. In contrast, section (f) of fig. 2 shows that ExpressPass with DCQCN cannot achieve convergence in a multi-protocol network. However, when ω max is too small (ω max ═ 0.03), as shown in part (a) of fig. 2, although CCRP may also help DCQCN to preempt bandwidth, the result of convergence is not ideal.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data center network congestion control method based on credit and reaction types is characterized by comprising the following steps:

the size of the credit packet used is set to 84B, 5% of the network link capacity is used for credit information transmission, and the remaining 95% of the network link capacity is used for data packet information transmission; at the switch, it separates the data path and the credit path on a physical channel; the credit packet data is transmitted by using a credit channel, and the normal packet data is transmitted by using a data channel; when a sending end needs to send data to a receiving end, a credit request is sent to the receiving end through a credit channel, and the credit request contains the size of a data stream to be sent and source address information; the receiving terminal sends a credit packet to the sending terminal after receiving the credit request, the sending terminal sends data according to the received credit packet, and one credit packet can only schedule one data packet; when the network is not detected to be in a congestion state, a credit feedback control method based on an active congestion control protocol is adopted, namely, data packets are sent as much as possible within a range allowed by link capacity; when the network is detected to be in a congestion state, a feedback control method based on an explicit congestion control ECN is adopted to enter a low sending rate mode: the reasonable ECN threshold value is deployed on the switch, when other inactive flows exist in the network, the length of a data queue in the switch exceeds the ECN threshold value, the switch marks the data packet with the ECN, and after a receiving end receives the data packet with the ECN mark, the receiving end immediately and automatically reduces the sending rate; when the receiving end does not receive the data packet with the ECN mark any more in the round trip time RTT of a plurality of data packets, the receiving end sends more credit information;

when the data flow adopting the active protocol coexists with the data flow adopting the reactive protocol in the network and the queue length of the switch exceeds a congestion threshold value, marking the switch with a congested ECN mark on a data packet; realizing a feedback congestion control method based on ECN at a receiving end; when the data packet reaches the receiving end, the receiving end detects the congestion by detecting whether the received data packet is marked with a congestion experience code point; ECN _ ratio is the ratio of the number of packets marked ECN to the number of all packets; and when the ECN _ ratio value is larger than 0, judging that the data link is in a congestion state, otherwise, judging that the data link is in a normal state.

2. The credit-and-reactive-based data center network congestion control method according to claim 1, wherein the steps comprise:

detecting the network congestion state, namely detecting the network congestion of a data queue by using ECN (equal cost network), and using the length of an instantaneous data queue as a judgment basis for marking the ECN; specifically, two queue length thresholds Kmax and Kmin are set in a data queue of the switch, the value of Kmin is set to be 5KB, and the value of Kmax is set to be 200 KB; if the length value of the data queue in the switch is below Kmin, the data is taken as common data to carry out forwarding operation, namely the probability of carrying out congestion marking is 0; if the length of the data queue in the switch is above Kmax, the switch considers that congestion occurs in the network, and the data packet is marked with a congestion mark, namely the probability that the data packet is marked to be congested is 1; if the length of a data queue in the switch is greater than Kmin and less than Kmax, performing analog calculation on the probability of the marked congestion of the data packet, and performing congestion marking on the data packet according to the calculation result probability;

3. The credit-and-reactive-based data center network congestion control method according to claim 1, wherein the steps comprise:

the congestion marking of the data packet is specifically realized by modifying a numerical value of data of an IP header of the data packet, wherein the data of the IP header of the data packet comprises a TOS (transmitter optical system) domain for representing differential service, and two idle numerical bits in the TOS domain are used for performing congestion marking; when the sending host does not support the ECN function, the two remaining value bits in the data packet are marked as 00; when the sending host supports the ECN function, the two remaining value bits in the data packet are marked as 01 or 10; when the packet experiences congestion, the two remaining bits of the value in the packet are marked 11.

4. The credit-and-reactive-based datacenter network congestion control method of claim 1,

when the network is judged to be in a normal state, namely when no data packet is marked with an ECN mark, an active congestion control protocol method is used for scheduling the data packet through a credit packet; during the first few RTTs, the network tends to send data traffic at the link rate, and after a plurality of RTTs, if any data packet with the ECN congestion flag is not obtained, it is determined that there is no other data traffic in the network using the reactive protocol; in this stage, the packet loss rate of the credit information is detected at the same time, when the packet loss rate of the credit information is smaller than the set credit packet loss threshold value target _ loss, the network enters a speed-up stage, and the corresponding credit sending rate is calculated by adopting the following method:

w＝(w+wmax)/2，

cur_rate＝(1-w)*tmp_rate+w*max_rate，

wherein, w is a calculation factor, cur _ rate is the current credit sending rate, tmp _ rate is the last round credit sending rate, max _ rate is the maximum sending rate of credit, and target _ loss is the set credit packet loss threshold, and in order to reduce the aggressiveness at the initial stage of deployment, the target _ loss value is set to 0; the value range of wmax is more than or equal to 0.04 and less than or equal to 0.07;

cur_rate＝(1-credit_loss_rate)*tmp_rate，

w＝max(wmin,w/2)，

max () represents the maximum value of two variables in brackets, and credit _ loss _ rate is the actual packet loss rate of the credit packet;

ECN_a＝(1-g)*ECN_a+g*ECN_ratio，

tmp_rate＝(1-(target_ECN_ratio+ECN_a)/2)*cur_rate，

wherein ECN _ a is an intermediate calculated quantity, g is a set weight value, and the value of g is 0.5; in a shallow buffered switch environment, to ensure that the average queue length in the switch is around the ECN threshold, the target ECN ratio is set to 0.