CN108683602B

CN108683602B - Data center network load balancing method

Info

Publication number: CN108683602B
Application number: CN201810772379.6A
Authority: CN
Inventors: 张弘; 张骏雪
Original assignee: Shenzhen Zhixing Technology Co Ltd
Current assignee: Shenzhen Zhixing Technology Co Ltd
Priority date: 2018-07-13
Filing date: 2018-07-13
Publication date: 2022-05-13
Anticipated expiration: 2038-07-13
Also published as: CN108683602A

Abstract

The invention provides a method for balancing network load of a data center, which evaluates path states (mainly path congestion and faults) through comprehensive perception and performs targeted load balancing based on the path states, namely, the path congestion and fault information obtained through perception is utilized, and timely and cautious routing/rerouting is performed by taking a packet as the minimum switchable granularity under the guidance of the path congestion and the fault information and the flow state information, so that the method can be used for solving the challenges brought to the load balancing by uncertain factors such as flow dynamic transmission, topological asymmetry, equipment faults and the like in the network of the data center. The invention also provides a load balancing system deployed at the host terminal of the data center and based on the method, and a data center network system composed of the host terminals for deploying the system.

Description

Data center network load balancing method

Technical Field

The invention relates to the technical field of flow load balancing in a data center network; in particular to a method for balancing the load of a data center network, a load balancing system based on the method and a data center network system.

Background

A data center network (Datacenter network) which refers to a network applied in a data center; the data center network connects a certain number of servers through network equipment such as switches and routers to form a server network with high bandwidth, high reliability and load balance, and can provide services such as calculation and storage for the outside.

To meet the requirements of large scale, high scalability and high robustness, a Clos network is usually adopted in the data center network, which is a Multi-root tree structure (Fat tree) such as Fat tree topology (Fat tree) and Leaf-spine topology (Leaf-spine). As shown in fig. 1, the above-described structure can provide multiple paths for each pair of host terminals (End host) in the network; therefore, it is highly necessary to achieve strict load balancing in the above-described network.

In fact, however, there are various uncertain factors such as Traffic dynamics (Traffic dynamics), Topology asymmetry (Topology asymmetry), and equipment Failures (failurs) in the data center network transmission process. Wherein traffic is dynamic during data center network transmission, so congestion may occur anywhere in the network (fig. 2); secondly, topology asymmetry can be generated by the introduction of heterogeneous devices (heterogeneous devices) and Link cut (Link cut) caused by faults (fig. 3); in addition, switch failures also occur frequently in data center networks (fig. 4), and typical switch failures include data forwarding black holes (Packet holes) and Random Packet drops (fig. 5).

ECMP is a typical load balancing scheme in data center networks. The implementation mode is that different equivalent paths are selected for routing according to equal probability according to different five-tuple hashes for different flows (flows). ECMP employs a random equivalent routing strategy, which is blindly inefficient in situations such as congestion of a portion of the path, asymmetric topology of the network, etc.

Therefore, there is a need to design an efficient and appropriate traffic load balancing scheme to meet the needs of a data center network. Under the influence of the uncertain factors, an ideal load balancing scheme should effectively sense the uncertain factors, namely, sense congestion or faults on paths, and make a response according to the sensed result, so as to properly distribute traffic loads to parallel paths as much as possible for load balancing.

In some existing data center network load balancing schemes, such as ECMP and other schemes mentioned above, such as RPS, DRB, Presto, ignore perceived path congestion, and perform poorly in asymmetric networks; the CONGA, HULA and DRILL realize the sensing of path congestion under the support of specific hardware (switches with related functions); but only based on the way-ECN that implements the path-congestion aware function on the host side, which achieves limited path Visibility (Visibility). Worse, most existing traffic balancing schemes cannot sense path failure, so that solving forwarding black holes and random packet loss cannot be mentioned.

Furthermore, existing solutions provide different ways to cope with load balancing, such as flow segment rerouting (flowet switching), strong rerouting (videorouting), etc. Wherein, stream segment rerouting is only rerouting when a stream segment occurs (as shown in fig. 6a, 6b, and 6 c), obviously, cannot be handled in time (Timely); strong rerouting exacerbates Packet reordering and causes Congestion mismatch. CN104767826A also provides a load balancing method under the guidance of end-based path sensing, which is used to adaptively balance load according to the guidance after sensing path congestion and failure. However, objectively, the accuracy of path sensing is not discussed for a while, only adaptive load balancing is used, which is actually to regulate and control the message sending speed, which is a passive countermeasure, and only can avoid further aggravation of the load pressure of the congested path, but cannot fundamentally solve the problems caused by the occurring congestion and faults.

Disclosure of Invention

In view of this, the present invention provides a method for load balancing of a data center network, so as to address challenges brought to load balancing by uncertain factors such as dynamic traffic transmission, topology asymmetry, and device failure in the data center network. In addition, the invention also provides a load balancing system which is deployed at the host terminal of the data center and is based on the method, and a data center network system which is composed of the host terminals for deploying the system.

In one aspect, the present invention provides a method for balancing a load of a data center network. The method comprises the steps that a flow state table and a path state table are maintained at a source host, the flow state table records flow state information sent from the source host, and the path state table records the congestion degree and the fault condition of all paths which can be reached from the source host; the method comprises Comprehensive Sensing (Comprehensive Sensing) and targeted load balancing of the path state:

the comprehensive perception comprises the following steps: updating the obtained path congestion and fault information into the path state table by evaluating the path state (including the congestion degree and the fault) by means of the transport layer information; the transport layer information includes transport layer signals (e.g., RTT, ECN, ACK signal, etc.), and events (e.g., Retransmission, Timeout (Timeout), etc.).

The targeted load balancing comprises the following steps: performing Routing/Rerouting (Routing) in time and Cautious (Cautious) with a Packet (Packet) as a minimum adjustable granularity (the minimum switch granularity) using the path congestion, the failure information, and the flow state information in the path state table and the flow state table.

In combination with the first aspect, in the method thereof,

preferably, the evaluating the path state by the transport layer information includes evaluating the congestion degree of the path by combining RTT and ECN signals; by setting RTT and ECN thresholds, the collected RTT and ECN values under each path are respectively compared with the RTT and ECN values so as to evaluate the congestion degree of the path.

Further, preferably, to accurately determine the congestion degree of the path, two RTT thresholds, i.e., a high threshold and a low threshold, are set, and the path is divided into a Congested path (i.e., in a Congested state), a Good path (i.e., in an underutilized state), and a Gray path (i.e., in a state between the Congested path and the Good path, including a moderate load) according to the utilization degree.

Preferably, the evaluating the path state by the transport layer information includes evaluating a path failure by using the transport layer information including transport layer event information (such as retransmission, timeout event, etc.); in a preferred implementation manner, the timeout times and retransmission values occurring in each path are compared one by setting a threshold value of the number of timeout events and a threshold value of the frequency of retransmission events in a single path, and the fault condition of each path is evaluated in combination with the Ack signal and the congestion degree of each path.

Preferably, the comprehensive sensing further comprises periodic Active Probing (Active Probing) with low cost to improve the visibility of the path.

Preferably, the guiding of the timely and cautious routing/rerouting with the packet as the minimum adjustable granularity by using the path state and the flow sending state includes:

when a packet is sent from a source host end, detecting a flow corresponding to the packet, and if the packet belongs to a new flow (namely the packet is the first data packet when the flow is sent), selecting a fault-free path with the lowest load to send according to the path state table; if the packet belongs to a flow which is subject to overtime, path congestion or path failure, the rerouting loss is evaluated according to the path congestion and the failure information in the path state table and the state information of the flow in the flow state table, and whether the rerouting and the rerouting path are determined under the strategy of striving for the maximum profit.

Further, preferably, the stream transmission status information includes a transmission speed of the stream, a size of the unsent part, and whether the stream is subjected to timeout; one common way to evaluate the size of the unsent part of the stream is to evaluate the size of the unsent part of the stream according to the size of the sent part of the stream; whether the flow is subjected to timeout is generally determined according to whether the time interval between the two adjacent packet Ack signals received from the flow exceeds a preset time interval.

On the other hand, in combination with the load balancing method of the first aspect, the present invention provides a load balancing system deployed in a data center network and based on the method. The system comprises:

a Sensing Module and a Routing Module; wherein the content of the first and second substances,

the path sensing module is used for comprehensively sensing the path state (including the congestion degree and the fault); said integrated sensing, i.e. as described for the integrated sensing in the method of the first aspect; the path congestion and fault information obtained by the comprehensive perception is updated to a path state table maintained at the source host end;

the routing module is used for executing timely and cautious routing/rerouting by taking a packet as the minimum adjustable granularity; the method utilizes the path congestion and fault information in the path state table and combines the flow state information in the flow state table maintained at the source host end to guide the timely and cautious routing/rerouting; the routing/rerouting in time and cautious with the path congestion, fault information and flow state information at the packet minimum adjustable granularity is as described in the method of the first aspect.

With reference to the second aspect, in the system of the present invention, preferably, the path sensing module includes an active detection sub-module, configured to perform active detection in the integrated sensing.

In another aspect, in combination with the second aspect, the present invention provides a data center network system, which includes a plurality of server host terminals that deploy the load balancing system according to the second aspect.

Compared with the prior art, the technical scheme provided by the invention has a plurality of beneficial effects, such as:

comprehensively, path congestion and faults are effectively detected through comprehensive perception, and a load balancing strategy is guided according to the path congestion and the faults;

the timeliness is established in the routing/rerouting with the packet as the minimum adjustable granularity, so that uncertain factors in the data center network can be quickly responded, and timely response can be made;

third, transmission friendliness, which also uses packets as the minimum adjustable granularity, through a prudent routing/rerouting strategy, well limits the adverse effects of packet disorder, congestion mismatch and other problems on traffic transmission;

and fourthly, deployability, which can be implemented on commercial hardware in the current data center environment without specific hardware support (such as CONGA) or modification of network system underlying code and the like.

Drawings

To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings related to a part of the embodiments of the present invention or the description in the prior art will be briefly introduced below.

FIG. 1 illustrates a data center network architecture (leaf-spine topology) diagram in the prior art;

FIG. 2 discloses a deficiency in the susceptibility of the data center network of FIG. 1 to congestion;

FIG. 3 reveals the cause of uncertainty in the data center network of FIG. 1, which is topology asymmetry;

FIG. 4 illustrates a data center network with switch failure;

fig. 5 illustrates a random packet loss problem under a switch failure in the prior art;

fig. 6a shows the situation when multiple streams are sent simultaneously between two points, fig. 6b shows the rerouting under the ideal condition, and fig. 6c shows the rerouting under the flow segmentation adjustment technique;

fig. 7 illustrates a network traffic load balancing system deployed on a host terminal according to an embodiment of the present invention;

fig. 8 shows a preferred implementation of the system provided in fig. 7, namely a network traffic load balancing system with active detection.

Detailed Description

The technical solution in the embodiments of the present invention is clearly and completely described below with reference to the drawings of the embodiments of the present invention. It is to be understood that the described embodiments are merely exemplary of a portion of the invention and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

Fig. 1 is a schematic diagram of a Clos network having a leaf-spine topology that provides multiple paths for each pair of host terminals in the network. Therefore, it is necessary to strictly balance the traffic load by using the network to transmit data in parallel. However, when the data center is in operation, and it is not easy to avoid the failure of individual switches, routers, and other devices in the network system, the traffic congestion caused by the routing algorithm cannot be avoided at all under the condition of multiple paths in the data transmission process.

The embodiment of the invention is applied to the field of data center network load balancing, and realizes efficient transmission of data center network information flow as far as possible.

Some embodiments of the present invention provide a method for balancing network load of a data center, where a flow state table and a path state table are maintained at a source host, where the flow state table records flow state information sent from the source host, and the path state table records congestion degrees and failure conditions of all paths reachable from the source host; the information of the flow state table and the path state table can set a life cycle according to needs, wherein each flow state information in the flow state table takes the period from the beginning to the completion of the sending as the period; the congestion information of each path in the path state table decays along with time, that is, once the congestion information of a certain path is not updated after exceeding the preset time (set according to the actual situation), the congestion degree of the path state table gradually decays;

as a key point of the technical scheme, the method comprises the steps of comprehensively sensing the path state and carrying out targeted load balancing:

the comprehensive perception comprises the following steps: the path status (i.e. congestion degree and failure) is evaluated by means of transport layer information, and the obtained path congestion and failure information is updated into the path status table, where the transport layer information typically includes transport layer signals (e.g. RTT, ECN, ACK signal, etc.), events (e.g. retransmission, timeout, etc.).

The targeted load balancing comprises the following steps: utilizing path congestion, failure information, and flow state information in the path state table and flow state table to perform timely and cautious routing/rerouting with packets as a minimum adjustable granularity.

In the above embodiments, some of the embodiments provide a preferred implementation manner, that when evaluating the path state with the transport layer information, the method includes evaluating the congestion degree of the path by using both RTT and ECN signals. In some related art schemes, RTT is typically taken to directly evaluate end-to-end path congestion level. However, although RTT is Informative, accurate measurement of RTT is difficult to achieve without advanced NIC hardware support. Therefore, a high RTT value does not necessarily indicate path congestion (for example, the delay of the network stack of the host terminal also increases the RTT value), but a small RTT value can clearly indicate that the path is not fully utilized, which is more informative to us. ECN is commonly used to capture single hop (hop) congestion, which is implemented with commercial switch support and is widely used in congestion control algorithms. However, the ECN signal can only reflect the congestion situation of the most congested hop in the path. While under a highly loaded network, congestion may accumulate from multiple hops. Therefore, a low ECN value does not necessarily indicate that no congestion has occurred, especially if the ECN sampling is insufficient.

Further, some of the above embodiments also provide a more preferable implementation manner, that is, two RTT thresholds are set, and as shown in the following table, according to the estimated utilization degree, the RTT thresholds are divided into a congested path, a gray path, and a good path:

in any of the above embodiments, some of the embodiments provide a preferred implementation manner, that when evaluating the path state with the transport layer information, the method includes evaluating the path failure with the transport layer event information retransmission, the timeout, and the transport layer signal Ack; by setting a threshold value of the number of overtime events and a threshold value of the frequency of retransmission events under a single path, the overtime times and the retransmission values of all paths are compared one by one, preset conditions (the number of overtime times and the frequency of retransmission events exceed the threshold values and Ack packets are received) are met, and when path congestion is eliminated, the path is considered to be in fault.

In any of the above embodiments, some of the embodiments provide a preferred implementation in which the integrated sensing further includes periodic, low-overhead active probing. The detection in some existing schemes adopts full path detection, but in fact, we do not need to detect all paths to grasp their congestion degree, but only randomly detect a small part of them, which also helps to improve load balancing paths and can greatly reduce overhead. Therefore, in active probing, several (e.g., two) paths selected at random can be probed. In addition, in the above embodiment, in order to improve the stability of detection and increase the possibility of finding an unused path, it is also possible to additionally detect a path with the best state observed before, at the same time as the random routing detection.

In any of the above embodiments, some of the embodiments provide a preferred implementation manner, that is, the method guides timely and cautious routing/rerouting with packets as the minimum adjustable granularity by using the path status and the flow sending status, and includes the following specific processes:

when a data packet is sent from a source host end, detecting a stream corresponding to the packet;

if the packet belongs to a new flow (namely the packet is the first data packet when the flow is sent), selecting a fault-free path with the lowest load to send according to the path state table;

if the packet belongs to a flow which is subject to overtime, path congestion or path failure, the rerouting loss is evaluated according to the path congestion and the failure information in the path state table and the state information of the flow in the flow state table, and whether the rerouting and the rerouting path are determined under the strategy of striving for the maximum profit.

Here, it is worth noting that, in the existing and the disclosed solutions for dealing with uncertainty of network congestion, failure and the like of the data center, the packet as the minimum adjustable granularity provided by the present invention is obviously a more timely and active solution with respect to stream segmentation adjustment; when congestion occurs, compared with strong rerouting, the Cautious rerouting established on the profit assessment has incomparable advantages. This is because when a flow is congested, it is speculatively rerouted and adjusted to the wrong path (e.g., a more congested or failed path), not only does it fail to solve the problem, but it also causes increased overhead and reduced revenue, more likely leading to further aggravation of the problem. Even with knowledge of the path status (e.g., the evaluation method provided by CN 104767826A), rerouting to a path with lower load is not all the most profitable options, such as one with less flow remaining, and the revenue is likely to be negative no matter how intelligent rerouting is performed. This phenomenon is particularly apparent when smaller flows are rerouted.

Further, some of the above embodiments also provide a more preferable implementation manner, that is, the stream transmission state information includes a transmission speed of the stream, a size of an unsent part, and a stream timeout transmission speed threshold; one common way to evaluate the size of the unsent part of the stream is to evaluate the size of the unsent part of the stream according to the size of the sent part of the stream; therefore, for a smaller flow, a flow with a smaller sent part, or a flow with a sending speed greater than a preset value, rerouting processing may not be performed. Whether the flow is subjected to timeout is generally determined according to whether the time interval between the two adjacent packet Ack signals received from the flow exceeds a preset time interval.

Fig. 7 is a schematic diagram of a data center network load balancing system according to another embodiment of the present invention. As shown, the system includes:

a path awareness module and a routing module; wherein the content of the first and second substances,

the path sensing module is used for comprehensively sensing the path state (including the congestion degree and the fault); said integrated sensing, i.e. as described in the method of the first aspect, its main functions are to sense congestion and to sense faults; the path congestion and fault information obtained by the comprehensive perception are inserted into a path state table which is also maintained at the source host end;

the routing module is used for executing timely and cautious routing/rerouting by taking a packet as the minimum adjustable granularity; the method utilizes the path congestion and fault information in the path state table and combines with the flow state information in the flow state table maintained at the source host end to guide the timely and cautious routing/rerouting; the routing/rerouting in time and cautious with the path congestion, fault information and flow state information at the packet minimum adjustable granularity is as described in the method of the first aspect.

Fig. 8 is a preferred implementation provided in some of the above embodiments, wherein the path sensing module includes active probing for performing active probing.

In addition, another partial embodiment of the present invention provides a data center network system, which includes a plurality of server host terminals that deploy the load balancing system described in any of the above embodiments.

The above description is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto.

Claims

1. A method for balancing network load of data center is characterized in that,

maintaining a flow state table and a path state table at a source host end, wherein the flow state table records flow state information sent from the source host, and the path state table records the congestion degree and the fault condition of all paths which can be reached from the source host;

the method comprises comprehensive perception of path states and targeted load balancing;

the comprehensive perception comprises the steps of evaluating the path state by means of transmission layer information, wherein the state comprises path congestion and failure; updating the obtained path congestion and fault information into the path state table; the transmission layer information comprises transmission layer signals and time;

the targeted load balancing includes performing timely and cautious routing/rerouting with packets as a minimum adjustable granularity using path congestion, failure information, and flow state information in the path state table and flow state table;

the information of the flow state table and the path state table is set with a life cycle, wherein the congestion information of each path in the path state table is not updated after exceeding the preset time, and the congestion degree of each path is gradually decayed;

the guiding of timely and cautious routing/rerouting with packets as minimum adjustable granularity by utilizing the path state and the flow sending state comprises the following steps: when a packet is sent from a source host end, detecting a flow corresponding to the packet, and if the packet belongs to a new flow, selecting a fault-free path with the lowest load to send according to the path state table; if the packet belongs to a flow which is subjected to overtime, path congestion or path failure, evaluating the rerouting loss and determining whether to reroute or not and a rerouted path according to the path congestion and failure information in the path state table and in combination with the state information of the flow in the flow state table; the flow state information comprises the sending speed of the flow, the size of the part which is not sent and whether the flow is overtime, and the rerouting processing is not carried out on the flow with smaller size, the flow with smaller part which is not sent or the flow with the sending speed greater than the preset value;

the method comprises the steps of evaluating the state of a path by using transmission layer information, wherein the path fault is evaluated by using transmission layer event information retransmission, overtime and a transmission layer signal Ack; setting a threshold value of the number of overtime events and a threshold value of the frequency of retransmission events under a single path, comparing the overtime times and the retransmission values of all paths one by one, and considering that the path fails when the following preset conditions are met and path congestion is eliminated: the number of timeout events, the frequency of retransmission events exceeding a threshold value, and no Ack packet received.

2. The load balancing method according to claim 1,

the method for evaluating the path state by the transmission layer information comprises the steps of evaluating the congestion degree of a path by combining RTT and ECN signals; by setting RTT and ECN thresholds, RTT and ECN values of each path are respectively compared with the RTT and ECN thresholds so as to evaluate the congestion degree of the path.

3. The load balancing method according to claim 2,

setting high and low RTT thresholds, and dividing the path into a congestion path, a good path and a gray path.

4. The load balancing method according to claim 1,

the comprehensive sensing also comprises periodic low-open small active detection; the active detection comprises randomly selecting a plurality of paths for detection.

5. The load balancing method according to claim 4,

the active detection also comprises the step of additionally detecting the path with the best observed state at the same time of randomly selecting the path detection.

6. A data center network load balancing system is characterized in that,

the system comprises a path sensing module and a routing module; wherein the content of the first and second substances,

the path sensing module is used for sensing path states comprehensively, wherein the states comprise path congestion and faults, and the comprehensive sensing is the comprehensive sensing in the load balancing method according to any one of claims 1 to 5; the path congestion and fault information obtained by the comprehensive perception is inserted into a path state table maintained at a source host end;

the routing module is used for executing timely and cautious routing/rerouting by taking a packet as the minimum adjustable granularity; the routing module guides the timely and cautious routing/rerouting by utilizing the path congestion and fault information in the path state table and combining with the flow state information in the flow state table maintained at the source host end; the routing/rerouting utilizing the path congestion, the failure information and the flow state information with the packets as the minimum adjustable granularity is timely and cautious, i.e. the routing/rerouting utilizing the path congestion, the failure information and the flow state information with the packets as the minimum adjustable granularity is timely and cautious in the load balancing method as claimed in any one of claims 1 to 5.

7. A data center network system, characterized in that,

the data center network system comprising a plurality of server host terminals deploying the load balancing system of claim 6.