JP7177548B2 - Load Balancing Method Based on NAT in DPDK Environment - Google Patents

Description

The present invention belongs to the field of network communication, and specifically relates to a load balancing method based on NAT (Network Address Translation) in a DPDK (Data Plane Development Kit) environment.

Today, with the explosive increase in computer users, network traffic is getting bigger and bigger, and the load capacity of servers is becoming a big issue. Even the best performing single server today can't keep up with today's high traffic demands. Therefore, it is very important to use the idea of clustering to achieve load balancing. Traditional NAT-based load balancing, placed on the traditional TCP/IP protocol stack, is undoubtedly time consuming due to frequent system interruptions and continuous scheduling of contexts. Therefore, in the conventional use of NAT to achieve load balancing, the scheduler itself becomes the bottleneck of load balancing. Due to scheduler performance issues, typically only about 10 servers can be scheduled. This is applicable in most cases, but now that network traffic is exploding, especially with the maturity of DDOS technology, attacking hundreds of GB/S level traffic is no longer a problem. . In such a background, the conventional load balancing based on NAT is obviously not applicable, so there is an urgent need to find a load balancing technology that can withstand large-scale traffic.

In the case of traditional NAT-based load balancing, for external requests, after interception by the scheduler, we generally use the method of polling, randomly finding the address of an internal server from the address pool to replace the destination address, and the source Keep the address as is. It is then sent to an internal server for processing. Internal and external communication is a reverse process, in which the scheduler replaces the source address of the data packet sent from the server with the all-stations address possessed by the scheduler, holds the destination address as it is, and transmits.

An advantage of conventional NAT-based load balancing is that the server can run any operating system that supports TCP/IP. Therefore, it is more flexible and saves public network IP addresses because it only puts one IP in the scheduler and the servers can use their own IP addresses.

However, the scheduler needs to perform frequent search and replace operations on data packets. This workload will be much less than the detailed processing of data packets by the server, but as the workload increases, the scheduler will not be able to sustain such high frequency work. In general, when the number of server nodes increases to 20, the scheduler itself can become the new bottleneck of the system.

Also, in the case of conventional NAT-based load balancing, the polling method is commonly considered. That is, each server has an equal chance of being prioritized by the scheduler, and to some extent the use of each server can be balanced. However, in this situation, differences in the performance of the servers themselves are ignored. For example, if server A's load capacity is ten times greater than server B's, such a polling method will reach 1/10th of server A's capped load even when server B's load capped. This is unfair to server B and will definitely degrade the overall system performance.

In response to the above problems in the prior art, which cannot handle large-scale traffic and the overall system performance is degraded due to unfair scheduling, the present invention provides a NAT-based load balancing method in a DPDK environment. be. In this method, the DPDK module is used as the processing module of the scheduler, and the fast processing capability of the DPDK for data packets combines the status record table of the NAT module to schedule and forward the data packets. This overcomes the performance limitations of the scheduler itself, improves load balancing ceilings, and provides balanced transfers. Specific technical proposals are as follows.

A NAT-based load balancing method in a DPDK environment, comprising:
The method is implemented by a scheduler including a DPDK module and a NAT module,
step S1 of initializing the DPDK module and the NAT module, locating the send/receive queues of the scheduler and the corresponding ports, and setting up the corresponding NAT rule table and status record table;
step S2 of recording the source/destination IP and port of the data packet received by the scheduler by the DPDK module and performing hash calculation based on the source/destination IP and port to obtain a hash value of the data packet;
Based on the hash value, traverse the NAT rule table, search whether the NAT rule table contains the IP and port information of the internal server corresponding to the hash value, and if so, the NAT step S3 of replacing the source and destination IPs based on a rule table and sending the data packet to the corresponding server, if not present, performing step S4;
If the IP and port information of the internal server corresponding to the hash value does not exist in the NAT, the server with the largest residual load is searched for in the status record table, and the data packet is sent to the server with the largest residual load. a step S4 of recording communication information between the server with the largest remaining load and the client side in the NAT rule table and updating the NAT rule table;
including.

Furthermore, the step S2 further includes:
determining whether the data packet is sent from the client side to the server or from the server to the client side, if the data packet is sent from the client side to the server, performing steps S3-S4; , perform the following steps,
Traversing the NAT rule table based on the hash value, retrieving source/destination IP and port information corresponding to the hash value in the NAT rule table, replacing the source/destination IP and port, and Send the packet to the corresponding client side.

Furthermore, the step S1 further includes:
The servers in the status record table are graded according to a predetermined rule based on the remaining load.

Further, the DPDK module receives the data packet through the rte_eth_tx_burst( ) function.

Compared with the prior art, the NAT-based load balancing method in the DPDK environment of the present invention has the following beneficial effects.
(1) It has adaptability for large-scale traffic. By using DPDK instead of traditional protocol stacks to process data packets faster, overcoming the scheduler limitations of traditional NAT-based load balancing, the scheduler can schedule more servers. , to achieve load balancing for large-scale traffic. (2) It has good balance and can improve the performance of the whole server cluster. By using the values obtained by comparing the residual load and performance of the status record table, the server with the highest true residual load can be found quickly, and the number of servers can be reduced compared to the conventional method of polling each server. ensure load balancing by accounting for performance differences between (3) It has extensibility. Software can improve the processing performance of the scheduler and realize load balancing for large-scale traffic. Compared to hardware systems, there are many general-purpose API interfaces that provide better support for subsequent development, and software can also be further developed according to actual needs.

Fig. 4 is a flow chart for realizing one-time communication between the client side and the server according to the method of the present invention; FIG. 4 is a structural schematic diagram of a NAT rule table according to an embodiment of the present invention; FIG. 4 is a schematic diagram of a model structure for completing the method of an embodiment of the present invention;

In order to enable those skilled in the art to better understand the present invention, the following clearly and completely describes the embodiments of the present invention with reference to the drawings.

Embodiments of the present invention provide a NAT-based load balancing method in a DPDK environment. As shown in FIG. 3, a DPDK module and a NAT module are provided within the scheduler of the present invention. The method of the present invention is implemented by said scheduler. FIG. 1 is a flow chart for realizing one-time communication between the client side and the server according to the method of the present invention. As can be seen from the figure, the method of the present invention includes the following steps.

S1: Initialize the DPDK module and the NAT module, configure the scheduler's transmit and receive queues and the corresponding ports, and set up the corresponding NAT rule table and status record table.

In the embodiment of the present invention, the DPDK module implements data packet transmission/reception processing, and the NAT module maintains the NAT rule table and the status record table. A NAT rule table is a hash table. As shown in Figure 2, each key value corresponds to two values. One value replaces the destination IP and port when receiving and is used to send to the internal server, and the other value replaces the source address with the IP address and port of the external scheduler when sending before sending to the client side. used for The NAT rule table is created at initialization and updated when the scheduler receives data packets. Each time, it is hashed based on the IP and port information of the client side, the IP and port information of the internal server is placed in the first position, and the external IP and port information used by the scheduler is placed in the second position. When the scheduler searches, it first determines whether this data packet is to be received by the external client side or transmitted by the internal server according to the source end port address of the data packet, and then Determine whether to use the first value of the key instead of the destination IP, port or the second value instead of the source IP port by hashing based on the IP and port information of .

The status record table of the present invention is used to record specific situations of servers with residual load. In the present invention, the inquiry period of the internal server of one NAT module is set. Specifically, it is set according to the actual situation. In contrast, the present invention does not limit and fix. Within one query cycle, the NAT module periodically queries the internal servers to retrieve the current load of the current internal servers, and combine with the performance of each internal server for comprehensive consideration. The overall consideration process is as follows. for t←0ton; state record[t]←server_load[t]*server_kind[t]*r, for server_kind=[S1, S2 . . . Sn]. The performance information of each server is recorded, the higher the number, the stronger the server's performance, and the current load server_load and server_kind are calculated to obtain the true residual load of the server. where r is a calibration constant that limits state_record between 1-100. The last obtained grade record is inserted into the status record table state_record. According to such a scheme, it is possible to prevent the deterioration of the performance of the entire system due to ignoring the difference in server performance in the conventional NAT scheme using polling. Preferably, in this embodiment, the internal server load status is graded into a total of 100 grades from 1-100. It should be noted that this is only a preferred embodiment of the present invention and does not limit the present invention.

S2: The DPDK module records the source/destination IP and port of the data packet received by the scheduler, and performs hash calculation based on the source/destination IP and port to obtain the hash value of the data packet.

In a specific embodiment of the invention, the DPDK module receives data packets via the rte_eth_tx_burst() function. The process implementation of the rte_eth_tx_burst() function is as follows.

Place the send and receive queues from rte_eth_dev_config(), and copy the configuration parameters to the data area of the facility by the memcpy(&dev->data->dev_conf, sizeof(dev->data->dev_conf)) function, (*dev-> Equipment information is acquired by dev_ops->dev_infos_get) (dev, &dev_info). At the same time, the present invention initializes the port for receive queues by the rte_eth_rx_queue_setup() function, populates 8 queues, and rxq=rte_zmalloc_socket("ethdev RX queue", sizeof(structixgbe_rx_queue), RTE_CACHE_LINE_SIZE, by the function RTE_CACHE_LINE_SIZE, socket) After allocating a queue structure, filling the structure, and completing disposition, the rte_eth_tx_burst( ) function receives the data packet and records the source and destination IP and port information of the corresponding data packet. The scheduler determines whether the received data packet was sent from the client side to the server or from the server to the client side. If the data packet is sent from the server to the client side, hash it based on the IP and port to obtain the corresponding hash value, traverse the NAT rule table based on the hash value, and Search the source/destination IP and port information corresponding to the hash value, replace the source/destination IP and port, and send the data packet to the corresponding client side through the DPDK module by the rte_eth_tx_burst( ) function. Otherwise, that is, if the data packet is sent from the client side to the server, similarly obtain the hash value of the data packet by performing hash calculation based on the source/destination IP and port, and obtain The following steps S3-S4 are executed based on the hash value.

S3: Based on the hash value, traverse the NAT rule table, search the NAT rule table to see if the IP and port information of the internal server corresponding to the hash value exists, and if so, add the information to the NAT rule table. replace the source/destination IP based on this, and send the data packet to the corresponding server; if not, execute step S4.

Specifically, after the rte_eth_tx_burst() function receives a data packet, it records the information of the data packet source and destination IP address port, the scheduler identifies this as the received data packet, the source port, Hash calculation is performed based on the IP information, and based on the hash value, it is searched whether or not the corresponding IP and port information of the internal server exists in the NAT rule table. If so, replace the source/destination IP based on the NAT rule table and send the data packet to the corresponding server. Otherwise, execute step S4.

S4: If the IP and port information of the internal server corresponding to the hash value does not exist in the NAT, search for the server with the largest residual load in the status record table, and send the data packet to the server with the largest residual load. Then, the communication information between the server with the maximum remaining load and the client side is recorded in the NAT rule table, and the NAT rule table is updated.

Specifically, if no corresponding information is found from the NAT rule table, this indicates that this is the first client-side request. In this case, the scheduler retrieves the highest record value from the status record table, retrieves the IP of the corresponding internal server, and then the scheduler replaces the destination address of this data packet with an internal Send to the server, record the server and client-side communication information with the highest remaining load in the NAT rule table, and update the NAT rule table for the next use.

The process of performing a single communication between the server and the client side according to the method of the present invention has been described above with reference to FIG. detailed description is omitted.

From the above, compared to the prior art, the method of the present invention speeds up the processing of data packets by using DPDK instead of the traditional protocol stack, and overcomes the scheduler limitations of traditional NAT-based load balancing. By overcoming it, the scheduler can schedule more servers, achieve load balancing in large-scale traffic, and be applicable to larger-scale traffic, making the field of application wider. By using the values obtained by comparing the residual load and performance of the status record table, the present invention can quickly find the server with the highest true residual load, which is different from the conventional method of polling each server. By taking into account the performance differences of the servers, the load distribution can be reliably achieved, the loads among the servers can be well balanced, and the overall performance of the server cluster can be improved. INDUSTRIAL APPLICABILITY The present invention can improve the processing performance of a scheduler by software, and can realize load balancing of large-scale traffic. Compared with the hardware system, the scheduler has good scalability because there are many general-purpose API interfaces that provide better support for subsequent development, and the software can be further developed according to actual needs.

The above descriptions are only preferred embodiments of the present invention and do not limit the protection scope of the present invention. Although the present invention is described in detail by the above embodiments, those skilled in the art can modify the technical means of the above embodiments or replace them with equivalents. Any equivalent structure obtained on the basis of the content of this specification and drawings and its use in related technical fields shall fall within the protection scope of the present invention.

Claims (4)

A NAT-based load balancing method in a DPDK environment, comprising:
The method is implemented by a scheduler including a DPDK module and a NAT module,
Initializing the DPDK module and the NAT module, populating the scheduler's send and receive queues and corresponding ports, setting up corresponding NAT rule tables and status recording tables for recording the status of servers with remaining load capacity. S1 and
step S2 of recording the source/destination IP and port of the data packet received by the scheduler by the DPDK module and performing hash calculation based on the source/destination IP and port to obtain a hash value of the data packet;
Based on the hash value, traverse the NAT rule table, search whether the NAT rule table contains the IP and port information of the internal server corresponding to the hash value, and if so, the NAT step S3 of replacing the source and destination IPs based on a rule table and sending the data packet to the corresponding server, if not present, performing step S4;
If the IP and port information of the internal server corresponding to the hash value does not exist in the NAT, the server with the largest remaining load capacity is searched for in the status record table, and the data packet is sent to the server with the largest remaining load capacity. a step S4 of recording the communication information between the server with the largest remaining load capacity and the client side in the NAT rule table and updating the NAT rule table;
including
The method, wherein the NAT module maintains the NAT rule table and the status record table, and periodically queries the internal server to retrieve the remaining load capacity of the internal server . The step S2 is
determining whether the data packet is sent from the client side to the server or from the server to the client side;
if the data packet is sent from the client side to the server, performing steps S3-S4; otherwise:
Traversing the NAT rule table based on the hash value, retrieving source/destination IP and port information corresponding to the hash value in the NAT rule table, replacing the source/destination IP and port, and 2. A method according to claim 1, characterized by performing the step of sending the packet to the corresponding client side. The step S1 further includes:
2. The method of claim 1, wherein the servers in the status record table are graded according to a predetermined rule based on their remaining load capacity. Method according to any one of claims 1 to 3, characterized in that in said DPDK module, said data packet is received by means of the rte_eth_tx_burst() function.

Images