CN112671667B - A forwarding rate limiting method and system for a virtual forwarding device - Google Patents

Abstract

The present invention provides a forwarding rate limiting method and system for a virtual forwarding device, including obtaining the average data packet size and flow number of a service provider network through measurement as traffic characteristics. The service provider network includes a virtual forwarding device and multiple tenants, using The traffic generation tool generates traffic that conforms to this characteristic. By forwarding the data packets of the traffic in the virtual forwarding device and measuring the CPU resources used for forwarding to achieve each bandwidth, the corresponding relationship between CPU resources and bandwidth is constructed; according to the basic bandwidth of each tenant Requirements and the corresponding relationship, allocate basic CPU resources to each tenant, and obtain the remaining CPU resources of the service provider network after satisfying the basic CPU resources, so that each tenant can re-allocate idle CPU resources according to the weight. The forwarded final CPU resource is the sum of the basic CPU resource and idle CPU resource allocated to it; the virtual forwarding device uses the final CPU resource to complete the forwarding of the data packet to its corresponding tenant.

Description

A forwarding rate limiting method and system for a virtual forwarding device

technical field

The invention relates to a speed limiting method, and proposes a speed limiting method and system with enhanced isolation under software forwarding.

Background technique

With the evolution of network equipment towards generalization and high cost performance, software forwarding has gradually replaced traditional hardware switches and routers, and is widely used in many scenarios such as cloud computing and network function virtualization (NFV). With the sharp increase of network traffic and the richness of traffic types, higher requirements are placed on the performance isolation of different traffic in these virtual forwarding devices.

The functions of software forwarding devices are similar to hardware forwarding devices. Data packets in the network are forwarded to corresponding ports or links according to their quintuple information. Typical implementations include virtual switches (vSwitch) and virtual routers (vRouter). The operating mode of a software forwarding device is usually to start several polling (PMD) threads to forward traffic, and these PMD threads are bound to several dedicated CPU cores on a general-purpose server to provide the physical computing resources required for forwarding. Therefore, the forwarding performance that the software forwarding device can provide, that is, the upper limit of the bandwidth, is provided by the CPU sum dedicated to forwarding.

However, the forwarding capabilities that these CPUs can provide are not fixed, and the forwarding rates/bandwidths they can provide are also different under different traffic characteristics. For example, a CPU core can forward a data stream with a packet size of 1518 bytes to 10Gbps, but it can forward a data stream with a packet size of 64 bytes only to 2Gbps bandwidth. This brings challenges to the bandwidth guarantee of tenants.

The existing rate limiting methods are all based on the port rate, that is, the rate is directly limited according to the bandwidth indicator, and the traffic higher than the indicator is not released, such as the widely used bits per second (BPS) or packets per second ( PPS) token bucket strategy, in the software forwarding device to forward traffic according to the token, but this method is easy to fail due to the competition of traffic forwarding tasks for CPU resources. It is very common that when a flow suddenly changes the packet size (such as from 1518 bytes to 64 bytes), in a software forwarding device, although the bandwidth/forwarding rate of the flow remains the same and does not exceed the limit, However, the CPU resource consumption of the flow will increase several times, which preempts the forwarding CPU resources originally belonging to other flows, reduces the CPU resources available to other flows, and reduces the bandwidth.

This phenomenon will cause some flows to be disturbed because other flows have changed the traffic characteristics, the bandwidth will be unstable, and the isolation of network performance will be poor. However, the existing rate limiting methods do not consider the competition of traffic for CPU resources and the unstable processing capability of the software forwarding device in implementation. Therefore, a software forwarding rate limiting method with good resource isolation needs to be proposed.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the present invention proposes a forwarding rate limiting method for a virtual forwarding device, including:

Step 1. Obtain the average data packet size and flow number of the service provider network as traffic characteristics through measurement. The service provider network includes a virtual forwarding device and multiple tenants. Use a traffic generation tool to generate traffic that meets the characteristics. The forwarding device forwards the data packets of the traffic and measures the CPU resources used for forwarding to reach each bandwidth, and constructs the corresponding relationship between CPU resources and bandwidth;

Step 2. According to the basic bandwidth requirements of each tenant and the corresponding relationship, allocate basic CPU resources to each tenant, and obtain the remaining CPU resources of the service provider network after the basic CPU resources are satisfied, so that each tenant can be regenerated according to the weight. Allocate idle CPU resources, and the final CPU resources used by each tenant for forwarding are the sum of the basic CPU resources and idle CPU resources allocated to them;

Step 3: The virtual forwarding device uses the final CPU resource to complete the forwarding of the data packet to its corresponding tenant.

The forwarding rate limiting method of the virtual forwarding device, wherein step 3 includes: the virtual forwarding device utilizes the final CPU resource to limit the speed of the data packet forwarded by the tenant, and the data traffic when the final CPU resource is exceeded will wait or be discarded .

In the method for limiting the forwarding rate of the virtual forwarding device, the units of the basic CPU resource, the idle CPU resource and the final CPU resource are all CPUcycles/s.

In the method for limiting the forwarding rate of the virtual forwarding device, the step 3 includes: using a token bucket or a fair queue to limit the rate of the data packets forwarded by the tenant.

The forwarding rate limiting method of the virtual forwarding device, wherein the virtual forwarding device is a virtual router or a virtual switch.

The present invention also proposes a forwarding rate limiting system for a virtual forwarding device, which includes:

Module 1 is used to measure and obtain the average data packet size and flow number of the service provider network as traffic characteristics. The service provider network includes virtual forwarding devices and multiple tenants, and the traffic generation tool is used to generate traffic that meets the characteristics. The virtual forwarding device forwards the data packets of the traffic and measures the CPU resources used for forwarding to reach each bandwidth, and constructs the corresponding relationship between CPU resources and bandwidth;

Module 2 is used to allocate basic CPU resources to each tenant according to the basic bandwidth requirements of each tenant and the corresponding relationship, and obtain the remaining CPU resources of the service provider network after meeting the basic CPU resources, so that each The weight redistributes idle CPU resources. The final CPU resource used by each tenant for forwarding is the sum of the basic CPU resources and idle CPU resources allocated to it. The virtual forwarding device uses the final CPU resource to complete the forwarding of data packets to its corresponding tenant. .

The forwarding rate limiting system of the virtual forwarding device, wherein the virtual forwarding device utilizes the final CPU resource to limit the speed of the data packets forwarded by the tenant, and the data traffic exceeding the final CPU resource will wait or be discarded.

In the forwarding rate limiting system of the virtual forwarding device, the units of the basic CPU resource, the idle CPU resource and the final CPU resource are all CPUcycles/s.

In the forwarding rate limiting system of the virtual forwarding device, the step 3 includes: using a token bucket or a fair queue to limit the rate of the data packets forwarded by the tenant.

The forwarding rate limiting system of the virtual forwarding device, wherein the virtual forwarding device is a virtual router or a virtual switch.

As can be seen from the above scheme, the advantages of the present invention are:

(1) Isolation. This method fundamentally eliminates the competition of different flows or tenants for CPU resources in the software forwarding device. By rationally allocating CPU resources according to the corresponding relationship between bandwidth and CPU resources, each flow or tenant can enjoy its own required CPU resources. To do traffic forwarding without being affected by other users, and achieve bandwidth isolation.

(2) Robustness. Robustness is achieved under the premise of isolation. After the CPU-cycle-based speed limit method we propose guarantees the basic bandwidth, in order to make full use of the CPU resources in the software forwarding device, the CPU resources are still idle. It is assigned to flows or tenants by weight, which can resist packet loss caused by traffic bursts to a certain extent.

(3) Platform independent. What we propose is a method that has platform-independent features, ranging from establishing bandwidth-CPU correspondence, allocating CPU resources to different flows or tenants, and can be implemented on any software-implemented forwarding device, such as vRouter , vSwitch, etc.

To sum up, the present invention focuses on the bandwidth isolation problem of the rate limiting method under software forwarding, and proposes a rate limiting method based on CPU-cycle. The rate-limiting indicator is hardware CPU resources rather than traditional bandwidth indicators such as BPS/PPS. By building the bandwidth-CPU correspondence in the soft forwarding device, allocating the required forwarding CPU resources for flows or tenants. way to achieve rate limiting and isolation for these flows or tenants. The allocation and isolation at the CPU resource level ensures that these flows or tenants are not interfered with, and at the same time fully allocates idle CPU resources and resists the impact of burst traffic packet loss, achieving isolation and robust rate limiting.

Description of drawings

Fig. 1 is the method flow chart of the present invention;

2 is a schematic diagram of an embodiment of the present invention;

Figure 3 is a flow chart of each batch forwarding of the CPUcycle-based token bucket mechanism.

Detailed ways

In this design, in order to solve the problem of poor isolation of the existing rate limiting methods in software forwarding devices, in order to limit CPU competition and reasonably allocate forwarding CPU resources, we propose a CPU-cycle-based rate limiting method. allocation to realize the function of speed limit.

The biggest difference between this method and the traditional speed limit method is the speed limit index. The traditional method directly limits the speed according to the bandwidth index (ie, BPS or PPS), and discards or waits for the traffic higher than the bandwidth index. Therefore, the competition of CPU resources will seriously affect the speed limiting effect and damage the isolation of network performance. We start from the hardware resources of the most basic forwarding engine. Since the forwarding rate of the software forwarding device is provided by dedicated CPU resources, the CPU-cycle-based rate limiting method we propose no longer judges the traffic allowable according to the bandwidth index. Forwarding is not allowed, but by limiting the number of CPU cycles (CPU cycles, a unit of CPU resources) that tenants' forwarding tasks consume per second in the forwarding device, for example, a 2.0GHz CPU core has 2.0G CPUcycles per second. Limit the forwarding rate. Tenants can be virtual machines connected to virtual switches or various virtual devices connected to virtual routers.

In order to limit the forwarding rate by limiting the CPUcycles/s consumed by each flow/tenant, the corresponding relationship between CPUcycles/s and bandwidth in the forwarding device needs to be established first. In different software forwarding devices, the conditions that affect the bandwidth-CPU correspondence roughly include two categories:

1) Flow characteristics. The packet size and the number of concurrent streams will affect this correspondence. For example, under the same bandwidth, 64-byte data packets consume more CPU resources, and the number of concurrent streams will also lead to slower table lookup steps in the forwarding device, consuming more CPU resources. Therefore, in order to eliminate the impact of such conditions on the corresponding relationship between bandwidth and CPU, service providers can constrain the bandwidth when selling bandwidth to a specific packet size and a specific number of streams, such as collecting data on the forwarding device for a period of time ( e.g. one hour) to obtain the average packet size and number of flows in the traffic flowing through the forwarding device. And use this to determine the corresponding relationship between bandwidth and CPU resources.

2) Platform features. The hardware performance of different CPU platforms is different, and the bandwidth that can be forwarded is also different. Therefore, the platform can easily affect the corresponding relationship between bandwidth and CPU. The number and complexity of rules such as routing tables and flow tables in the software forwarding device will also affect the corresponding relationship. Construct. Therefore, these platform characteristics need to be measured on the corresponding platform.

After considering the above two characteristics, a measurement-based bandwidth-CPU correspondence can be constructed. Under the guidance of this relationship, each flow or tenant can obtain the basic CPU resources C _basic required to achieve its purchased bandwidth in actual scenarios. (unit is CPUcycles/s), this can guarantee the basic bandwidth. After the basic bandwidth is guaranteed, in order to make full use of CPU resources, it is also necessary to re-allocate idle CPU resources C _idle for each stream according to the weight of the remaining resources on the current CPU core to make full use of idle resources to achieve the maximum bandwidth of all users. The C _idle here contains two parts of idle CPU resources: one is the unallocated CPU resources on the core, and the other is the allocated but unused resources. In the end, each stream or tenant is allocated the CPU resource for forwarding as C _basic + C _idle , which ensures both isolation and robustness in terms of bandwidth guarantee and speed limit, and can resist to a certain extent The impact of traffic bursts.

After the CPU resources are allocated, the next step is to use only the allocated CPU resources to forward traffic for the corresponding flow or tenant. You can use token buckets, fair queues, etc. to use these allocated limited CPU resources to limit the speed. For a flow or tenant, traffic beyond the processing capacity of the CPU resources allocated by these units will wait or be blocked. throw away. In this way, the isolation of streams or tenants at the hardware CPU resource level is achieved, thereby achieving good isolation of bandwidth.

In order to make the above-mentioned features and effects of the present invention more clearly and comprehensible, embodiments are given below, and detailed descriptions are given below in conjunction with the accompanying drawings.

The following is an embodiment of the speed limiting method proposed by the present invention.

In this embodiment, openvSwitch is selected as a typical software forwarding platform, and openvSwitch is a benchmark of virtual switches in the industry and is widely used. We implemented this rate limiting method on it using a CPU-cycle-based token bucket. The number of tokens in the token bucket is the remaining CPUcycles, and the token generation rate unit is CPUcycles/s.

The implementation of the speed limiting method is shown in Figure 1, which is divided into the following steps:

1. Measure the bandwidth-CPU correspondence of average traffic on a server platform (including multiple virtual machines and an openvSwitch). To measure this correspondence, you need to first measure the average packet size and flow number of the traffic passing through the open vSwitch for a period of time, and then use the traffic generation tool to generate traffic that matches the characteristics of the average packet size and flow number, and forward it in the openvSwitch packets of this traffic and measure the CPU resources required for forwarding to achieve each bandwidth specification, ie C _basic .

2. Assign a token generation rate (i.e. CPUcycles/s) to each tenant based on their purchased bandwidth. Through the correspondence established in the first step above, find the corresponding CPU cycles/s and idle CPU resources according to the purchased bandwidth, and set the token generation rate of the corresponding token bucket. As shown in Figure 2, 4 VMs use a 2.2GHz CPU core for forwarding in openvSwitch, assuming that their C basics according to the bandwidth-CPU correspondence are 0.2Gcycles/s, 0.2Gcycles/s, _0.4G respectively cycles/s, 0.4Gcycles/s, and there are 1Gcycles/s of idle CPU resources left, then the allocation is based on weights on the basis of ensuring isolation. Therefore, the actual token generation rates set by each VM are 0.36Gcycles/s and 0.36Gcycles, respectively. /s, 0.72Gcycles/s, 0.72Gcycles/s.

3. Use CPU-cycle-based token buckets to limit the rate during forwarding. The operation process of the token bucket mechanism is shown in Figure 3. Every time there is traffic to be forwarded, first check whether there are CPUcycles in the token bucket of the corresponding tenant. If the remaining CPUcycles is greater than 0, forward a batch of data packets for it, and subtract the consumed CPUcycles from the token bucket after the forwarding process; if the remaining CPUcycles is less than 0, skip this batch forwarding, Forward traffic for the next flow or tenant.

In the above process, we show how to use a CPU-cycle-based token bucket mechanism to implement the rate limiting method, in which the token generation rate in the token bucket and the action mechanism of the token bucket during the forwarding process are realized The point of this rate-limiting method. However, this rate limiting method can not only be implemented with token buckets, but some other mechanisms such as fair queues can also allocate and isolate CPU resources to implement this rate limiting method.

Through this method, the isolation and allocation of CPU resources dedicated to forwarding in the software forwarding device realizes the isolation of the speed limit, so that the flow and the tenant will no longer affect each other, and can be flexibly applied in various software forwarding scenarios, providing users with Stable bandwidth guarantee service.

The following are system embodiments corresponding to the foregoing method embodiments, and this implementation manner may be implemented in cooperation with the foregoing implementation manners. The related technical details mentioned in the foregoing embodiment are still valid in this embodiment, and are not repeated here in order to reduce repetition. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the above-mentioned embodiments.

The present invention also proposes a forwarding rate limiting system for a virtual forwarding device, which includes:

Module 1 is used to measure and obtain the average data packet size and flow number of the service provider network as traffic characteristics. The service provider network includes virtual forwarding devices and multiple tenants, and the traffic generation tool is used to generate traffic that meets the characteristics. The virtual forwarding device forwards the data packets of the traffic and measures the CPU resources used for forwarding to reach each bandwidth, and constructs the corresponding relationship between CPU resources and bandwidth;

Module 2 is used to allocate basic CPU resources to each tenant according to the basic bandwidth requirements of each tenant and the corresponding relationship, and obtain the remaining CPU resources of the service provider network after meeting the basic CPU resources, so that each The weight redistributes idle CPU resources. The final CPU resource used by each tenant for forwarding is the sum of the basic CPU resources and idle CPU resources allocated to it. The virtual forwarding device uses the final CPU resource to complete the forwarding of data packets to its corresponding tenant. .

The forwarding rate limiting system of the virtual forwarding device, wherein the virtual forwarding device utilizes the final CPU resource to limit the speed of the data packets forwarded by the tenant, and the data traffic exceeding the final CPU resource will wait or be discarded.

In the forwarding rate limiting system of the virtual forwarding device, the units of the basic CPU resource, the idle CPU resource and the final CPU resource are all CPUcycles/s.

In the forwarding rate limiting system of the virtual forwarding device, the step 3 includes: using a token bucket or a fair queue to limit the rate of the data packets forwarded by the tenant.

The forwarding rate limiting system of the virtual forwarding device, wherein the virtual forwarding device is a virtual router or a virtual switch.

Claims (10)

1. a forwarding rate limiting method of a virtual forwarding device, is characterized in that, comprising: Step 1. Obtain the average data packet size and flow number of the service provider network as traffic characteristics through measurement. The service provider network includes virtual forwarding devices and multiple tenants. Use a traffic generation tool to generate traffic that meets the traffic characteristics. The virtual forwarding device forwards the data packets of the traffic and measures the CPU resources used for forwarding to reach each bandwidth, and constructs the corresponding relationship between CPU resources and bandwidth; Step 2. According to the basic bandwidth requirements of each tenant and the corresponding relationship, allocate basic CPU resources to each tenant, and obtain the remaining CPU resources of the service provider network after the basic CPU resources are satisfied, so that each tenant can be regenerated according to the weight. Allocate idle CPU resources, and the final CPU resources used by each tenant for forwarding are the sum of the basic CPU resources and idle CPU resources allocated to them; Step 3: The virtual forwarding device uses the final CPU resource to complete the forwarding of the data packet to its corresponding tenant. 2 . The forwarding rate limiting method for a virtual forwarding device according to claim 1 , wherein step 3 comprises: the virtual forwarding device uses the final CPU resource to limit the speed of the data packets forwarded by the tenant, and exceeds the final CPU resource. 3 . When the data traffic will wait or be dropped. 3 . The forwarding rate limiting method of a virtual forwarding device according to claim 1 , wherein the units of the basic CPU resource, the idle CPU resource and the final CPU resource are all CPU cycles/s. 4 . 4 . The forwarding rate limiting method for a virtual forwarding device according to claim 2 , wherein step 3 comprises: using a token bucket or a fair queue to limit the rate of data packets forwarded by the tenant. 5 . 5 . The forwarding rate limiting method of a virtual forwarding device according to claim 2 , wherein the virtual forwarding device is a virtual router or a virtual switch. 6 . 6. A forwarding rate limiting system of a virtual forwarding device, characterized in that, comprising: Module 1 is used to measure and obtain the average data packet size and flow number of the service provider network as traffic characteristics. The service provider network includes virtual forwarding equipment and multiple tenants. The traffic generation tool is used to generate traffic that meets the traffic characteristics. The virtual forwarding device forwards the data packets of the traffic and measures the CPU resources used for forwarding to reach each bandwidth, and constructs the corresponding relationship between the CPU resources and the bandwidth; Module 2 is used to allocate basic CPU resources to each tenant according to the basic bandwidth requirements of each tenant and the corresponding relationship, and obtain the remaining CPU resources of the service provider network after meeting the basic CPU resources, so that each The weight redistributes idle CPU resources. The final CPU resource used by each tenant for forwarding is the sum of the basic CPU resources and idle CPU resources allocated to it. The virtual forwarding device uses the final CPU resource to complete the forwarding of data packets to its corresponding tenant. . 7. The forwarding rate limiting system of a virtual forwarding device according to claim 6, wherein the virtual forwarding device utilizes the final CPU resource to limit the speed of the data packet forwarded by the tenant, and the data flow when exceeding the final CPU resource will be wait or be discarded. 8 . The forwarding rate limiting system of a virtual forwarding device according to claim 6 , wherein the units of the basic CPU resource, the idle CPU resource and the final CPU resource are all CPU cycles/s. 9 . 9 . The forwarding rate limiting system of a virtual forwarding device according to claim 7 , wherein the module 2 comprises: using a token bucket or a fair queue to limit the rate of data packets forwarded by the tenant. 10 . 10 . The forwarding rate limiting system of a virtual forwarding device according to claim 7 , wherein the virtual forwarding device is a virtual router or a virtual switch. 11 .

Images