CN114422445A

CN114422445A - Method for realizing load balance and out-of-order recombination

Info

Publication number: CN114422445A
Application number: CN202210172819.0A
Authority: CN
Inventors: 杨成勇
Original assignee: Chengdu Beizhong Network Core Technology Co ltd
Current assignee: Chengdu Beizhong Network Core Technology Co ltd
Priority date: 2022-02-24
Filing date: 2022-02-24
Publication date: 2022-04-29
Anticipated expiration: 2042-02-24
Also published as: CN114422445B

Abstract

The invention relates to a method for realizing load balancing and out-of-order recombination, belonging to the field of networks. The invention is applied to a system comprising equipment A, equipment B and parallel mature equipment, wherein the equipment A and the equipment B comprise a load balancing module and an out-of-order recombination module, the mature equipment is positioned between the equipment A and the equipment B, the load balancing module distributes non-IP messages and IP messages to the mature equipment, the mature equipment outputs the processed flow to the out-of-order recombination module, the equipment A or the equipment B distributes an increasing global sequence number Gseq and a snap flag according to the inlet sequence of each IP message, after the IP messages are processed by the mature equipment, the message sequence number Gseq and the snap flag information extracted from the out-of-order recombination end are recombined according to the message sequence number Gseq and the sequence number (Next Gseq) of the message to be sent locally and currently, and the message is sent after message recovery is carried out according to the snap flag. The invention improves the network processing performance by one order of magnitude, and greatly reduces the research and development cost.

Description

Method for realizing load balance and out-of-order recombination

Technical Field

The invention belongs to the field of network communication and network security, and particularly relates to a method for realizing load balancing and out-of-order recombination.

Background

Network communication and network security often require full packet filtering, or full packet inspection, or full packet encryption and decryption of messages on the line. The limited full packet filtering and the processing performance limitation of the full packet encryption and decryption engine often need to connect a plurality of processing devices in parallel, and perform the full packet filtering and the full packet encryption and decryption simultaneously. After the data are processed by the multiple devices in parallel, the data streams of the devices are needed to be recombined out of order and then merged for output, so that the bandwidth of 100G or more can be achieved. Each mature device has effective processing capacity, can only process standard messages, has high development cost of performance upgrading, and becomes an optimal scheme by stacking the existing devices in parallel.

As shown in fig. 1, there are two main load balancing methods for parallel processing of a plurality of processing devices: destination-based network (IP load-sharing per-destination) and packet-based (IP load-sharing per-packet).

Target network based load balancing is the distribution of load according to target address. Assuming that there are two paths to a network, a packet destined for a first destination in the network passes through the first path, a packet destined for a second destination in the network passes through the second path, all packets destined for a third destination in the network also pass through the first path, and so on. As shown in fig. 2:

the advantages of this loading mode are: 1. the realization is simple, namely, the target address is subjected to hash operation, and a path is selected according to the operation result; 2. the messages in each data stream can be processed in order preserving mode, the data stream merging stage is simple, and disorder recombination is not needed.

The disadvantages of this loading approach: 1. the elephant flow (the large rate, the long-time flow is the elephant flow) with the rate exceeding the load balancing output sub-port cannot be load balanced, for example, 100G of flow needs to be load balanced to 10 devices, the processing bandwidth of each device is 10G, and if the bandwidth of a certain elephant flow reaches 50G, the elephant flow cannot be load balanced to a plurality of devices based on the target network; 2. another disadvantage is that the traffic cannot be uniformly load balanced to each device because the traffic size of each data stream is too different and simple hash calculation cannot uniformly distribute the traffic.

Load balancing based on data packets does not force messages of the same flow to be sent to the same port, and the data packets are balanced to the ports based on the existing flow of each port. As shown in fig. 3:

the advantages are that: the traffic can be evenly distributed to each port and the traffic of the elephant flow can be evenly distributed to each port.

The disadvantages are as follows: when data streams are combined, the out-of-order recombination is needed to ensure that the messages of the same stream are output after being combined in sequence, and how to carry out the out-of-order recombination is a difficult problem.

Therefore, the current common load balancing technologies have the defects that the flow can be uniformly distributed and the order preserving function of the same flow can be realized, and for a single elephant flow with an ultra-large bandwidth, the traditional load balancing technology cannot realize the splitting and recombination of the elephant flow.

Disclosure of Invention

Technical problem to be solved

The technical problem to be solved by the invention is how to provide a method for realizing load balancing and disorder recombination, so as to solve the problems that the current common load balancing technology is difficult to realize, namely, the flow can be uniformly distributed, the order preserving function of the same flow can be realized, and for a single elephant flow with a super-large bandwidth, the splitting and recombination of the elephant flow can not be realized by the traditional load balancing technology.

(II) technical scheme

In order to solve the technical problem, the invention provides a method for realizing load balancing and disorder recombination, which is applied to a system comprising a device A, a device B and 2N parallel mature devices, wherein the device A and the device B are the same device and respectively comprise a load balancing module and a disorder recombination module, and the mature devices are positioned between the device A and the device B;

the load balancing module of the device A distributes non-IP messages to mature devices 1, IP message flow is uniformly distributed and output to mature devices 2-N, after the flow is processed by the N mature devices, the flow is output to the disorder recombination module of the butt joint device B, and the flow is output through the device B after disorder recombination;

the load balancing module of the equipment B distributes the non-IP messages to mature equipment N +1, IP message flow is uniformly distributed and output to the mature equipment N + 2-2N, after the flow is processed by the N mature equipment, the non-IP messages are output to the disorder recombination module of the butt joint equipment A, and the non-IP messages are output through the equipment A after the disorder recombination;

for each IP message, according to the entry sequence, the equipment A or the equipment B allocates an increasing global sequence number Gseq and a snap flag, wherein the snap flag is used for indicating whether the message is an IP message in a snap format or not, converting the IP message into a standard format of the snap IP message, and filling the snap flag and the Gseq into a manufacturer code domain in the snap IP message format; after an IP message is processed by mature equipment, disorder reaches a disorder restructuring module, message serial number Gseq and snap flag information extracted by a disorder restructuring end are restructured according to the message serial number Gseq and a local current message to be sent serial number (Next Gseq), and the message is sent after message recovery is carried out according to snap flag marks.

Further, N is 12.

Further, the 2N parallel mature devices are 24 parallel mature devices with the processing performance of 10 Gbps.

Further, for the non-IP messages, the mature device 1 and the mature device N +1 are sequentially forwarded to the out-of-order reassembly module, and the out-of-order reassembly module does not perform out-of-order reassembly on the non-IP messages and alternates into the IP messages according to the input sequence of the non-IP messages for forwarding.

Further, the Gseq is 23 bits, the snap flag is 1bit, if the incoming message is an IP message in snap format, the snap flag is 1, and if the incoming message is not an IP message in snap format, the snap flag is 0.

Further, when the equipment A or the equipment B processes, each IP message is converted into a standard format of a snap IP message, if the incoming message is the snap IP message, the format of the snap IP message is reserved, and 3 Bytes are formed by snap flag and Gseq and filled into a manufacturer code domain in the snap IP message format.

Further, the load balancing module counts the flow output to each mature device in real time, and always sends the current message to the mature device with the minimum flow displayed by the current flow statistics.

Further, the mature device is a DPI device or an encryption and decryption device.

Further, the specifically recombining the sequence of the messages according to the message sequence number Gseq and the sequence number (Next Gseq) of the message to be sent currently locally includes: inquiring whether a sequence number (Next Gseq) of a local current message to be sent is equal to the Gseq of the message, if so, immediately sending the message, and performing incremental operation on the Next Gseq; if the sequence number of the Next Gseq exceeds the sequence number of the message, the time of delaying arrival of the message exceeds the set time, and other messages have changed the Next Gseq due to waiting overtime, so the message needs to be sent immediately without changing the Next Gseq; if the Gseq of the message exceeds the Next Gseq, the message before the message is not sent and cannot be sent immediately, delaying for a period of time, then inquiring the Next Gseq, if the Next Gseq is equal to or exceeds the Gseq of the message, immediately sending the message, otherwise continuing delaying for inquiring, if the time of the cyclic inquiry is overtime, immediately sending the message, and modifying the Next Gseq into the value of Gseq +1 of the message.

Further, the sending the message after the message recovery according to the snap _ flag specifically includes: if snap _ flag is 0, it indicates that the original message is not snap message, and restores the message to Ethernet II message format; if SNAP _ flag is 1, the original message is the SNAP message, the SNAP manufacturer code transmits Gseq and SNAP flag, and the SNAP manufacturer code is cleared by 0; and after the message is recovered, recalculating the FCS according to the sequence of the message after disorder and recombination, and sending the message one by one.

(III) advantageous effects

The invention provides a method for realizing load balancing and out-of-order recombination, which utilizes an 802.3/802.2 standard protocol, does not modify the existing mature processing equipment, improves the network processing performance by an order of magnitude through connecting the mature equipment in parallel (see the technical scheme and the implementation mode in detail), and greatly reduces the research and development cost.

Drawings

FIG. 1 is a schematic diagram of a load balancing method for parallel processing of a plurality of processing devices in the prior art;

FIG. 2 is a schematic diagram of a conventional target network-based load balancing;

FIG. 3 is a diagram illustrating conventional packet-based load balancing;

FIG. 4 is a connection diagram of the apparatus of the present invention;

FIG. 5 is a diagram illustrating two types of Ethernet message formats;

FIG. 6 is a snap message format defined by 802.3;

FIG. 7 is a diagram illustrating a manufacturer code domain carrying a message sequence number and a snap flag;

FIG. 8 is a block diagram of a process according to an embodiment of the present invention;

FIG. 9 is a flow chart of message processing at the load balancing end of the present invention;

fig. 10 is a flow chart of message processing at the out-of-order reassembly end according to the present invention.

Detailed Description

In order to make the objects, contents and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.

The invention aims to solve the load balancing problem that the flow can be balanced uniformly and the order of the same flow can be kept.

The invention mainly solves the following three difficult problems:

1. the load balancing end needs to print the messages of the inlet with the sequence numbers (numbering each message) which are increased one by one according to the inlet sequence;

2. the flow merging end needs to carry out-of-order recombination according to the serial number of each message, and the serial number is sent before;

3. the middle message processing engine is a mature device, can only recognize messages in a standard format, and how to bring the serial number of the message from the load balancing end to the flow merging end along with the path is a difficult problem to be solved by the patent.

The method for realizing load balancing and disorder recombination is applied to a system comprising a device A, a device B and 2N parallel mature devices, wherein the device A and the device B are the same device and respectively comprise a load balancing module and a disorder recombination module, and the mature devices are positioned between the device A and the device B;

As shown in fig. 4, the apparatus of the present invention comprises two parts, one is a load balancing module and the other is an out-of-order reassembly module, and the two modules are in the same apparatus. In general, two identical devices A and B are adopted, and a plurality of mature devices are connected in parallel in the middle. For example, as shown in fig. 4, two devices of the present invention are connected in parallel to 24 mature devices with processing performance of only 10Gbps, so as to realize bidirectional processing performance of 200 Gbps:

the 100Gbps input flow of the equipment A can distribute non-IP messages (with small flow ratio) to the mature equipment 1 through a load balancing module of the equipment A, the flow of the IP messages is uniformly distributed and output to 10G input ports of the mature equipment 2-12, the 12 mature equipment outputs a disorder recombination module of the butt joint equipment B after the flow is processed, and the disorder recombination module outputs the messages through a 100Gbps port of the equipment B after the disorder recombination.

In the other direction, 100Gbps flow of the equipment B is input, non-IP messages (with small flow ratio) can be distributed to the mature equipment 13 through the load balancing module of the equipment B, the flow of the IP messages is uniformly distributed and output to the input ports of the mature equipment 14-24, the 12 mature equipment outputs the flow to the disorder recombination module of the butt joint equipment A after the flow is processed, and outputs the flow through the 100Gbps port of the equipment A after the disorder recombination.

More than 95% of network messages are IP messages, and the method performs load balancing and out-of-order recombination on the IP messages.

For non-IP packets, their traffic is small and IP packets and non-IP packets do not need to be order-preserved. The non-IP messages are uniformly forwarded to a fixed mature device (such as a mature device of a channel 0), the mature device can sequentially forward the non-IP messages to a disorder recombination module of another device, as the path of the non-IP messages is single, disorder cannot be generated, the disorder recombination module cannot perform disorder recombination processing on the non-IP messages, the non-IP messages are inserted into the IP messages according to the input sequence of the non-IP messages for forwarding, and as long as the non-IP messages are not disordered;

IP messages can be encapsulated in two types of ethernet message formats. The first was the Ethernet II format proposed in the last 80 th century. The second is the IEEE802.3 format proposed in 1983. The IP packet in the SNAP format is one of the packet formats in IEEE 802.3. As shown in fig. 5:

for IP messages, firstly, according to the entry sequence, the equipment A or the equipment B allocates an increasing global serial number Gseq (23 bits) to each IP message and a snap flag (bi t, if the incoming message is the IP message in snap format, snap flag is 1; if the incoming message is not the IP message in snap format, snap flag is 0); and converting each IP message into the standard format of a snap IP message. If the incoming message is a snap IP message, the format of the snap IP message is reserved. The snap flag and the Gseq form 3 bytes (24 bits) and are filled in a manufacturer code domain in a snap IP message format, the manufacturer code in the snap message does not influence the transmission of the message, and the Gseq and the snap mark of the message are transmitted through the manufacturer code. The modified message is in a standard message format defined by 802.3, and each mature device can normally recognize and process the messages.

The load balancing module counts the flow output to each mature device in real time and always sends the current message to the mature device with the minimum flow displayed by the current flow statistics. The load balancing module can count the output flow of each 10Gbps in real time and always send the current message to the 10G output port with the minimum flow displayed by the current flow statistics, and the method can achieve the approximately same flow of each 10Gbps output port;

after being processed by each external mature device (generally, a DPI device or an encryption and decryption device), the IP messages arrive at a disorder recombination module of another device in a disorder manner; as the mature equipment does not modify the standard 802.3 message format, the manufacturer code of the snap message reaching the disordered recombination end carries the message serial number Gseq, snap flag and other information; and the message sequence number Gseq, snap flag and other information extracted by the disordered recombining end is recombined and sent according to the message sequence number Gseq and the local current message sequence number (Next Gseq) to be sent.

Inquiring whether a sequence number (Next Gseq) of a local current message to be sent is equal to the Gseq of the message, if so, immediately sending the message, and performing incremental operation on the Next Gseq; if the sequence number of the Next Gseq exceeds the sequence number of the message, the time of delaying arrival of the message exceeds the set time, and other messages have changed the Next Gseq due to waiting overtime, so the message needs to be sent immediately without changing the Next Gseq; if the Gseq of the message exceeds the Next Gseq, the message before the message is not sent and cannot be sent immediately, delaying for a period of time, then inquiring the Next Gseq, if the Next Gseq is equal to or exceeds the Gseq of the message, immediately sending the message, otherwise continuing delaying for inquiring, if the time of the cyclic inquiry is overtime, immediately sending the message, and modifying the Next Gseq into the value of Gseq +1 of the message.

After the disorder reorganization is finished, the disorder reorganization end needs to perform actions such as message recovery according to the snap _ flag mark. If snap _ flag is 0, it indicates that the original message is not a snap message, and the format of the snap message is changed to that of the snap message in order to transmit Gseq and snap flag, and the message needs to be restored to the Ethernet II message format; if SNAP _ flag is 1, the original message is the SNAP message, the SNAP manufacturer code transmits Gseq and SNAP flag, and the SNAP manufacturer code is cleared by 0; after the message is recovered, recalculating FCS (CRC check code of the message) according to the sequence after the message is out-of-order recombined, and sending the message one by one;

fig. 6 shows a snap message format defined by 802.3; fig. 7 shows a modified message format carrying a message serial number and a snap flag through a manufacturer code domain. Fig. 8-10 are block diagrams and flow diagrams of processes of the present invention.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims

1. A method for realizing load balancing and disorder recombination is characterized in that the method is applied to a system comprising a device A, a device B and 2N parallel mature devices, the device A and the device B are the same device and respectively comprise a load balancing module and a disorder recombination module, and the mature devices are positioned between the device A and the device B;

2. The method of claim 1 wherein N is 12.

3. The method of claim 1, wherein the 2N parallel mature devices are 24 parallel mature devices with processing performance of 10 Gbps.

4. The method as claimed in claim 1, wherein for the non-IP packets, the mature device 1 and the mature device N +1 are sequentially forwarded to the out-of-order reassembly module, and the out-of-order reassembly module does not perform out-of-order reassembly on the non-IP packets, and alternates the non-IP packets into the IP packets according to the input sequence of the non-IP packets for forwarding.

5. The method according to claim 1, wherein the Gseq is 23 bits, the snap flag is 1bit, if the incoming packet is an IP packet in snap format, the snap flag is 1, and if the incoming packet is not an IP packet in snap format, the snap flag is 0.

6. The method as claimed in claim 5, wherein the device A or the device B converts each IP packet into a standard format of a snap IP packet during processing, if the incoming packet is a snap IP packet, the format of the snap IP packet is reserved, and the snap flag and the Gseq are combined into 3 bytes and filled into a manufacturer code field in the snap IP packet format.

7. The method as claimed in claim 1, wherein the load balancing module counts the traffic output to each mature device in real time, and always sends the current message to the mature device with the smallest traffic as the current traffic statistics shows.

8. The method of claim 1, wherein the mature device is a DPI device or an encryption/decryption device.

9. The method according to any one of claims 1 to 8, wherein the reassembling the order of the packets according to the packet sequence number Gseq and a local sequence number (Next Gseq) of a currently-to-be-transmitted packet specifically comprises: inquiring whether a sequence number (Next Gseq) of a local current message to be sent is equal to the Gseq of the message, if so, immediately sending the message, and performing incremental operation on the Next Gseq; if the sequence number of the Next Gseq exceeds the sequence number of the message, the time of delaying arrival of the message exceeds the set time, and other messages have changed the Next Gseq due to waiting overtime, so the message needs to be sent immediately without changing the Next Gseq; if the Gseq of the message exceeds the Next Gseq, the message before the message is not sent and cannot be sent immediately, delaying for a period of time, then inquiring the Next Gseq, if the Next Gseq is equal to or exceeds the Gseq of the message, immediately sending the message, otherwise continuing delaying for inquiring, if the time of the cyclic inquiry is overtime, immediately sending the message, and modifying the Next Gseq into the value of Gseq +1 of the message.

10. The method according to claim 9, wherein the sending the message after the message is recovered according to the snap _ flag specifically comprises: if snap _ flag is 0, it indicates that the original message is not snap message, and restores the message to Ethernet II message format; if SNAP _ flag is 1, the original message is the SNAP message, the SNAP manufacturer code transmits Gseq and SNAP flag, and the SNAP manufacturer code is cleared by 0; and after the message is recovered, recalculating the FCS according to the sequence of the message after disorder and recombination, and sending the message one by one.