CN114268518B - Method and system for realizing forwarding acceleration of sdwan data tunnel - Google Patents
Method and system for realizing forwarding acceleration of sdwan data tunnel Download PDFInfo
- Publication number
- CN114268518B CN114268518B CN202111576110.9A CN202111576110A CN114268518B CN 114268518 B CN114268518 B CN 114268518B CN 202111576110 A CN202111576110 A CN 202111576110A CN 114268518 B CN114268518 B CN 114268518B
- Authority
- CN
- China
- Prior art keywords
- data packet
- link
- sdwan
- data
- packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a method and a system for realizing forwarding acceleration of an sdwan data tunnel, and belongs to the technical field of data processing. The method comprises the following steps: after receiving the data, storing the data into a hardware queue; judging whether a packet receiving control technology is opened, if so, carrying out multi-core distribution accelerated forwarding processing on a downlink data packet, identifying and accelerating sdwan data tunnel messages by each CPU through an independent data packet accelerated forwarding processing module, and if not, identifying and accelerating sdwan data tunnel messages through an independent data packet accelerated forwarding processing module. The independent kernel module takes over sdwan's data packet and carries out the packet forwarding, and the flow is simpler, need not carry out route inquiry to all data packets, reduces a large amount of legality detection actions, and the forwarding performance obviously promotes.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method for realizing the forwarding acceleration of an sdwan data tunnel, and a system for realizing the method for realizing the forwarding acceleration of the sdwan data tunnel.
Background
Sdwan (Soft Define WAN translation: software defined Wide area network) tunneling: sdwan is considered a cost effective technique for connecting remote sites requiring ultra-low reliability connections to achieve low latency and critical service applications. SD-WANs require a method of connection between two or more locations. Such connections typically include MPLS, internet broadband, or both. The goal is to extend the use of low cost connections while meeting the same latency and throughput requirements. Currently, the SD-WAN tunneling technology mainly includes a tunnel based on TCP/UDP socket (performance is poor, high performance requires support of CPU), netmap/DPDK (very suitable for realizing a chained flow of SD-WAN tunneling data forwarding), hardware support of SmartNIC, and the latter two have higher software requirements, the first performance is the worst, but the SD-WAN tunneling technology is suitable for software of operating systems of various architectures, and the SD-WAN tunneling technology is suitable for constructing an SD-WAN network based on a router, and has low cost.
As a service formed by applying the SDN technology to a wide area network scenario, sdwan connects enterprise networks, data centers, internet applications, and cloud services over a wide geographic range, helping users reduce the cost of wide area networks and improve network connection flexibility. Compared with the traditional wide area network (MPLS-VPN, IPSEC-VPN and the like), the technology is used for solving the problems of instability and high cost of a special line of the traditional Internet line and meeting the instantaneity and instantaneity of the future line to the application. And the separation of network control and forwarding is realized.
At a deployment end of the network equipment, a main node/data center equipment serving as a network control and data sharing center is often network equipment with high-performance forwarding data, and has the characteristics of high equipment cost and high private network cost, and as a branch node of a resource acquisition end (which can also share local area network resources), the problem that resources in a local area network and the problem of communication with the only main node/data center are emphasized is considered, and the situation that only micro-enterprise equipment with poor hardware performance is often used is sufficient.
However, as a special tunnel packet, software acceleration is not supported in most network devices, and the advantage of multi-core parallel processing of the tunnel packet cannot be achieved even in a multi-core hardware environment.
RPS (Receive Package packing Steering translation) technique: the software implementation of the RSS technology performs hash value calculation (differences may exist in each kernel version algorithm) according to a source ip, a source port, a destination ip, a destination port, a protocol (a four-layer protocol TCP/UDP, a three-layer protocol IPV4ORIPV 6), a flag (VLAN ID), and the like of each packet, and then matches different streams to cpus that process the streams according to the hash values (the hash values of packets of each stream are the same, so that the packets of each stream can be selected to the same cpu), so as to implement load balancing of the received packets among the cpus, and the RPS technology is suitable for a single-queue network card or a virtual network card.
For sdwan data tunnel messages using UDP protocols, RSS and RPS can only identify all tunnel data as one data stream, and all tunnel data can only be processed by one CPU, resulting in low utilization rate of multi-core CPUs and poor forwarding performance.
The memory and CPU of the network device using the sdwan data tunnel of the UDP protocol are poor, and the tunnel data stream itself consumes a lot of time on the forwarding path of the kernel protocol stack in addition to the predetermined encryption and decryption processes, and at this time, the forwarding performance of the network data will become a big pain point of the product.
Therefore, the method for realizing the forwarding acceleration of the sdwan data tunnel (UDP socket) is provided, and the method has technical reference value for other manufacturers with sdwan functions.
Disclosure of Invention
In order to solve the problem of poor performance in the prior art, the invention provides a method and a system for realizing the forwarding acceleration of an sdwan data tunnel.
The method comprises the following steps:
firstly, after receiving data, storing the data into a hardware queue;
judging whether a packet receiving control technology is opened or not, if so, performing multi-core distribution accelerated forwarding processing on a downlink data packet, identifying and accelerating sdwan data tunnel messages by each CPU through an independent data packet accelerated forwarding processing module, and if not, identifying and accelerating sdwan data tunnel messages through an independent data packet accelerated forwarding processing module;
the identification and acceleration of the sdwan data tunnel message processing process comprises the following steps:
(1) Releasing the data packets of the upstream and downstream front parts of each link into a protocol stack, and walking a normal protocol stack flow, wherein an sdwan kernel module creates an acceleration tunnel link and an acceleration tunnel fragment IP link while creating an sdwan tunnel, and creates sdwan links needing accelerated forwarding through the released data packets, wherein the sdwan links comprise a wanip link and a client link;
(2) Respectively accelerating the subsequent data packets of the link according to the types of the data packets, if the subsequent data packets are uplink packets, matching the client link with the corresponding wanip link to acquire wan Ethernet information sent by the data packets, then sending the data packets according to the acquired information,
and if the data packet is the downlink packet, directly matching the client link after decryption, acquiring the Ethernet information of the uplink direction interface stored in the client link, and sending the data packet according to the acquired information.
The invention is further improved, and the processing method for multi-core distribution accelerated forwarding processing comprises the following steps:
a1: registering a hook point, and redistributing a data packet processing CPU through the hook point;
a2: judging whether the downlink sdwan data packet is a fragment data packet, if not, matching the MAC information carried by the head of the sdwan data packet with the acceleration tunnel link, and if the matching is successful, successfully identifying; if the data packet is matched with the fragment link established by the accelerated tunnel fragment IP link, the successfully matched fragment packet is queued to each fragment queue, the matching of the accelerated tunnel link is carried out through the first fragment packet of each fragment queue to determine whether the data packet is the fragment queue of the sdwan data packet, the matching is failed, the data packet is sent back to the protocol stack for processing, and the data packet is recombined if the matching is successful;
a3: and allocating the processing CPU according to the hash value carried by the sdwan data packet head.
The invention is further improved, in the step a2, if the data packet is a fragmented data packet, the specific processing method is as follows:
a21: the source IP of the fragment data packet is matched with a fragment link established based on the acceleration tunnel, and the IP data packet of the non-acceleration tunnel is filtered;
a22: acquiring information of a data packet to initialize a fragmentation queue;
a23: if the data packet is the first fragment, whether the matching acceleration tunnel link is an sdwan data packet indeed or not is judged, if the matching is successful, the information of the data packet is obtained to initialize the fragment queue, the MAC layer information of the downlink data packet of the acceleration tunnel link is updated, then step a24 is executed, and if the matching fails or the data packet is not the first fragment, step a24 is directly executed;
a24: matching the fragment data packet with a fragment queue through self-carried information, marking the fragment queue of the data packet with an sdwan mark, and if the fragment queue does not exist, newly adding the fragment queue;
a25: stripping a pppoe header and a vlan header of all the fragmented data packets, and then enqueuing;
a26: aiming at the sdwan fragmentation queue, receiving the coming fragments, marking the arrival of the first fragment and the arrival of the last fragment, counting the length sum of all fragments, and sequencing the fragmentation data packets;
a27: and c, judging whether the first fragment and the last fragment arrive and the total length of the data packet is equal to the sum of the lengths of all the fragment data packets, if so, repeating the data packet, then executing the step a3, and if not, finishing.
The invention further improves the method, in the step (1), a client link is established through quintuple information of an uplink data packet of an sdwan client, a link pointer is mounted under link tracking and deleted along with aging of the link tracking, the client link is used for accelerating matching of sdwan downlink data packets after decryption and identifying the sdwan data packets as sdwan data packets, the link records Ethernet MAC address, vlan and PPPOE header information carried by the uplink sdwan data packets and is used for encapsulating MAC layer information before the downlink data packets are sent out, and when the client link is established, the wanip link matched by the link is searched and stored.
The invention is further improved, the created client link is successfully matched with the wanip link, and is judged to be TCP or UDP, if the client link is TCP, MSS during three-way handshake of TCP connection is modified, and if the client link is UDP, UDP fragmentation link is created based on the destination IP for matching uplink fragmentation packets.
The invention is further improved, in the step (2), if the packet is an uplink packet, the specific packet sending processing process is as follows:
b1: acquiring the header information of the data packet, judging whether the data packet is a fragment data packet, if so, executing the step b2, and if not, executing the step b3;
b2: accelerating tunnel fragment IP link matching through a target IP, if the target IP is not the first fragment, executing a step b4, if the target IP is not the first fragment, acquiring a packet quintuple matching client link, updating packet Ethernet header information, updating a vlan/ppp header information to the client link, updating the link tracking aging corresponding to the client connection, and then executing the step b4;
b3: acquiring a data packet quintuple matching client link, updating data packet Ethernet header information, connecting vlan/ppp header information to the client link, updating link tracking aging corresponding to the client link, judging whether the data packet is the uplink front part of the client link, if so, finishing, and if not, executing the step b4;
b4: removing the vlan head and the ppp head;
b5: encrypting a data packet, adding an sdwan header, and encapsulating a UDP header and an IP header;
b6: data packet fragmentation processing;
b7: adding MAC layer information carried by a client link matched with the data packet, and then packaging an uplink data packet;
b8: and calling a packet sending function according to the MAC layer information to send the data packet.
The invention is further improved, in step (2), if the data packet is a downlink data packet, the downlink sdwan data packet is marked when the multi-core distribution accelerated forwarding processing is performed, the identification can be directly performed, and the processing process included in the data successfully identified is as follows:
c1: matching the wanip link through the destination IP to further obtain an accelerated tunnel link;
c2: stripping an IP (Internet protocol) header and a UDP (user Datagram protocol) header of the sdwan downlink packet;
c3: decrypting the data packet, judging whether the decrypted data packet is a fragment data packet, if so, executing the step c4, and if not, executing the step c5;
c4: accelerating tunnel fragment IP link through source IP matching, if not the first fragment, executing step c6, if yes, acquiring a packet quintuple matching client link, and then executing step c6;
c5: acquiring a data packet quintuple matching client link, then judging whether the data packet is a data packet of a downlink front part of the client link, if so, updating the sdwan downlink IP link, ending, otherwise, carrying out fragmentation processing on the data packet, and then executing the step c6;
c6: adding MAC layer information carried by a client link matched with the data packet, and then packaging a downlink data packet;
c7: and calling a packet sending function according to the MAC layer information to send the data packet.
The invention also provides a system for realizing the method for realizing the forwarding acceleration of the sdwan data tunnel, which comprises the following steps:
an enqueue module: the device is used for storing the data into a hardware queue after receiving the data;
receive package control technology and open the module: the method is used for opening a packet receiving control technology to realize multi-core distribution accelerated forwarding;
the multi-core distribution accelerated forwarding module: the system is used for carrying out multi-core distribution accelerated forwarding processing on the downlink data packets;
the accelerated forwarding processing module: the method is used for identifying uplink and downlink data packets and accelerating sdwan data tunnel messages;
a creation module: the system is used for establishing an sdwan tunnel, an acceleration tunnel link and an acceleration tunnel fragment IP link, and establishing each sdwan link needing accelerated forwarding through a released data packet;
a storage module: the link is used for storing various links created by the creation module;
a bag sending module: the function is used for calling the packet sending function and sending the data packet.
The invention is further improved, and the storage module comprises an acceleration tunnel link memory pool, an acceleration tunnel fragment IP link memory pool, a wanip link memory pool and a client link memory pool.
In a further improvement of the present invention, the multi-core allocation acceleration forwarding processing module includes:
a registration module: the method is used for registering a hook point, replacing an original queue distribution function through the hook point and reallocating a data packet processing CPU;
a judgment module: the system is used for judging whether the downlink sdwan data packet is a fragment data packet or not;
a matching identification module: aiming at the non-fragmented data, the method is used for matching MAC information carried by the head of the sdwan data packet with the accelerated tunnel link, and if the matching is successful, the identification is successful; aiming at the fragmented data packets, firstly matching the fragmented links created by the accelerated tunnel fragmented IP links, queuing the fragmented packets after successful matching to each fragmented queue, carrying out matching of the accelerated tunnel links through the first fragmented packet of each fragmented queue to confirm whether the fragmented packets are the fragmented queues of the sdwan data packets, sending the data packets back to the protocol stack for processing if the matching is failed, and sending the data packets into the data packet recombination module for processing if the matching is successful;
a data packet reorganization module: the device is used for carrying out recombination processing on the data packet;
a distribution module: and the CPU is used for allocating the processing CPU according to the hash value carried by the sdwan data packet head.
Compared with the prior art, the method and the device analyze the characteristic of poor performance of the SD-WAN data tunnel using the UDP protocol, provide a solution for improving the performance and effectively improve the data forwarding efficiency. In particular, the following advantages are provided:
(1) The information of the stream to which the data packet belongs is added at the sdwan head, so that the data stream of the UDP tunnel can be acquired and distributed to different CPUs for processing according to the data stream to which the data packet belongs under the condition that the data stream cannot be processed by the RSS/RPS in a multi-CPU balanced manner, the utilization rate of the CPUs is improved, and the downlink forwarding performance is obviously improved.
(2) The single kernel module takes over sdwan data packets for packet forwarding, and compared with normal PPP drive processing, the protocol stack performs forwarding processing for multiple times, so that the flow is simpler, routing query does not need to be performed on all data packets, and a large amount of legality detection actions are reduced;
(3) Because the sdwan data packet is encrypted, an sdwan header and a tunnel header are encapsulated, the length of the data packet is increased, and each node of the data packet in the transmission process needs to be fragmented and recombined. By adjusting the MSS connected between the client and the TCP of the sdwan server, the sdwan data packet does not need to be fragmented when being output after being encrypted, fragmentation recombination processing in the forwarding process is avoided, processing time delay on a network path is reduced, and bandwidth competition capability of the sdwan data packet is enhanced.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a flow diagram of a method for accelerated forwarding processing of data packets prior to entering a protocol stack;
FIG. 3 is a flowchart of a method for multi-core distribution expedited forwarding processing;
FIG. 4 is a flowchart of a method for accelerating tunnel connection and the like while tunnel connection is established by the kernel module;
fig. 5 is a flowchart of a method for creating links when a packet is released to walk through a protocol stack.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples.
The invention mainly aims at the characteristic of poor performance of the SD-WAN data tunnel using the UDP protocol to analyze and provide a solution for improving the performance. Since the SD-WAN function implementation of each manufacturer is different, but the kernel data tunnel is forwarded via the protocol stack, the present invention is described in the form of kernel module, and is intended to present the optimization scheme as a whole.
The Sdwan is not accelerated in the normal packet forwarding process, and when only the WAN side is in a dynamic/static state and the PPPOE access mode is considered (VPN dual access is not considered), the tunnel packet forwarding in the network device needs to be processed by multiple IP protocol stack processing, sdwan kernel module processing, and PPP driving processing. Based on the characteristic that the quintuple of the sdwan data tunnel is consistent, the network card multi-queue cannot play a role in the downlink direction of the WAN side of the data packet, the advantage of multi-core forwarding cannot be applied, the time consumption is high in the whole forwarding process in the data packet encryption and decryption process, the performance of the downlink direction is limited, and the forwarding performance of the whole sdwan data tunnel is greatly reduced.
The whole acceleration module of the invention accelerates the data packet forwarding from two directions, and optimizes the multi-core distribution forwarding and protocol stack forwarding processes of the downlink tunnel packet.
As shown in fig. 1, the present invention comprises the steps of:
firstly, after receiving data, storing the data into a hardware queue;
judging whether a packet receiving control technology is opened or not, if so, performing multi-core distribution accelerated forwarding processing on a downlink data packet, identifying and accelerating sdwan data tunnel messages by each CPU through an independent data packet accelerated forwarding processing module, and if not, identifying and accelerating sdwan data tunnel messages through an independent data packet accelerated forwarding processing module;
the identification and acceleration of the sdwan data tunnel message processing process comprises the following steps:
(1) Releasing partial data packets at the upper and lower front parts of each link into a protocol stack, and walking a normal protocol stack flow, wherein an sdwan kernel module creates an acceleration tunnel link and an acceleration tunnel fragment IP link while creating an sdwan tunnel, and creates sdwan links needing accelerated forwarding through the released data packets, wherein the sdwan links comprise wanip links and client links;
(2) Respectively accelerating the subsequent data packets of the link according to the types of the data packets, if the subsequent data packets are uplink packets, matching the client link with the corresponding wanip link to acquire wan Ethernet information sent by the data packets, then sending the data packets according to the acquired information,
and if the packet is a downlink packet, the decrypted packet is directly matched with the client link, the Ethernet information of the uplink direction interface stored in the client link is acquired, and the data packet is sent according to the acquired information.
The acceleration of the protocol stack of the invention depends on the establishment of the uplink and downlink forwarding information links of the data packets, and before the acceleration, a part of the data packets need to be released to establish the links. And in consideration of link tracking creation, sdwan link creation and link tracking timer updating, releasing 3 data packets in the uplink direction of each client link.
As shown in fig. 4, the application layer sends a command to the sdwan kernel module to create a tunnel, where the command includes MAC information of the peer node device, a tunnel encryption mode, and a service IP provided by sdwan; the kernel needs to create corresponding sdwan network equipment, tunnel links and service links provided by sdwan; in this embodiment, the acceleration module creates an acceleration tunnel link while creating a tunnel, and the IP-based acceleration tunnel segments the IP link. Respectively stored in corresponding caches (memory pools).
The accelerated tunnel link sdw _ tunc is created based on the MAC of the end node device, and the link includes MAC information of the node devices at both ends of the tunnel, IP port information, sdwan network device interface MAC information, tunnel encryption information, and MAC layer encapsulation information of the downlink packet. The downstream sdwan packet matches and updates sdw _ tunc MAC layer encapsulation information before decryption, and the upstream packet passed to the sdwan module updates sdw _ tunc port, IP and other information. An IP-based tunnel fragment link is created by tunnel IP (source IP of downstream packet) for matching of downstream fragment packets before decryption.
For the data packet of the UDP tunnel, in order to ensure the consistency of the cache, it is necessary to ensure that the same data stream is forwarded by the same CPU. Because the quintuple of the UDP tunnels is consistent, it is considered that from sdwan networking, the sdwan header carries a hash value of a data stream, the sdwan header is analyzed to obtain the hash value, then the CPU to be matched is calculated according to the hash value, and further the data packets of the same stream are distributed to the same CPU for processing.
Before accelerating the data packet, the ethernet layer encapsulation of the data packet needs to be considered, so this example needs to consider the WAN side access mode, and currently only considers the WAN port dynamic/static state and the PPPOE access mode under the IPV4 protocol for the sdwan tunnel application scenario.
As shown in fig. 3, the processing method for multi-core allocation accelerated forwarding processing includes:
a1: registering a hook point, and redistributing a data packet processing CPU through the hook point;
a2: judging whether the downlink sdwan data packet is a fragment data packet, if not, matching the MAC information carried by the head of the sdwan data packet with the acceleration tunnel link, and if the matching is successful, successfully identifying; if the data packet is matched with the fragment link established by the accelerated tunnel fragment IP link, the successfully matched fragment packet is queued to each fragment queue, the matching of the accelerated tunnel link is carried out through the first fragment packet of each fragment queue to determine whether the data packet is the fragment queue of the sdwan data packet, the matching is failed, the data packet is sent back to the protocol stack for processing, and the data packet is recombined if the matching is successful;
a3: and allocating the processing CPU according to the hash value carried by the sdwan data packet head.
In step a2, if the data packet is a fragmented data packet, the specific processing method is as follows:
a21: the source IP of the fragment data packet is matched with a fragment link established based on the acceleration tunnel, and the IP data packet of the non-acceleration tunnel is filtered; a22: acquiring information of a data packet to initialize a fragmentation queue;
a23: if the data packet is the first fragment, whether the matching acceleration tunnel link is an sdwan data packet indeed or not is judged, if the matching is successful, the information of the data packet is obtained to initialize the fragment queue, the MAC layer information of the downlink data packet of the acceleration tunnel link is updated, then step a24 is executed, and if the matching fails or the data packet is not the first fragment, step a24 is directly executed;
a24: matching the fragment data packet with a fragment queue through self-carried information, marking the fragment queue of the data packet with an sdwan mark, and if the fragment queue does not exist, newly adding the fragment queue;
a25: stripping a pppoe header and a vlan header of all the fragmented data packets, and then enqueuing;
a26: aiming at the sdwan fragmentation queue, receiving the coming fragments, marking the arrival of the first fragment and the arrival of the last fragment, counting the length sum of all fragments, and sequencing the fragmentation data packets;
a27: and c, judging whether the first fragment and the last fragment arrive and the total length of the data packet is equal to the sum of the lengths of all fragment data packets, if so, resetting the data packet, then executing the step a3, and if not, ending the step.
In the embodiment, a hook point is added on the interface of the netif _ receive _ skb _ internal (), and the get _ rps _ CPU () is replaced by the hook point to perform the CPU allocation on the sdwan downlink data packet. Identifying sdwan packets requires matching MAC information carried by sdwan headers with acceleration tunnel links sdw _ tunc, and identifying fragmented packets requires packet reassembly, and in order to avoid processing other fragmented packets before identification, the fragmented packets need to be matched with the fragmented links created by tunnel IP. And the successfully matched fragment packets are queued to each fragment queue, and the first fragment packet of each fragment queue is used for matching tunnel links to determine whether the fragment queue is the sdwan data packet or not, so far, sdwan downlink data packet identification is successful, the fragment queue with failed matching is still sent back to the protocol stack for processing, and data packet reassembly is performed if the identification is successful. And identifying that the successful sdwan data packet needs to strip off the vlan and PPPOE headers, acquiring hash values of different data streams marked on the sdwan headers, and processing the distribution of the CPU.
Based on the characteristic of private construction of an SD-WAN (secure digital-to-Wide area network), the information of the stream to which the data packet belongs is added at the head of the sdwan, so that the head of the downlink data packet sdwan can be obtained and distributed to different CPUs for processing according to the data stream to which the data packet belongs under the condition that the data stream of the UDP tunnel cannot be processed by the RSS/RPS (received signal strength/resilient packet) in a multi-CPU (central processing unit) balance manner, the utilization rate of the CPU is improved, and the downlink forwarding performance is obviously improved.
As shown in fig. 2, in step (1), a client link is created by quintuple information of an upstream packet of the client of sdwan, and the link pointer is mounted under the link trace and deleted as the link trace ages. The client links the sdwan downlink data packet used for acceleration, matches and identifies the sdwan downlink data packet after decryption, and the link records the Ethernet MAC address, vlan and PPPOE header information carried by the uplink sdwan data packet and is used for encapsulating MAC layer information before the downlink packet is sent out. When a client link is created, the wanip link matched with the link is searched and stored (namely, the link is bound with the wanip forwarding the link)
And successfully matching the created client link with the wanip link, judging whether the client link is TCP or UDP, if the client link is TCP, modifying MSS during TCP connection three-way handshake, and if the client link is UDP, creating UDP fragmentation link based on the destination IP for matching uplink fragmentation packets. MSS: TCP is submitted to the IP layer for the maximum segment size, does not contain TCP Header and TCP Option, only contains TCP Payload, and MSS is TCP used for limiting the maximum number of bytes sent by the application layer.
Because the sdwan data packet is encrypted, an sdwan header and a tunnel header are encapsulated, the length of the data packet is increased, and each node of the data packet in the transmission process needs to be fragmented and recombined. By adjusting the MSS connected between the client and the TCP of the sdwan server, the sdwan data packet does not need to be fragmented when being output after being encrypted, fragmentation recombination processing in the forwarding process is avoided, processing time delay on a network path is reduced, and bandwidth competition capability of the sdwan data packet is enhanced.
In step (2), the accelerated packet needs to identify the sdwan packet and then process the sdwan packet. If the packet is an uplink packet, firstly acquiring MAC layer information of the packet, the fragmented packet needs to be linked through IP matching fragmentation, then client side link in the fragmentation link is performed (matching is performed through a first fragmentation packet quintuple), a non-fragmented downlink packet can be directly matched to the client side link through the quintuple, then the MAC layer information of the uplink packet is stored in the client side link, through calculation of the packet passing in the uplink direction of the client side link, the packet is released at the moment, after VLAN/PPPOE header information is removed, an acceleration tunnel link sdw _ tu is obtained through client side connection, then the packet is encrypted by using encryption information in sdw _ tu, then a UDP header and an IP header are packaged, the length of the packaged packet is increased, at the moment, the packet needs to be fragmented, and finally the MAC layer information (VLAN header, PPPOE header, MAC information) of the packet in downlink is acquired through a wanip link in the client side link, the uplink packet is converted, and then the uplink packet is packaged through 3763 zxft _ start _ packet. The specific process of the bag sending treatment is as follows:
b1: acquiring the header information of the data packet, judging whether the data packet is a fragment data packet, if so, executing the step b2, and if not, executing the step b3;
b2: accelerating tunnel fragment IP link matching through a target IP, if the target IP is not the first fragment, executing a step b4, if the target IP is not the first fragment, acquiring a packet quintuple matching client link, updating packet Ethernet header information, updating a vlan/ppp header information to the client link, updating the link tracking aging corresponding to the client connection, and then executing the step b4;
b3: acquiring a data packet quintuple matching client link, updating data packet Ethernet header information, connecting vlan/ppp header information to the client link, updating link tracking aging corresponding to the client link, judging whether the data packet is the uplink front part of the client link, if so, finishing, and if not, executing the step b4;
b4: removing the vlan head and the ppp head;
b5: encrypting a data packet, adding an sdwan header, and encapsulating a UDP header and an IP header;
b6: data packet fragmentation processing;
b7: adding MAC layer information carried by a client link matched with the data packet, and then packaging an uplink data packet;
b8: and calling a packet sending function according to the MAC layer information to send the data packet.
In step (2), if the packet is a downlink packet: the downlink sdwan data packet is marked when the multi-core CPU distributes acceleration, the identification can be directly carried out, the data packet after the identification is successful is matched with wanip link through a target IP, and then an acceleration tunnel link sdw _ tunc is obtained; removing the UDP header of the IP header of the downlink packet, and then decrypting the data packet by using the encryption information in sdw _ tunc; the decrypted data packet needs to consider whether the data packet is a fragment packet or not, the fragment data packet needs to be linked through IP matching fragments, then client side links (matched through a first fragment packet quintuple) in the fragment links are used for directly matching non-fragmented downlink packets to the client side links through the quintuple, the data packet is released through calculation of the data packet passing the downlink direction of the client side links, MAC layer information (vlan header, PPPOE header and MAC information) when the data packet is uplink is stored in the client side links, the data packet is packaged after conversion, and then the data packet is sent out through ndo _ start _ xmit. The specific treatment process comprises the following steps:
c1: matching the wanip link through a destination IP to obtain an acceleration tunnel link;
c2: stripping an IP header and a UDP header of the sdwan downlink packet;
c3: decrypting the data packet, judging whether the decrypted data packet is a fragment data packet, if so, executing the step c4, and if not, executing the step c5;
c4: accelerating tunnel fragment IP link through source IP matching, if not the first fragment, executing step c6, if yes, acquiring a packet quintuple matching client link, and then executing step c6;
c5: acquiring a data packet quintuple matching client link, then judging whether the data packet is a data packet of a downlink front part of the client link, if so, updating sdwan downlink IP link, ending, if not, carrying out fragmentation processing on the data packet, and then executing the step c6;
c6: adding MAC layer information carried by a client link matched with the data packet, and then packaging a downlink data packet;
c7: calling a packet sending function according to the MAC layer information to send the data packet
In the uplink data packet of sdwan in this example, after sdwan is encrypted, an sdwan data header, a UDP header, and an IP header are added, the length of the data packet is increased, and the sdwan data packet exiting from a WAN port needs to be fragmented, which causes fragmentation and reassembly in the entire transmission network, increases processing delay of sdwan tunnel packets, and causes performance degradation. By modifying the MSS during the three-way handshake of each TCP connection, the data packets connected with the TCP can be ensured not to be subjected to fragmentation and reassembly after being encrypted and sent out by the tunnel, and the forwarding performance is improved. When the client link is created, a UDP fragmentation link based on a destination IP (sdwan service IP address) in the uplink direction is created at the same time and is used for matching the uplink fragmentation packets.
According to the invention, the single kernel module takes over sdwan data packets for packet forwarding, compared with normal PPP drive (PPPOE access) processing, the protocol stack forwards the sdwan data packets for multiple times, so that the flow is simpler, routing query does not need to be carried out on all data packets, and a large amount of legality detection actions are reduced.
As shown in fig. 2 and fig. 5, the uplink packet matches the client link, but how to obtain the MAC layer packet information sent from the WAN is a difficult point to be solved. When the acceleration tunnel link is created, the MAC layer information of the downlink packet is created by the sdwan downlink packet and stored in the acceleration tunnel link, and the sdwan data packet of the uplink client needs to obtain the information of the acceleration tunnel link by means of the wanip link forwarded by the data packet. When the uplink packet updates the tunnel link, a link based on wanip is created, and the acceleration tunnel link is added to the structure of the wanip link, so that the uplink packet can acquire the MAC layer information to be encapsulated when being forwarded from the WAN port through the client link (the wanip link is stored in the client link).
Since the MTU driving the dev taken by the downstream packet is not necessarily the minimum MTU of the local forwarding path interface, such as the PPPOE access scenario (the MTU taken from the downstream packet is the MTU of the real WAN port, but the MTU when the packet is forwarded out is the MTU of the PPPOE interface), the MTU-to-wanip link needs to be updated at this time.
The key points and difficulties for realizing accelerated forwarding of the invention are as follows:
the invention accelerates UDP data tunnel of sdwan, which is characterized in that the invention constructs the connection of data packet forwarding between network card receiving interface, sdwan interface and sending interface, and the difficulty is that the invention processes the fragment data packet in the downlink and uplink of the data packet, including the process of receiving the fragment data packet from the driver in the uplink direction, and the fragment process of the data packet after encryption; the downstream reverse slave driver receives the fragment reorganization processing of the fragment packets, and the fragment packet processing is found after the data packet decryption, and the processing logic of the fragment packets is complex and the processing difficulty is high.
The specific treatment principle is as follows:
1. the accelerated data packet needs to encapsulate ethernet information of the transmission interface when being transmitted, so the link matched with the accelerated packet needs to contain the information. For a downlink acceleration packet, the data packet can be directly matched with the client link after being decrypted, and the Ethernet information of the uplink direction interface stored in the client link is obtained; for the uplink data packet, not only the client link but also the corresponding wanip link need to be matched, so as to obtain the wan port ethernet information sent out by the data packet. It is particularly important that the corresponding client link (corresponding data stream) matches the wan port (wanip link) sent when creating the client link and wanip link.
2. The fragmentation data packet is a processing difficulty in the acceleration process of sdwan, firstly, a fragmentation link (matching a link of an IP) and a fragmentation queue (a fragmentation queue constructed based on all fragments of a single data packet) which need to be created are complex in structure, and since a non-first fragmentation data packet does not carry header information of a transmission layer and a data part, when sdwan header information matching and link information matching are carried out, the part of data packet needs to be subjected to enqueue processing before matching, and after the matching information is obtained through the first fragmentation packet, other fragmentation packets can be matched with the response information. In addition, the processing in different directions and different stages is considered, the processing of the queued fragment packets is different, the downlink fragment packets need to be recombined before decryption, the decryption processing can be carried out after the recombination, and the decrypted fragment packets need to be packaged and then sent after being queued; the fragmented data packet in the uplink direction needs to be queued before encryption, encrypted after matching with the client link information, and whether fragmentation processing is performed again needs to be considered after encryption. The processing of the block can refer to the processing method of the kernel fragment packets, including data structures, processing after the fragment packets arrive in the fragment queue, aging and deleting of the fragment queue, and the like, but the final processing results of the fragment packets in different stages are different.
The above-described embodiments are intended to be illustrative, and not restrictive, of the invention, and all changes that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Claims (10)
1. A method for realizing the forwarding acceleration of an sdwan data tunnel is characterized by comprising the following steps:
firstly, after receiving data, storing the data into a hardware queue;
judging whether a packet receiving control technology is opened or not, if so, performing multi-core distribution accelerated forwarding processing on a downlink data packet, identifying and accelerating sdwan data tunnel messages by each CPU through an independent data packet accelerated forwarding processing module, and if not, identifying and accelerating sdwan data tunnel messages through an independent data packet accelerated forwarding processing module;
the identification and acceleration of the sdwan data tunnel message processing process comprises the following steps:
(1) Releasing the data packet of the front part of the uplink and the downlink of each link into a protocol stack, and walking a normal protocol stack flow, wherein an sdwan kernel module creates an acceleration tunnel link and an acceleration tunnel fragment IP link while creating an sdwan tunnel, and creates each sdwan link needing to be accelerated and forwarded through the released data packet, wherein each sdwan link comprises a wanip link and a client link, the acceleration tunnel link is created based on the MAC of opposite-end node equipment, and the link comprises the MAC information, the IP port information, the MAC information of an sdwan network equipment interface, tunnel encryption information and the MAC layer encapsulation information of a downlink packet;
(2) Respectively accelerating the subsequent data packets of the link according to the types of the data packets, if the subsequent data packets are uplink packets, matching the client link with the corresponding wanip link to acquire wan Ethernet information sent by the data packets, then sending the data packets according to the acquired information,
and if the data packet is the downlink packet, directly matching the client link after decryption, acquiring the Ethernet information of the uplink direction interface stored in the client link, and sending the data packet according to the acquired information.
2. The method of claim 1 for implementing sdwan data tunnel forwarding acceleration, wherein: the processing method for multi-core distribution accelerated forwarding processing comprises the following steps:
a1: registering a hook point, and redistributing a data packet processing CPU through the hook point;
a2: judging whether the downlink sdwan data packet is a fragment data packet, if not, matching the MAC information carried by the head of the sdwan data packet with the acceleration tunnel link, and if the matching is successful, successfully identifying; if the data packet is matched with the fragment link established by the accelerated tunnel fragment IP link, the successfully matched fragment packet is queued to each fragment queue, the matching of the accelerated tunnel link is carried out through the first fragment packet of each fragment queue to determine whether the data packet is the fragment queue of the sdwan data packet, the matching is failed, the data packet is sent back to the protocol stack for processing, and the data packet is recombined if the matching is successful;
a3: and allocating the processing CPU according to the hash value carried by the sdwan data packet head.
3. The method of claim 2 for implementing sdwan data tunnel forwarding acceleration, wherein: in step a2, if the data packet is a fragmented data packet, the specific processing method is as follows:
a21: the source IP of the fragment data packet is matched with a fragment link established based on the acceleration tunnel, and the IP data packet of the non-acceleration tunnel is filtered;
a22: acquiring information of a data packet to initialize a fragmentation queue;
a23: if the data packet is the first fragment, determining whether the matching acceleration tunnel link is indeed an sdwan data packet, if the matching is successful, acquiring information of the data packet to initialize a fragment queue, updating MAC (media access control) layer information of a downlink data packet of the acceleration tunnel link, and then executing a step a24, if the matching fails or is not the first fragment, directly executing the step a24;
a24: matching the fragment data packet with a fragment queue through self-carried information, marking the fragment queue of the data packet with an sdwan mark, and if the fragment queue does not exist, newly adding the fragment queue;
a25: stripping a pppoe header and a vlan header of all the fragmented data packets, and then enqueuing;
a26: aiming at the sdwan fragmentation queue, receiving the coming fragments, marking the arrival of the first fragment and the last fragment, counting the length sum of all fragments, and sequencing the fragment data packets;
a27: and c, judging whether the first fragment and the last fragment arrive and the total length of the data packet is equal to the sum of the lengths of all the fragment data packets, if so, repeating the data packet, then executing the step a3, and if not, finishing.
4. The method for implementing sdwan data tunnel forwarding acceleration according to claim 1, wherein: in the step (1), a client link is created through quintuple information of an uplink data packet of an sdwan client, a link pointer is mounted under link tracking and deleted along with link tracking aging, the client link is used for accelerating matching and identification of an sdwan downlink data packet after decryption, the link records Ethernet MAC address, vlan and PPPOE header information carried by the uplink sdwan data packet and is used for encapsulating MAC layer information before the downlink packet is sent out, and a wanip link matched with the link is searched and stored when the client link is created.
5. The method of implementing sdwan data tunnel forwarding acceleration according to claim 4, wherein: and successfully matching the created client link with the wanip link, judging whether the client link is TCP or UDP, if the client link is TCP, modifying MSS during three-way handshake of TCP connection, and if the client link is UDP, creating UDP fragmentation link based on the destination IP for matching uplink fragmentation packets.
6. The method of claim 4, wherein the sdwan data tunnel forwarding acceleration is realized by: in the step (2), if the packet is an uplink packet, the specific packet sending processing process is as follows:
b1: acquiring the header information of the data packet, judging whether the data packet is a fragment data packet, if so, executing the step b2, and if not, executing the step b3;
b2: accelerating tunnel fragment IP link matching through a target IP, if the target IP is not the first fragment, executing a step b4, if the target IP is not the first fragment, acquiring a packet quintuple matching client link, updating packet Ethernet header information, updating a vlan/ppp header information to the client link, updating the link tracking aging corresponding to the client connection, and then executing the step b4;
b3: acquiring a packet quintuple matching client link, updating the Ethernet header information of the packet, sending vlan/ppp header information to the client link, updating the link tracking aging corresponding to the client link, judging whether the packet is the packet of the uplink front part of the client link, if so, finishing, and if not, executing the step b4;
b4: removing the vlan head and the ppp head;
b5: encrypting a data packet, adding an sdwan header, and encapsulating a UDP (user Datagram protocol) header and an IP header;
b6: data packet fragmentation processing;
b7: adding MAC layer information carried by a client link matched with the data packet, and then packaging an uplink data packet;
b8: and calling a packet sending function according to the MAC layer information to send the data packet.
7. The method of claim 4, wherein the sdwan data tunnel forwarding acceleration is realized by: in step (2), if the data packet is a downlink data packet, the downlink sdwan data packet is marked when the multi-core distribution accelerated forwarding processing is performed, and the identification can be directly performed, and the processing process included in the successfully identified data is as follows:
c1: matching the wanip link through the destination IP to further obtain an accelerated tunnel link;
c2: stripping an IP (Internet protocol) header and a UDP (user Datagram protocol) header of the sdwan downlink packet;
c3: decrypting the data packet, judging whether the decrypted data packet is a fragment data packet, if so, executing the step c4, and if not, executing the step c5;
c4: accelerating tunnel fragment IP link through source IP matching, if not the first fragment, executing step c6, if yes, acquiring a packet quintuple matching client link, and then executing step c6;
c5: acquiring a data packet quintuple matching client link, then judging whether the data packet is a data packet of a downlink front part of the client link, if so, updating sdwan downlink IP link, ending, if not, carrying out fragmentation processing on the data packet, and then executing the step c6;
c6: adding MAC layer information carried by a client link matched with the data packet, and then packaging a downlink data packet;
c7: and calling a packet sending function according to the MAC layer information to send the data packet.
8. A system for implementing sdwan data tunnel forwarding acceleration, configured to implement the method for implementing sdwan data tunnel forwarding acceleration in any one of claims 1 to 7, comprising:
an enqueue module: the device is used for storing the data into a hardware queue after receiving the data;
receive package control technology and open the module: the method is used for opening a packet receiving control technology to realize multi-core distribution accelerated forwarding;
the multi-core distribution accelerated forwarding module: the system is used for carrying out multi-core distribution accelerated forwarding processing on the downlink data packets;
the accelerated forwarding processing module: the method is used for identifying uplink and downlink data packets and accelerating sdwan data tunnel messages;
a creation module: the system is used for creating an sdwan tunnel, an acceleration tunnel link and an acceleration tunnel fragment IP link, and creating sdwan links needing accelerated forwarding through released data packets;
a storage module: the link is used for storing various links created by the creation module;
a bag sending module: the function is used for calling the packet sending function and sending the data packet.
9. The system of claim 8, wherein: the storage module comprises an acceleration tunnel link memory pool, an acceleration tunnel fragment IP link memory pool, a wanip link memory pool and a client link memory pool.
10. The system of claim 8, wherein: the multi-core distribution accelerated forwarding processing module comprises:
a registration module: the method is used for registering a hook point, replacing an original queue distribution function through the hook point and reallocating a data packet processing CPU;
a judgment module: the system is used for judging whether the downlink sdwan data packet is a fragment data packet or not;
a matching identification module: aiming at the non-fragmented data, the method is used for matching MAC information carried by the head of the sdwan data packet with the accelerated tunnel link, and if the matching is successful, the identification is successful; aiming at the fragmented data packets, firstly matching the fragmented links created by the accelerated tunnel fragmented IP links, queuing the fragmented packets after successful matching to each fragmented queue, carrying out matching of the accelerated tunnel links through the first fragmented packet of each fragmented queue to confirm whether the fragmented packets are the fragmented queues of the sdwan data packets, sending the data packets back to the protocol stack for processing if the matching is failed, and sending the data packets into the data packet recombination module for processing if the matching is successful;
a data packet reorganization module: the device is used for recombining the data packets;
a distribution module: and the method is used for allocating the processing CPU according to the hash value carried by the sdwan data packet head.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111576110.9A CN114268518B (en) | 2021-12-21 | 2021-12-21 | Method and system for realizing forwarding acceleration of sdwan data tunnel |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111576110.9A CN114268518B (en) | 2021-12-21 | 2021-12-21 | Method and system for realizing forwarding acceleration of sdwan data tunnel |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114268518A CN114268518A (en) | 2022-04-01 |
| CN114268518B true CN114268518B (en) | 2023-04-07 |
Family
ID=80828518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111576110.9A Active CN114268518B (en) | 2021-12-21 | 2021-12-21 | Method and system for realizing forwarding acceleration of sdwan data tunnel |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114268518B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115442183B (en) * | 2022-08-02 | 2024-01-02 | 天翼云科技有限公司 | Data forwarding method and device |
| CN116016035B (en) * | 2023-02-16 | 2023-06-13 | 北京天维信通科技有限公司 | Method for applying multipath service in same tunnel by utilizing quintuple |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101938415A (en) * | 2010-08-30 | 2011-01-05 | 北京傲天动联技术有限公司 | Rapid forwarding method for network forwarding device |
| CN103475586A (en) * | 2013-08-22 | 2013-12-25 | 东软集团股份有限公司 | Method, device and system for forwarding network data messages |
| CN107659515A (en) * | 2017-09-29 | 2018-02-02 | 曙光信息产业(北京)有限公司 | Message processing method, device, message processing chip and server |
| CN108833548A (en) * | 2018-06-20 | 2018-11-16 | 中国联合网络通信集团有限公司 | SD-WAN network system and for network flow accelerate optimization method |
| US10212089B1 (en) * | 2017-09-21 | 2019-02-19 | Citrix Systems, Inc. | Encapsulating traffic entropy into virtual WAN overlay for better load balancing |
| CN111614538A (en) * | 2020-04-30 | 2020-09-01 | 网络通信与安全紫金山实验室 | Message forwarding method based on IPsec encapsulation protocol |
-
2021
- 2021-12-21 CN CN202111576110.9A patent/CN114268518B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101938415A (en) * | 2010-08-30 | 2011-01-05 | 北京傲天动联技术有限公司 | Rapid forwarding method for network forwarding device |
| CN103475586A (en) * | 2013-08-22 | 2013-12-25 | 东软集团股份有限公司 | Method, device and system for forwarding network data messages |
| US10212089B1 (en) * | 2017-09-21 | 2019-02-19 | Citrix Systems, Inc. | Encapsulating traffic entropy into virtual WAN overlay for better load balancing |
| CN107659515A (en) * | 2017-09-29 | 2018-02-02 | 曙光信息产业(北京)有限公司 | Message processing method, device, message processing chip and server |
| CN108833548A (en) * | 2018-06-20 | 2018-11-16 | 中国联合网络通信集团有限公司 | SD-WAN network system and for network flow accelerate optimization method |
| CN111614538A (en) * | 2020-04-30 | 2020-09-01 | 网络通信与安全紫金山实验室 | Message forwarding method based on IPsec encapsulation protocol |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114268518A (en) | 2022-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11178262B2 (en) | Fabric control protocol for data center networks with packet spraying over multiple alternate data paths | |
| US9781052B2 (en) | Virtual machine and application movement over local area networks and a wide area network | |
| US10749752B2 (en) | Methods and systems for managing VPN tunnels | |
| US20180288179A1 (en) | Proxy for serving internet-of-things (iot) devices | |
| US8175116B2 (en) | Multiprocessor system for aggregation or concatenation of packets | |
| US6157649A (en) | Method and system for coordination and control of data streams that terminate at different termination units using virtual tunneling | |
| CN114268518B (en) | Method and system for realizing forwarding acceleration of sdwan data tunnel | |
| US20130286840A1 (en) | Method And System For Offloading Tunnel Packet Processing In Cloud Computing | |
| JP2019528604A (en) | System and method for virtual multipath data transport | |
| WO2020063298A1 (en) | Method for processing tcp message, toe assembly, and network device | |
| WO2021037216A1 (en) | Message transmission method and device, and computer storage medium | |
| US20140294018A1 (en) | Protocol for layer two multiple network links tunnelling | |
| US9445384B2 (en) | Mobile device to generate multiple maximum transfer units and data transfer method | |
| US8355405B2 (en) | Selective session interception method | |
| EP3119057A1 (en) | Packet conversion device and method for allowing transparent packet-based multipath bundling | |
| KR100748698B1 (en) | Apparatus and method of packet processing in security communication system | |
| WO2023151264A1 (en) | Load balancing method and apparatus, node, and storage medium | |
| CN113395212B (en) | Network device, method of operating the same, and non-transitory computer readable medium | |
| CN108282391B (en) | VXLAN message fragmentation method and device | |
| Seggelmann et al. | SSH over SCTP—Optimizing a multi-channel protocol by adapting it to SCTP | |
| CN108064441B (en) | Method and system for accelerating network transmission optimization | |
| CN113965518A (en) | Message processing method and device | |
| JP2002026927A (en) | Capsulating method and unit, and program recording medium | |
| CN110601950B (en) | VPN gateway system based on DTLS protocol and implementation method | |
| WO2006064561A1 (en) | Virtual private network system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |