WO2014064486A1 - Procédé, dispositif et système de prioritisation de paquets d'encapsulage dans plusieurs connexions réseau logiques - Google Patents

Procédé, dispositif et système de prioritisation de paquets d'encapsulage dans plusieurs connexions réseau logiques Download PDF

Info

Publication number
WO2014064486A1
WO2014064486A1 PCT/IB2012/055862 IB2012055862W WO2014064486A1 WO 2014064486 A1 WO2014064486 A1 WO 2014064486A1 IB 2012055862 W IB2012055862 W IB 2012055862W WO 2014064486 A1 WO2014064486 A1 WO 2014064486A1
Authority
WO
WIPO (PCT)
Prior art keywords
packet
sequence number
priority
encapsulating
packets
Prior art date
Application number
PCT/IB2012/055862
Other languages
English (en)
Inventor
Alex Wing Hong CHAN
Ho Ming Chan
Patrick Ho Wai SUNG
Original Assignee
Pismo Labs Technology Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pismo Labs Technology Limited filed Critical Pismo Labs Technology Limited
Priority to PCT/IB2012/055862 priority Critical patent/WO2014064486A1/fr
Priority to CN201280076668.3A priority patent/CN104956637B/zh
Priority to GB1415039.5A priority patent/GB2514060B/en
Priority to US14/003,237 priority patent/US9369398B2/en
Publication of WO2014064486A1 publication Critical patent/WO2014064486A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H04L47/62—Queue scheduling characterised by scheduling criteria
    • H04L47/624—Altering the ordering of packets in an individual queue
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/20—Support for services
    • H04L49/205—Quality of Service based
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/302—Route determination based on requested QoS
    • H04L45/308—Route determination based on user's profile, e.g. premium users
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/22—Alternate routing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/74—Address processing for routing
    • H04L45/741—Routing in networks with a plurality of addressing schemes, e.g. with both IPv4 and IPv6
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/34—Flow control; Congestion control ensuring sequence integrity, e.g. using sequence numbers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/30—Peripheral units, e.g. input or output ports
    • H04L49/3009—Header conversion, routing tables or routing tags
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1867—Arrangements specially adapted for the transmitter end
    • H04L1/1887—Scheduling and prioritising arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2212/00—Encapsulation of packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/302—Route determination based on requested QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2408—Traffic characterised by specific attributes, e.g. priority or QoS for supporting different services, e.g. a differentiated services [DiffServ] type of service
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H04L47/62—Queue scheduling characterised by scheduling criteria
    • H04L47/6215—Individual queue per QOS, rate or priority
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H04L47/62—Queue scheduling characterised by scheduling criteria
    • H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
    • H04L47/6275—Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority

Definitions

  • the invention relates generally to transmitting and receiving encapsulating packets via a plurality of logical network connections, more particularly, to allow prioritization of encapsulating packets in the plurality of logical network connections.
  • encapsulating packet header can be used to differentiate an encapsulating packet in order to allow a network device to treat the packet with high priority or more importance.
  • Packet marking technique to identify packets may include manipulation of the Differentiated Services Code Point (DSCP) sub-field of the Internet Protocol (IP) header Type of Service (TOS) field.
  • DSCP Differentiated Services Code Point
  • IP Internet Protocol
  • TOS Type of Service
  • DiffServ may provide a networking architecture for classification and management of network traffic as well as Quality of Service (QoS) mechanisms.
  • the DiffServ field may be used in a network layer protocol (e.g., IPv4, IPv6 and mobile IPv6) to make per-hop behavior (PHB) decisions about packet classification and traffic conditioning functions, such as metering, marking, shaping and policing.
  • the DSCP field is within the Type of Service (ToS) field of the encapsulating packet header.
  • the DSCP field may provide an indication of the abstract parameters of the quality of service desired. These parameters may be used to guide the selection of actual service parameters when transmitting a packet through a particular network.
  • Several networks offer service precedence, which may treat high priority traffic as more important than other traffic (generally by accepting only traffic above certain precedence at time of high load).
  • the standardized DiffServ field of the packet may be marked with a value so that the packet receives a particular forwarding treatment or PHB, at each network node.
  • the encapsulating packets may be received by the receiver not in sequential order. It is known to those skilled in the art that packets decapsulated from the encapsulating packets should be sorted before being transmitted to the designated recipients. It is also known to those skilled in the art that the sorting can be carried out by using a global sequence number and per logical network connection sequence number encapsulated in the encapsulating packets along with the packets. However, sorting packets can result in delay of transmitting packets when the global sequence numbers are not in sequential order at the receiving network device. When the transmissions of packets with higher priority are delayed due to sorting, the impact of such delay is larger than those of packets with lower priority.
  • the encapsulating packet may have a packet format of a usual IPv4 header shown in FIG. 2. It is to be noted that in the above description, it is assumed that the IPv4 header shown in FIG. 2 is used. However, the IPv4 header does not necessarily need to be used, and a packet format of another arbitrary protocol, including IPv6 and mobile IPv6, may be used.
  • Prioritizing packets into different queues at a network device Prioritizing packets into different queues at a network device.
  • the network device selects one of a plurality of logical network connections to transmit the packet.
  • a global sequence number and a priority sequence number are embedded along with the packet in an embedding packet. Packets with the highest priority are transmitted earlier than packets with lower priority.
  • At the receiving network device there is one queue for storing the received embedding packets.
  • the receiving network device determines whether to store the embedded packets or to transmit the packets embedded in the embedding packet to the intended recipient based on the global sequence number and a priority sequence number.
  • the present invention has been made in view of the above circumstances, and an objective of the invention is to use priority queues, priority level and priority sequence numbers to solve the above-described problem.
  • a host can be a computing device, a laptop computer, a mobile phone, a smart-phone, a desktop computer, a switch, a router or an electronic device that is capable of transmitting and receiving packets.
  • a transmitting host is a host transmitting a packet.
  • a transmitting host can also be a network device receiving packets from a host and then transmitting the packets according to policies and/or determined routes.
  • a receiving host is a host receiving a packet.
  • a receiving host can also be a network device receiving packets from a host and then transmitting the packets according to policies and/or determined routes. Therefore, a host can be a transmitting host and a receiving host.
  • the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged.
  • a process is terminated when its operations are completed, but could have additional steps not included in the figure.
  • a process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
  • the term “storage medium” or ⁇ computer readable storage medium ⁇ may represent one or more devices for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, Flash, non-volatile core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information.
  • ROM read only memory
  • RAM random access memory
  • Flash non-volatile core memory
  • magnetic disk storage mediums magnetic disk storage mediums
  • optical storage mediums flash memory devices and/or other machine readable mediums for storing information.
  • computer readable storage medium includes, but is not limited to portable or fixed storage devices, optical storage devices, wireless channels and various other mediums capable of storing, containing or carrying instruction(s) and/or data.
  • a queue may be implemented by a section or a plurality sections in a storage medium.
  • embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof.
  • the program code or code segments to perform the necessary tasks may be stored in a machine readable medium such as storage medium.
  • a processing unit(s) may perform the necessary tasks.
  • a processing unit(s) by a CPU, an ASIC semiconductor chip, a semiconductor chip, a logical unit, a digital processor, an analog processor, a FPGA or any processor that is capable of performing logical and arithmetic functions.
  • a code segment may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements.
  • a code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
  • a network device is capable of transmitting out encapsulating packets belonging to an aggregated logical network connection and receiving encapsulating packets belonging to the aggregated logical network connection.
  • the network device when the network device transmits encapsulating packets, the network device is defined as a VPN Sender Device and when the network device receives encapsulating packets belonging to an aggregated logical network connection, the network device is defined as a VPN Receiver Device. Therefore, network device 103 and 105 can be VPN Sender Device and VPN Receiver Device at the same time.
  • FIG. 1 illustrates a network environment in which two networks can be connected together via a plurality of logical network connections.
  • the same network environment can also be employed to connect three or more networks via a plurality of logical network connections.
  • Layer 2 network protocols that can be employed in the present invention includes Ethernet, Token Ring, Frame Relay, PPP, X.25 and ATM.
  • Layer 3 network protocols that can be employed in the present invention include Internet Protocol (IP) version 4, IPv6, mobile IPv4, mobile IPv6, or the like.
  • IP Internet Protocol
  • Computing devices 101a, 101b and 101c are connected to switch 107 and are in the same network.
  • Computing devices 102a, 102b and 102c are connected to switch 106 and are in another network.
  • a virtual private network has to be established among computing devices 101a, 101b, 101c, 102a, 102b and 102c.
  • Network device 103 and network device 105 can be routers.
  • Network device 103 and network device 105 together establish a logical network connection through the Internet 104 between switch 107 and switch 106.
  • a switch may be combined with a router to form a networking device capable of connecting a plurality of networking devices and/or computing devices and form one or more VPNs.
  • switch 107 can be combined with network device 103.
  • computing device 102a When computing device 102a transmits a packet to computing device 101a, computing device 102a transmits the packet to switch 106. Switch 106 then transmits the packet to network device 105 via network link 121c. Network device 105 encapsulates the packet into one or more encapsulating packets, and then transmits the one or more encapsulating packets through Internet 104 using one of a plurality of logical network connections to network device 103.
  • An aggregated logical network connection is formed by connecting networks through aggregating the plurality of logical network connection.
  • a logical network connection can use connection-oriented protocol, such as Transmission Control Protocol (TCP), or a connectionless protocol, such as User Datagram Protocol (UDP), to transmit encapsulating packets.
  • TCP Transmission Control Protocol
  • UDP User Datagram Protocol
  • a logical network connection is also known as a tunnel.
  • a logical network connection between network device 103 and network device 105 is established by using one of network access links 120a, 120b and 120c, one of network access links 121a and 121b, and the Internet 104.
  • Network access links 120a, 120b, and 120c are connected to network device 103 through three network interfaces respectively.
  • network access links 121a and 121b are connected to network device 105 through two network interfaces respectively.
  • a network access link is in form of optical fiber, Ethernet, ATM, Frame Relay, T1/E1, IPv4, IPv6, wireless technologies, Wi-Fi, WiMax, High-Speed Packet Access technology, 3GPP Long Term Evolution (LTE) or the like.
  • an aggregated logical network connection packets belonging to a session, a presentation, or an application may be transmitted and received via different logical network connections of the aggregated logical network connection. From the perspective of a session, a presentation, or an application, the plurality of logical network connections act like a single logical network connection.
  • An aggregated logical network connection is also known as aggregated tunnels or bonded VPN.
  • network device 103 When network device 103 receives the one or more encapsulating packets from network device 105 via logical network connections in the aggregated logical network connection, network device 103 decapsulates the one or more encapsulating packets to retrieve the packet and then transmits the data back to a designated computing device, i.e. computing device 101a via network link 120d and switch 107.
  • a designated computing device i.e. computing device 101a via network link 120d and switch 107.
  • FIG. 3 is an illustration of a format of data field 203 of FIG. 2 belonging to an encapsulating packet and payload according to one of the embodiments of the present invention.
  • An encapsulating packet is an encapsulating packet encapsulating a packet which is either a layer 2 packet or a layer 3 packet.
  • the packet is encapsulated in data field 301.
  • Global sequence number (GSEQ) field 305, logical network connection sequence number (TSEQ) field 304, priority sequence number (PSEQ) field 303, other options field 302 and data field 301 together form the payload of an encapsulating packet, which is data field 203.
  • Priority level of the packet is also stored in the payload in the encapsulating packet and can be represented by two or more bits.
  • length of the priority level is represented by three bits and therefore can accommodate up to eight different priorities.
  • the length of the priority level is represented by one eight-bit byte and therefore can accommodate up to two hundred and fifty-six different priorities.
  • the length of the priority level is represented by sixteen bits and therefore can accommodate up to sixty five thousand five hundred and thirty five different priorities.
  • Destination address of the encapsulating packet can be the IP address of the VPN Receiver Device.
  • Source address of the encapsulating packet is one of the IP addresses of the VPN Sender Device.
  • the formats of IP address for IPv4 and IPv6 follow IETF RFC 791 and IETF RFC 2460 respectively.
  • Encapsulating packet format follows IPv4 description in IETF RFC 791, IPv6 description in IETF RFC 2460, and different IETF RFC documents.
  • headers of an encapsulating packet contains version, IHL, type of service, total length, identification, flags, fragment offset, time to live, protocol and header checksum.
  • GSEQ GSEQ
  • TSEQ TSEQ
  • PSEQ Packet Sequence Protocol
  • GSEQ field 305 stores GSEQ, which is used to indicate sequence of packets received by the network device.
  • a GSEQ is unique during the life of the aggregated logical network connection unless the GSEQ is larger than the value which GSEQ field 305 can hold and in that case, the GSEQ will restart from zero.
  • sequence of encapsulating packets leaving the VPN Sender Device may not follow the GSEQ.
  • an encapsulating packet transmitted by the VPN Sender Device to the VPN Receiver Device earlier has a lower GSEQ than another encapsulating packet transmitted by the VPN Sender Device to the VPN Receiver Device later to indicate the sequence of packets leaving the VPN Sender Device.
  • a first encapsulating packet leaving the VPN Sender Device earlier may have a higher GSEQ than the GSEQ of a second encapsulating packet leaving the VPN Sender Device later even though the packet encapsulated in the first encapsulating packet arrived at the VPN Sender Device later than the packet encapsulated in the second encapsulating packet.
  • TSEQ field 304 holds TSEQ, which is used to indicate a sequence of packets transmitted by a VPN Sender Device, transmitted via a logical network connection.
  • a TSEQ is unique during the life of a logical network connection unless the TSEQ is larger than the value that TSEQ field 304 can hold and in that case, the TSEQ will restart from zero.
  • the encapsulating packet is placed at a queue of the logical network connection.
  • encapsulating packets in a logical network connection queue are transmitted by the VPN Sender Device sequentially according to the TSEQ.
  • the number of logical network connection queues corresponds to the number of logical network connections. For example, if there are five logical network connections in a VPN, there are five logical network connection queues.
  • the TSEQs of encapsulating packets in each logical network connection should be in order when the VPN Sender Device channel.
  • encapsulating packets in a logical network connection queue are transmitted by the VPN Sender Device sequentially according to the TSEQ and the priority level and therefore the TSEQs of encapsulating packets leaving the VPN Sender Device may not be in order. Therefore, in this embodiment, if there are five logical network connections in a VPN, there are five logical network connection queues.
  • the TSEQs of encapsulating packets in each logical network connection may not be in order when leaving each corresponding channel because encapsulating packets with higher priority packet may be transmitted earlier in a logical network connection.
  • Packet with higher priority should be transmitted sooner than packet with lower priority.
  • Those skilled in the art may have different techniques to indicate priority for a packet.
  • PSEQ field 303 holds PSEQ, which is used to indicate the sequence of packets having the same priority.
  • the PSEQ of each priority level is unique during the life of an aggregated plurality of logical network connection unless the PSEQ is larger than the value that PSEQ field 303 can hold and in that case, the PSEQ will restart from zero.
  • the lengths of PSEQ field 303, TSEQ field 304 and GSEQ file 305 do not need to be the same and are at least eight bits long.
  • the number of bits used to represent PSEQ, TSEQ and GSEQ are thirty-two bits and therefore the length of PSEQ field 305, TSEQ field 304 and GSEQ field 305 are thirty-two bits long.
  • options field 302 is optional. In one variant, there is no other options field 302 in one of the embodiments of the present invention. In one of the embodiments of the present invention, there is other options field 302 to store information that can assist the VPN Sender Device and/or VPN Receiver Device to process the encapsulating packet. For example, other options field 302 may store network information, latency information, error correction information, authentication information, encryption information, and etc.
  • GSEQ, TSEQ, PSEQ and other options are placed in an extension header and are placed after the IPv6 header. In one of the embodiments of the present invention, GSEQ, TSEQ, PSEQ and other options are placed in the payload of an IPv6 packet.
  • a packet with smaller GSEQ should be received by a VPN Sender Device earlier than a packet with a larger GSEQ.
  • the opposite is true. Therefore a packet with smaller GSEQ is received by a VPN Sender Device later than a packet with a larger a GSEQ.
  • the VPN Sender Device is consistent in how GSEQ is set according to the chronology of packet arrival and the VPN Receiver Device is aware of the setting.
  • the same also applies to TSEQ, PSEQ, and priority level that as long as the VPN Sender Device uses a consistent scheme to set values of TSEQ, PSEQ, and priority level and the VPN Receiver Device is aware of the consistent way to set the values.
  • Those skilled in the arts have many different schemes to set GSEQ, TSEQ, PSEQ, and priority level.
  • FIG. 5, which should be viewed in conjunction with FIG. 1 and FIG. 4, is a flow-chart illustrating a process in which a VPN Sender Device transmits an encapsulating packet.
  • FIG. 4 illustrates the relationship among packets received by network device 105, which is the VPN Sender Device in this illustration, priority queues 401, 402 and 403, logical network connections 421, 431 and 432, and network access links 420 and 430 according to one of the embodiments of the present invention.
  • Network device 105 receives packets from switch 106, via a first network interface of network device 105.
  • Switch 106 receives the packets from one of computing devices 102a, 102b and 102c.
  • the packets are intended to be transmitted by network device 105 to network device 103, which is a VPN Receiver Device in this illustration.
  • Network access links 420 and 430 are network access links 121a and 121b respectively in this illustration.
  • Priority queues 401, 402 and 403 are three queues belonging to different priorities. For example, if there are three priorities, priority queue 401 is a queue for the highest priority, priority queue 402 is a queue for the middle priority, and priority queue 403 is a queue for the lowest priority. For example, if there are sixteen priorities, there are sixteen priority queues.
  • Priority queue may be an array, a linked list, a tree or other kinds of data structure that can be implemented in a computer readable storage medium. A priority queue is able to hold data, including a plurality of packets or encapsulating packets. Those skilled in the art should appreciate that there are many known techniques to implement queues. For example, different priority queues can be implemented by a single queue with different indices, by a common array, by a single memory unit, etc. Packets are assigned with PSEQ according to the priority queues in which the packets are stored.
  • Network access link 420 has one logical network connection 421 established inside.
  • Network access link 430 has two logical network connections 431 and 432 established inside.
  • Logical network connections 421, 431 and 432 are aggregated together to form one logical network connection from computing devices ⁇ perspectives. For example, each Ethernet packet originating from computing device 102a may be received by computing device 101a via any of the logical network connections 421, 431 or 432.
  • each encapsulating packet belonging to a TCP session originating from computing device 102c may be received by computing device 101b via logical network connections 421, 431 or 432. Therefore, the first encapsulating packet of a TCP session may be received by computing device 101b via logical network connections 432 and the second encapsulating packet of the same TCP session may be received by computing device 101b via logical network connections 421.
  • network device 105 When network device 105 receives a packet from the first network interface at step 501, network device 105 determines which priority level the packet belongs to at step 502. Network device 105 may determine the priority level according to port number of the packet, content of the packet, type of service field in the packet, and other common packet inspection techniques known by those skilled in the art. At step 503, network device 105 stores the packet in a priority queue corresponding to the priority level determined.
  • network device 105 determines that it is able to transmit a packet to network device 103 at step 510
  • network device 105 at step 511 retrieves a packet from one of the priority queues which has lowest PSEQ in the priority queue.
  • select the priority queue includes selecting the highest priority queue first and the lowest priority queue last, selecting the priority queue which has a packet having been stored for the longest time, selecting the priority queue with the largest number of packets, selecting a priority queue according to an equation, and selecting a priority queue according to a policy.
  • a packet that has been stored the longest in the highest priority queue is selected first, and if there is no packet in the highest priority queue, a packet that has been stored the longest in the next highest priority queue is then selected.
  • the packet selection process continues until the lowest priority queue is selected. If there is still no packet available in the lowest priority queue, no packet is retrieved from any of the priority queues and no encapsulating packet is transmitted to network device 103.
  • network device 105 first tries to retrieve a packet with lowest PSEQ from priority queue 401. If there is no packet in priority queue 401, network device 105 then tries to retrieve a packet with lowest PSEQ from priority queue 402.
  • network device 105 then tries to retrieve a packet with lowest PSEQ from priority queue 403. This retrieving arrangement tries to have packets with the highest priority transmitted first. The reason a packet with lowest PSEQ in a priority queue is selected is that a lowest PSEQ indicates that the packet has arrived at network device 105 the earliest among all packets in that priority queue. It would be appreciate that other techniques can be employed if ordering of PSEQ is not used to indicate a sequence of arrival of packets at network device 103.
  • network device 105 makes a decision to select logical network connection among logical network connections 421, 431 and 432 to be used to transmit the packet.
  • the decision to select logical network connections to be used depends on policies configurable by the administrator of network device 105, policies determined by the manufacturer of network device 105, or a combination of both.
  • network device 105 assigns a TSEQ according to the logical network connection selected in step 513.
  • GSEQ, TSEQ, PSEQ, the priority level determined and the packet retrieved are combined together to form data payload of an encapsulating packet, which is then transmitted to network device 103 via one of the logical network connections of logical network connections 421, 431 and 432.
  • GSEQ, TSEQ, PSEQ, the priority level determined and the packet retrieved are encapsulated in the encapsulating packet.
  • the TSEQ is determined by network device 105 and is stored at the TSEQ field of an encapsulating packet in the selected logical network connection queue.
  • the GSEQ and PSEQ which have been assigned already are also stored in the encapsulating packet in their respective fields along with the packet encapsulated in the data field.
  • the creation and the format of the encapsulating packet are in accordance to the descriptions in relation to FIG. 3 discussed above.
  • the encapsulating packet is re-transmitted either using the same logical network connection or a second logical network connection.
  • the TSEQ, PSEQ and GSEQ remain the same.
  • the GSEQ and PSEQ in the encapsulating packet remain the same but the TSEQ is different because the second logical network connection has different TSEQ. Therefore, the encapsulating packet is stored in the logical network connection queue of the second logical network connection before being transmitted through the second logical network connection.
  • the decision concerning when to use the same logical network connection and when to use the second logical network connection depends on policies configured by the network administrator of network device 105 or predefined by the manufacturer of network device 105. Those skilled in the art will appreciate many different ways to configure or define the policies.
  • header fields of the encapsulating packet such as destination address, source address and checksum are updated to reflect the move. Those skilled in the art would know what headers fields have to be updated. This also applies to a situation when a logical network connection is no longer in operation and all encapsulating packets that are stored in the first logical network connection are moved to logical network connection queues of other logical network connections.
  • the process of transmitting the encapsulating packet stops at step 515.
  • network device 105 is ready to transmit an encapsulating packet when the it has enough processing power, a predefined storage time for a packet is reached, the network access link is ready, a logical network connection is ready, there is enough buffer at the logical network connection and/or other reasons to allow the VPN Sender Device to transmit the encapsulating packet that are commonly known to those skilled in the arts.
  • GSEQ, PSEQ and TSEQ are assigned before the packet is retrieved from priority queue and forwarded by network device 105 at step 514.
  • GSEQ is assigned when the packet is stored in a priority queue; PSEQ and TSEQ are then assigned after the packet is retrieved from priority queue and before a corresponding encapsulating packet is created.
  • the assignments of GSEQ, PSEQ and TSEQ can be implemented at different stages.
  • the decision of how and when the GSEQ, PSEQ and TSEQ are assigned matches with how network device 105 sorts encapsulating packets using GSEQ, PSEQ and TSEQ. It is known to those skilled in the art that other fields of the encapsulating packet have to be updated, including checksum, when GSEQ, PSEQ, TSEQ and priority level are stored and/or modified.
  • network device 105 follows policies for selecting logical network connections.
  • Policies include using a logical network connection with lowest latency for highest priority level packets, using a logical network connection with largest transfer rate for highest priority level packets, using a logical network connection with lowest cost for lowest priority level packets, using all the logical network connections evenly when transfer rate and latency of the logical network connections are within predefined ranges for packets that do not belong to the highest priority level, distributing packets to all the logical network connections with a priority level weighted distribution, and allowing the use of lowest latency logical network connection for non-highest priority level packets only when there is no highest priority level packets waiting to be sent.
  • the aims of using policies to select which logical network connections to be used include reducing latency, effective use of bandwidth and lowering cost.
  • the policies that are used to select logical network connections include policies using latency, packet drop, delay, bandwidth, number of sessions allowed, price, and usage.
  • a policy for selecting logical network connection is to prefer using the logical network connection with lowest latency.
  • a policy for selecting logical network connection is based on the bandwidth available for each of the logical network connection. It is preferred to transmit the encapsulating packet via the logical network connection which has more bandwidth.
  • a policy for selecting logical network connection is based on the number of packet drops in a period of time experienced in each of the logical network connection.
  • a policy for selecting logical network connection is based on the delay experienced in each of the logical network connection. It is preferred to transmit the encapsulating packet via the logical network connection which has less delay.
  • a policy for selecting logical network connection is based on the usage of the each of the logical network connection. It is preferred to transmit the encapsulating packet via the logical network connection which has been used less frequent.
  • a policy for selecting logical network connection is based on the number of sessions carried by each of the logical network connection. It is preferred to transmit the encapsulating packet via the logical network connection which has fewer sessions, such as TCP sessions.
  • a policy for selecting logical network connection is based on the pricing of transmitting encapsulating packets at the time of transmission. It is preferred to transmit the encapsulating packet via the logical network connection which costs less.
  • only the payload of an encapsulating packet is encrypted. Therefore only the packet encapsulated in the encapsulating packet is encrypted while GSEQ, PSEQ, TSEQ, and priority levels are not encrypted. Additionally or alternatively, at least one of the priority level, global sequence number and priority sequence number stored at the other options field of the encapsulating packet is encrypted.
  • FIG. 6 illustrates a relationship among encapsulating packets transmitted by network device 105 and received by network device 103 through the Internet or inter-connected networks that are connected to one of network interfaces of network device 105, queue 610, and network access links 601, 602 and 603 according to one of the embodiments of the present invention.
  • Network access links 601, 602 and 603 are network access links 120a, 120b and 120c respectively in this illustration.
  • Logical network connections 611, 612 and 613 are established in network access link links 601, 602 and 603 respectively.
  • logical network connection 421 and logical network connection 611 are the same logical network connection
  • logical network connection 431 and logical network connection 612 are the same logical network connection
  • logical network connection 432 and logical network connection 612 are the same logical network connection. Therefore, an encapsulating packet may be transmitted by network device 105 through logical network connection 431, which is also logical network connection 612, and received by network device 103.
  • Queue 610 may be an array, a linked list, a tree or other kinds of data structure that can be implemented in a computer readable storage medium. Queue 610 holds a plurality of elements. Each element is a unit of storage. According to one of the embodiments of the present invention, the format of the encapsulating packet follows the description of FIG. 3 above.
  • Logical network connections 611, 612 and 613 are aggregated together to form one logical connection.
  • each Ethernet packet originating from computing device 102a may be received by computing device 101a via logical network connections 603, 604 and 605.
  • each encapsulating packet belonging to a video stream session originating from computing device 102c may be received by computing device 101b via logical network connections 603, 604 and 605. Therefore, a first encapsulating packet of the video stream may be received by computing device 101b via logical network connections 603 and a second encapsulating packet of the same video stream may be received by computing device 101b via logical network connections 602.
  • E-PSEQ For each priority level, there is one expected priority sequence number (E-PSEQ). E-PSEQ is used to hold the PSEQ expected for an encapsulating packet with a priority arriving next. For example, if there are eight priorities, there are eight E-PSEQs.
  • Elements in queue 610 are used to store encapsulating packets.
  • the reason for having queue 610 is to sort encapsulating packets.
  • E-GSEQ expected global sequence number
  • FIG. 7, which should be viewed in conjunction with FIG.1 and FIG. 6, is a flow-chart illustrating a process of a VPN Receiver Device, such as network device 103, receiving an encapsulating packet.
  • network device 103 After network device 103 has received an encapsulating packet through one of the aggregated logical network connections from one of the network interfaces of network device 103 at step 701, network device 103 identifies the priority level of the packet encapsulated in the encapsulating packet at step 702. At step 703, network device 103 determines whether or not to store the encapsulating packet in a queue 610. The encapsulating packet is stored in queue 610 at step 704 if network device 103 determines to store the encapsulating packet.
  • the encapsulating packet such as an IP packet or Ethernet packet, in the encapsulating packet is decapsulated to retrieve the packet and the packet is transmitted at step 705.
  • the encapsulating packet is decapsulated to retrieve the packet and it is the packet, instead of the encapsulating packet, being stored in queue 610 at step 704. Therefore, at step 703, network device 103 determines whether to store the packet in a queue 610. In addition, at step 705, there is no further need for decapsulating the packet before transmitting the packet.
  • network device 103 also identifies GSEQ, PSEQ, TSEQ of the packet encapsulated in the encapsulating packet at step 702.
  • the encapsulating packet is stored in one of the elements of queue 610 at step 704.
  • the encapsulating packet when the GSEQ is larger than the E-GSEQ but the PSEQ is smaller or equal to the E-PSEQ of the PSEQ corresponding priority level, the encapsulating packet is not stored in queue 610. Instead, a placeholder is stored in one of the elements of queue 610 at step 703 and the encapsulating packet is transmitted.
  • the first method is to retrieve an encapsulating packet according to a comparison made between PSEQ and the E-PSEQ of the corresponding priority level.
  • the first method is illustrated in Fig. 8A.
  • the second method is to retrieve an encapsulating packet according to a comparison made between GSEQ and E-GSEQ of the corresponding priority level. If the GSEQ is larger than E-GSEQ, the encapsulating packet is not retrieved. If the GSEQ is smaller than or equal to E-GSEQ, the encapsulating packet is retrieved.
  • FIG. 8A which should be viewed in conjunction with FIG. 6 and FIG.7, is a flow-chart illustrating the process of the first method.
  • a network device 103 is ready to retrieve an encapsulating packet from queue 610 for transmission. This happens when network device 103 has enough processing power, a predefined period of time is reached, a new encapsulating packet arrived, the receiver of the packet is ready to receive the packet, there is enough queue for receiving the packet and/or other reasons to allow network device 103 to retrieve the encapsulating packet from a corresponding element of queue 610 that are commonly known to those skilled in the arts.
  • encapsulating packets stored at queue 610 are examined to check whether their PSEQs are smaller than or equal to the E-PSEQ of their corresponding priorities. If there is no encapsulating packet with PSEQ that is smaller than or equal to the E-PSEQ of its corresponding priority level, no encapsulating packet is retrieved from queue 601, and the process stops at step 803. On the other hand, encapsulating packets with PSEQs that are smaller than or equal to the E-PSEQ of their corresponding priority levels are retrieved and transmitted at step 804.
  • one or more E-PSEQs that are corresponding to the priority levels of the one or more retrieved and transmitted encapsulating packets are updated to indicate that there are one or more encapsulating packets that have been retrieved and transmitted.
  • one or more placeholders are put in the queues at the positions where the retrieved encapsulating packets were originally placed. This step is to facilitate operation of the second method.
  • the format and content of a placeholder can be anything as long as it has the same GSEQ of the retrieved encapsulating packet.
  • the placeholder is used to indicate that an encapsulating packet has been retrieved during the operation of the second method. Packets encapsulated in the data field of the retrieved encapsulating packets can be transmitted between the steps of 804 and 805, between the steps of 805 and 806.
  • encapsulating packets are be transmitted according to the order of PSEQ of a priority level. Therefore, the encapsulating packets are transmitted in correct order of the priority level. This allows encapsulating packets to be transmitted without the use of GSEQ. Therefore, it is possible that encapsulating packets with GSEQs higher than E-GSEQ are transmitted earlier than encapsulating packets with GSEQs lower than E-GSEQ. It is also possible that encapsulating packets are transmitted not according to GSEQ order. The benefit of this is that encapsulating packets with higher priority levels are transmitted without waiting for encapsulating packets with lower GSEQ. Therefore the order of packets arriving at VPN Sender Device for VPN Receiver Device may be different from the order of encapsulating packets, which encapsulate the packets, transmitted by VPN Receiver Device.
  • FIG. 8B is an illustration of one of embodiments of present invention for a process to update E-PSEQ.
  • VPN Receiver Device receives an encapsulating packet, it identifies the PSEQ and the priority level of the encapsulating packet at step 811.
  • the PSEQ of the encapsulating packet is compared against the E-PSEQ of the identified priority level.
  • the PSEQ and priority level of an encapsulating packet are ten and three respectively, the PSEQ is compared to the E-PSEQ of priority level three. In another example, if the PSEQ and priority level of an encapsulating packet are twenty-three and one respectively, the PSEQ is compared to the E-PSEQ of priority level one.
  • the PSEQ of the encapsulating packet is smaller than the E-PSEQ of the identified priority level, there is no need to update the E-PSEQ because the VPN IP has arrived at the VPN Receiver Device later than expected. If the PSEQ of the encapsulating packet is equal to the E-PSEQ of the identified priority level, the E-PSEQ of the identified priority level is updated by increasing value of the E-PSEQ of the identified priority level by one at step 814 because the encapsulating packet has arrived at the VPN Receiver Device in sequence as expected. If the PSEQ of the encapsulating packet is larger than the E-PSEQ of the identified priority level, it is possible that the encapsulating packet has arrived earlier than expected or the estimation of the E-PSEQ of the identified priority level is not accurate.
  • the estimation of the E-PSEQ of the identified priority level is not accurate if there are one or more encapsulating packets unable to reach the VPN Receiver Device in time.
  • E-PSEQ of the identified priority level is not accurate and E-PSEQ of the identified priority level is updated to the largest PSEQ of the most recent encapsulating packet of the identified priority level.
  • E-PSEQ of the identified priority level is updated to the largest PSEQ among all the encapsulating packets stored at queue 610 and belonging to the identified priority level at step 814.
  • the value of the threshold for the number of encapsulating packets having their PSEQ larger than E-PSEQ of the identified priority level in a period of time can be set by administrator, by the manufacturer of the VPN Receiver Device or estimated by the VPN Receiver Device. According to one of the embodiments of the present invention, the threshold for the number of encapsulating packets having their PSEQ larger than E-PSEQ of the identified priority level in a predefined period of time is in the range of ten packets to one hundred packets. According to one of the embodiments of the present invention, the threshold for the number of encapsulating packets having their PSEQ larger than E-PSEQ of the identified priority level in a period of time is in the range of three packets to twenty packets for higher priority level traffic.
  • the value of the pre-defined period of time can also be set by an administrator, by the manufacturer of the VPN Receiver Device or estimated by the VPN Receiver Device. According to one of the embodiments of the present invention, the range of the predefined period of time is between 10 milliseconds to 5 seconds. According to one of the embodiments of the present invention, the range of the predefined period of time for higher priority level traffic is between 5 milliseconds to 100 milliseconds.
  • FIG. 8C is an illustration of one of embodiments of present invention for a process to update E-PSEQ.
  • the steps shown in FIG. 8C are similar to those in FIG. 8B.
  • the main difference between FIG. 8B and FIG. 8C is that the process of FIG. 8B is triggered by an arrival of an encapsulating packet at the VPN Receiver Device while the process of FIG. 8C is triggered by periodic examination of encapsulating packets that have been stored in queue 610.
  • the frequency of periodic examination can be set by administrator or by the manufacturer of the VPN Receiver Device and can be different or the same for different priority level traffic.
  • PSEQs of encapsulating packets corresponding to a priority level and having been stored at queue 610 are identified. Similar to step 813, at step 822, when is the number of a plurality of encapsulating packets having their PSEQ larger than E-PSEQ of the priority level in a period of time is larger than a threshold, it is an indication that the estimation of the E-PSEQ of the identified priority level is not accurate and E-PSEQ of the identified priority level is updated to the largest PSEQ of the most recent encapsulating packet of the identified priority level. E-PSEQ of the priority level is updated to the largest PSEQ among PSEQs of all the encapsulating packets stored at queue 610 and belonging to the priority level at step 823.
  • the value of the threshold for the number of encapsulating packets having their PSEQ larger than E-PSEQ of the identified priority level in a period of time can be set by an administrator, by the manufacturer of the VPN Receiver Device or estimated by the VPN Receiver Device at step 822.
  • the threshold for the number of encapsulating packets having their PSEQ larger than E-PSEQ of the identified priority level in a period of time is in the range of ten packets to one hundred packets.
  • the predefined threshold for the number of encapsulating packets having their PSEQ larger than E-PSEQ of the identified priority level in a period of time is in the range of three packets to twenty packets for higher priority level traffic.
  • an E-PSEQ of a priority level is updated at step 814 or step 823
  • the process shown in FIG. 8A can be performed again to retrieve and transmit encapsulating packets with their PSEQs that are smaller than or equal to the updated E-PSEQ of the priority level.
  • FIG. 9, which should also be viewed in conjunction with FIG. 6 and FIG. 8, is a flow-chart illustrating the process of the second method.
  • Step 901 is the same as step 801.
  • step 902 encapsulating packets stored at elements of queue 610 are examined to determine whether their GSEQs are smaller than or equal to the E-GSEQ. If there is no encapsulating packet with GSEQ that is smaller than or equal to the E-GSEQ, no encapsulating packet is retrieved from queue 601 and transmitted, and the process stops at step 907. On the other hand, if there is one or more encapsulating packets with GSEQ that is smaller than or equal to the E-GSEQ, step 903 is conducted.
  • the content of the encapsulating packet is examined to determine whether or not the encapsulating packet is a placeholder put at step 806. If the encapsulating packet is transmitted as indicated by the presence of a placeholder, the placeholder is not transmitted because the corresponding encapsulating packet has been retrieved and transmitted already at step 804. If the encapsulating packet is not a placeholder and is still in the corresponding element of queue 610, the encapsulating packet is transmitted at step 904. At step 905, the element of queue 610 that is used to hold the encapsulating packet or the placeholder is released to allow the element of queue 610 to be used for other purposes, including storing another encapsulating packet or another placeholder. The E-GSEQ is updated to reflect that expected GSEQ of the next encapsulating packet should be transmitted at step 906. The process stops at step 907.
  • encapsulating packets are transmitted according to the order of GSEQ. Therefore, the encapsulating packets are transmitted in the correct order. If there is an encapsulating packet not able to reach the VPN Receiver Device (Late Encapsulating Packet), the VPN Receiver Device cannot keep waiting for the arrival of the late encapsulating packet or retransmission of the late encapsulating packet as it is possible that the Late Encapsulating Packet will never arrive. Therefore, if there is a plurality of encapsulating packets with GSEQs higher than E-GSEQ arriving at the VPN Receiver Device in a period of time, E-GSEQ is updated to reflect the probability of arrival of Late Encapsulating Packet. If the period of time is too short, encapsulating packets are transmitted without proper ordering. If the predefined period of time is too large, encapsulating packets will experience unnecessary delay before being transmitted.
  • the packet encapsulated in the data field is first decapsulated and then either transmitted to a host connecting to a network interface of the VPN Receiver Device or be further processed by the VPN Receiver Device.
  • the packet is an encapsulating packet with destination address belonging to one of the network devices connected to the first network interface of the VPN Receiver Device.
  • the packet encapsulated in the encapsulating packet is a broadcast Ethernet packet that all network devices and computing devices connected to all the interfaces of the VPN Receiver Device will receive it.
  • the packet encapsulated in the encapsulating packet along with its corresponding GSEQ, TSEQ, PSEQ, and priority are retrieved at step 704. Therefore, it is not the encapsulating packet stored in queue 610; instead, it is the packet, which is originally encapsulated in the encapsulating packet stored in queue 610. Therefore, at steps 801, 804, 901, 902 and 903, it is the packet being stored, processed, retrieved or transmitted.
  • the GSEQ, TSEQ, PSEQ and priority level can be stored in corresponding element of queue 610 along with the packet or can be stored separately but linked with the packet. Those skilled in the art would appreciate different techniques to link the encapsulating packet stored in queue 610 with its corresponding GSEQ, TSEQ, PSEQ, and priority level stored in the same or different parts of a computer readable storage medium.
  • an element in queue 610 is a placeholder or an encapsulating packet.
  • one of the bits in each element in queue is used to indicate whether the element in queue 610 holds a placeholder or an encapsulating packet.
  • a separate memory unit is used to store information about whether an element in queue 610 holds a placeholder or an encapsulating packet.
  • the element if an element holds a predefined pattern of bits or bytes, the element holds a placeholder.
  • the first method shown in FIG. 8 is executed by one thread in a VPN Receiver Device and the second method shown in FIG. 9 is executed by another thread in the VPN Receiver Device.
  • the first method shown in FIG. 8 and the second method shown in FIG. 9 are executed by the same thread.
  • the first method shown in FIG. 8 is executed by one process in VPN Receiver Device and the second method shown in FIG. 9 is executed by another process in VPN Receiver Device.
  • the first method shown in FIG. 8 and the second method shown in FIG. 9 are executed by the same process.
  • the execution of the first method shown in FIG. 8 is in sequence with the execution of the second method shown in FIG. 9. According to one of the embodiments of the present invention, the execution of the first method shown in FIG. 8 does not need to be in sequence with the execution of the second method shown in FIG. 9.
  • the encrypted part is decrypted to retrieve the packet.
  • network device 103 and 105 can both act as a VPN Sender Device. Since the VPN Sender Device, in the case of a reverse direction of the data flow, can assume the function of VPN Sender Device, the following also applies to the network device 103 and 105; said network device 103 and 105 can furthermore be of identical design.
  • VPN Sender Device 1001 shown has been implemented in the form of a hardware router. It comprises a housing in which a circuit board 1004 with processing unit 1002 and computer readable storage medium 1003 are disposed. The above-described methods according to the present invention are implemented on the circuit board 1004 in the form of software.
  • the circuit board 1004 connects to communication modules 1005 in the housing.
  • the communication modules 1005 can be designed for identical or different network access links, such as network access links 121a, 121b, and 121c.
  • a communication module 1005 can be designed for an Ethernet network access link while another communication module 1005 can be designed for a LTE network access link and still another communication module can be designed for a WLAN network access link.
  • Communication modules 1005 can be part of circuit board 1004, can be connected to circuit board 1004 through bus architecture, such as PCI and USB, can be inserted into circuit board 1004 or can be connected to circuit board through a communication port.
  • Computer readable storage medium 1003 is capable of storing, containing or carrying instruction(s) for processing unit 1002 to perform and/or data for processing unit 1002 to use.
  • the communication modules 1005 comprise suitable connector means 1006, for example, in the form of socket connectors, a LAN socket connector, a PCI bus connector, a USB connector, a Thunderbolt connector or an Ethernet connector for connection with an associated network access link.
  • suitable connector means 1006 for example, in the form of socket connectors, a LAN socket connector, a PCI bus connector, a USB connector, a Thunderbolt connector or an Ethernet connector for connection with an associated network access link.
  • a packet which is to be transmitted via one of the logical network connections in one of the network access links to one of computing device 101a, 101b or 101c, reaches network device 105 (acting as a VPN Sender Device in this embodiment) through network access link 121c via the LAN socket connector 1006 and communication module 1005, is then transmitted to circuit board 1004.
  • Circuit board 1004 determines which priority level the packet belongs to. Circuit board 1004 determines the priority level according to the port number of the packet, the content of the packet, the type of service field in the packet, and/or other common packet inspection techniques known by those skilled in the art.
  • Circuit board 1004 then stores the packet in a priority queue, such as 401, 402 and 403, corresponding to the determined priority level at computer readable storage medium 1003. According to one of the embodiments of the present invention, at this point, circuit board 1004 determines the GSEQ, TSEQ, PSEQ and priority level of the packet. In one variant, circuit board 1004 does not determine TSEQ at this point, but rather determines TSEQ when the packet is about to be transmitted.
  • a plurality of logical network connections are established in network access links 121a and 121b by circuit board 1004 of network device 105 to connect to network device 103 (acting as a VPN Receiver Device in this embodiment).
  • circuit board 1004 determines that it is able to transmit a packet to network device 103, circuit board 1004 retrieves a packet which has the lowest PSEQ in the priority queue from the one of the priority queues at the computer readable storage medium 1003. According to one of the embodiments of the present invention, circuit board 1004 first tries to retrieve a packet with lowest PSEQ from priority queue 401. If there is no packet in priority queue 401, the network device 105 then tries to retrieve a packet with lowest PSEQ from priority queue 402. If there is also no packet in priority queue 402, circuit board 1004 then tries to retrieve a packet with lowest PSEQ from priority queue 403.
  • Circuit board 1004 selects the logical network connection in the network access link to transmit the packet retrieved.
  • the decision to select which of the of the logical network connections in network access links to be used depends on policies, which are stored in computer readable storage medium 1003, configurable by the administrator of network device 105, policies determined by the manufacturer of network device 105, or a combination of both.
  • Circuit board 1004 assigns a TSEQ according to the logical network connection selected. GSEQ, TSEQ, PSEQ, the priority level determined and the packet retrieved are combined by circuit board 1004 together to form data payload of an encapsulating packet, which is then transmitted to network device 103 via one of the logical network connections in one of the network access links and the corresponding communication module 1005. In one variant, circuit board 1004 encapsulates GSEQ, TSEQ, PSEQ, the priority level determined and the packet retrieved in the encapsulating packet.
  • Circuit board 1004 decides which logical network connection should be used for transmitting the encapsulating packet, such as logical network connections 421, 431 and 432, based on policies, which are stored at computer readable storage medium 1003, configurable by the administrator of the network device 103 or policies determined by the manufacturer of the network device 103.
  • circuit board 1004 is ready to transmit an encapsulating packet when circuit board 1004 has enough processing power, a predefined period of time is reached, the network access link is ready, a logical network connection is ready, there is enough buffer at the logical network connection and/or other reasons to allow circuit board 1004 to transmit the encapsulating packet that are commonly known to those skilled in the arts.
  • circuit board 1004. only the payload of an encapsulating packet is encrypted by circuit board 1004. Additionally or alternatively, at least one of the priority level, global sequence number and priority sequence number stored at the options field of the encapsulating packet is encrypted by circuit board 1004. Circuit board 1004 uses processing unit 1002 to carry out encryption and decryption. In one variant, there is a separate encryption and decryption processing unit to carry out encryption and decryption in order to lower the loading on processing unit 1002.
  • VPN Receiver Device can be identical in design to the VPN Sender Device. Therefore, the embodiment shown in FIG. 10 also applies for a VPN Receiver Device.
  • FIG 10 when viewed in conjunction with FIG. 1 and FIG. 6, illustrates one of the embodiments of a VPN Receiver Device of the present invention.
  • VPN Receiver Device shown has been implemented in the form of a hardware router. It comprises a housing in which circuit boards 1004 with processing unit 1002 and computer readable storage medium 1003 are disposed. The above-described methods according to the present invention are implemented on circuit board 1004 in the form of software. Circuit board 1004 connects to communication modules 1005 in housing 8.
  • the communication modules 1005 can be designed for identical or different network access links, such as network access links 120a, 120b, and 120c, and network link 120d.
  • a communication module 1005 can be designed for an Ethernet network access link while another communication module 1005 can be designed for a LTE network access link and still another communication module can be designed for a WLAN network access link.
  • Communication modules 1005 can be part of circuit board 1004, can be connected to circuit board 1004 through bus architecture, such as PCI and USB, can be inserted into circuit board 1004 or can be connected to circuit board through a communication port.
  • Computer readable storage medium 1003 is capable of storing, containing or carrying instruction(s) for processing unit 1002 to perform and/or data for processing unit 1002 to use.
  • the communication modules 1005 comprise suitable connector means, for example, in the form of socket connectors, a LAN socket connector, a PCI bus connector, a USB connector, a Thunderbolt connector or an Ethernet connector for connection with an associated network access link.
  • suitable connector means for example, in the form of socket connectors, a LAN socket connector, a PCI bus connector, a USB connector, a Thunderbolt connector or an Ethernet connector for connection with an associated network access link.
  • a plurality of logical network connections are established in access links 120a, 120b and 120c by circuit board 1004 of network device 103 (acting as a VPN Receiver Device in this embodiment)to connect to network device 105 (acting as a VPN Sender Device in this embodiment).
  • an encapsulating packet When an encapsulating packet reaches a network device 103 the via one of logical network connections in one of the network access links and socket connector 1006, the encapsulating packet is transmitted to circuit board 1004 via communication module 1005 for processing.
  • Circuit board 1004 identifies GSEQ, PSEQ, TSEQ and priority level of the encapsulating packet by examining the other options fields of the encapsulating packet. Processing unit 1002 then determines whether to store the encapsulating packet in one of the elements of queue 610 in computer readable storage medium 1003. The packet encapsulated in encapsulating packet, such as an IP packet or Ethernet packet, is retrieved by processing unit 1002 and transmitted if processing unit 1002 determines not to store the encapsulating packet.
  • Processing unit 1002 updates E-GSEQ and E-PSEQ and stores E-GSEQ and E-PSEQ at computer readable storage medium 1003. Processing unit 1002 also compares GSEQ and PSEQ of the encapsulating packet with E-GSEQ and E-PSEQ to determine whether to store the encapsulating packet in queue 610 and whether to retrieve an encapsulating packet from queue 610 for transmission. Computer readable storage medium 1003 stores queue 610.
  • processing unit 1002 stores a placeholder in one of the elements of queue 610 at and the encapsulating packet is transmitted via a communication module 1005 and socket connector 1006 to the receiving host.
  • the encrypted part is decrypted by processing unit 1002 or a decryption circuit on circuit board 1004 to retrieve the original packet.
  • processing unit 1002 there are many techniques for processing unit 1002 to identify whether an element in queue 610 is a placeholder or an encapsulating packet.
  • one of the bits in each element in queue 610 is used to indicate whether the element in queue 610 holds a placeholder.
  • a separate memory unit in computer readable storage medium 1003 is used to store information about whether or not an element in queue 610 holds a placeholder.
  • processing unit 1002 determines that the element holds a placeholder.
  • Those skilled in the art would appreciate different techniques to have instructions to instruct processing units how to identify whether an element in queue 610 is a placeholder.
  • the method implemented on the circuit board 1004 can also be implemented directly at a circuit board or a software layer for a PC, server, and cloud enabled server or the like. If it is a software layer, the method according to the present invention is then implemented as a software layer on the PC, server, and cloud enabled server or the like.
  • FIG. 1 is a network diagram illustrating the use of network devices via a plurality of logical network connections according to one of the embodiments of the present invention in a typical network environment.
  • FIG. 2 is a diagram showing one example of a packet format according to usual IPv4;
  • FIG. 3 is a diagram illustrating option fields and data fields of an encapsulating packet according to one of the embodiments of the present invention in a typical network environment
  • FIG. 4 is a diagram illustrating the relationship among priority queue, logical network connections, and network access links at a VPN Sender Device according to one of the embodiments of the present invention
  • FIG. 5A is a flow-chart illustrating the steps of determining priority level according to one of the embodiments of the present invention.
  • FIG. 5B is a flow-chart illustrating the steps of transmitting an encapsulating packet according to one of the embodiments of the present invention.
  • FIG. 6 is a diagram illustrating the relationship among a plurality of logical network connections, network access links and a queue at a VPN Receiver Device according to one of the embodiments of the present invention
  • FIG. 7 is a flow-chart illustrating the steps of receiving an encapsulating packet according to one of the embodiments of the present invention.
  • FIG. 8A is a flow-chart illustrating the steps of retrieving an encapsulating packet from a queue according to one of the embodiments of the present invention
  • FIG. 8B is a flow-chart illustrating the steps of a process to update E-PSEQ according to one of the embodiments of the present invention.
  • FIG. 8C is a flow-chart illustrating the steps of another process to update E-PSEQ according to one of the embodiments of the present invention.
  • FIG. 9 is a flow-chart illustrating the steps of a process to transmit a packet according to one of the embodiments of the present invention.
  • FIG. 10 is a network device according to one of the embodiments of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Quality & Reliability (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

L'invention concerne des procédé, des dispositifs de réseau et des systèmes qui reçoivent et transmettent des paquets par priorité consistant à : recevoir un paquet via une de plusieurs interfaces réseau ; déterminer un nombre de séquences global et un niveau de priorité du paquet ; stocker le paquet ; encapsuler le paquet dans des paquets d'encapsulation ; transmettre les paquets d'encapsulation via plusieurs connexions réseau logiques vers le second dispositif de réseau en fonction du nombre de séquences global et du niveau de priorité ; recevoir les paquets d'encapsulation ; désencapsuler le paquet des paquets d'encapsulation ; et transmettre le paquet au récepteur voulu en fonction du nombre de séquences global et du niveau de priorité.
PCT/IB2012/055862 2012-10-25 2012-10-25 Procédé, dispositif et système de prioritisation de paquets d'encapsulage dans plusieurs connexions réseau logiques WO2014064486A1 (fr)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/IB2012/055862 WO2014064486A1 (fr) 2012-10-25 2012-10-25 Procédé, dispositif et système de prioritisation de paquets d'encapsulage dans plusieurs connexions réseau logiques
CN201280076668.3A CN104956637B (zh) 2012-10-25 2012-10-25 在多个逻辑网络连接中优先化封装数据包的方法、装置以及系统
GB1415039.5A GB2514060B (en) 2012-10-25 2012-10-25 A method, device and system to prioritize encapsulating packets in a plurality of logical network connections
US14/003,237 US9369398B2 (en) 2012-10-25 2012-10-25 Method, device, and system to prioritize encapsulating packets in a plurality of logical network connections

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IB2012/055862 WO2014064486A1 (fr) 2012-10-25 2012-10-25 Procédé, dispositif et système de prioritisation de paquets d'encapsulage dans plusieurs connexions réseau logiques

Publications (1)

Publication Number Publication Date
WO2014064486A1 true WO2014064486A1 (fr) 2014-05-01

Family

ID=50544092

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2012/055862 WO2014064486A1 (fr) 2012-10-25 2012-10-25 Procédé, dispositif et système de prioritisation de paquets d'encapsulage dans plusieurs connexions réseau logiques

Country Status (4)

Country Link
US (1) US9369398B2 (fr)
CN (1) CN104956637B (fr)
GB (1) GB2514060B (fr)
WO (1) WO2014064486A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110808917A (zh) * 2019-10-22 2020-02-18 北京红云融通技术有限公司 多链路聚合数据重传方法及发送设备

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9531565B2 (en) * 2013-12-20 2016-12-27 Pismo Labs Technology Limited Methods and systems for transmitting and receiving packets
EP3167576B1 (fr) * 2014-07-08 2020-12-30 Telefonaktiebolaget LM Ericsson (publ) Noeuds de communication, procédés dans ceux-ci, programmes d'ordinateur et support de stockage lisible par ordinateur
CN110337121B (zh) * 2014-07-14 2023-03-24 柏思科技有限公司 用于评估聚合的连接的网络性能的方法和系统
US10805219B2 (en) * 2014-07-14 2020-10-13 Pismo Labs Technology Limited Methods and systems for evaluating network performance of an aggregated connection
AU2014408523A1 (en) * 2014-10-10 2017-04-27 Telefonaktiebolaget Lm Ericsson (Publ) Broadcast in meshed networks
WO2016058161A1 (fr) * 2014-10-16 2016-04-21 Intel Corporation Procédé, appareil, et système d'utilisation d'un équipement d'utilisateur en tant que petit nœud b évolué pour petite cellule
US10348635B2 (en) * 2014-12-08 2019-07-09 Huawei Technologies Co., Ltd. Data transmission method and device
US20170302522A1 (en) * 2016-04-14 2017-10-19 Ford Global Technologies, Llc Method and apparatus for dynamic vehicle communication response
US10764781B2 (en) 2016-05-03 2020-09-01 Qualcomm Incorporated Systems and methods for reordering data received from a plurality of radio access technologies (RATs)
US10044628B2 (en) * 2016-06-13 2018-08-07 Pismo Labs Technology Limited Methods and systems for receiving and transmitting packets based on priority levels
CN107612764B (zh) * 2016-07-12 2022-11-01 中兴通讯股份有限公司 一种传输网管数据采集装置和方法
US10511992B2 (en) * 2016-12-19 2019-12-17 Qualcomm Incorporated Prioritizing packets in wireless communications
GB2575228B (en) * 2017-03-20 2022-07-06 Pismo Labs Technology Ltd Methods and systems for evaluating network performance of an aggregated connection
CN107018578B (zh) * 2017-03-27 2019-09-20 海信集团有限公司 数据传输方法及装置
US11245546B2 (en) * 2017-11-27 2022-02-08 Pismo Labs Technology Limited Methods and systems for transmitting and receiving data packets through a bonded connection
US20210119930A1 (en) * 2019-10-31 2021-04-22 Intel Corporation Reliable transport architecture
CN111163058B (zh) * 2019-12-09 2021-11-02 京信网络系统股份有限公司 Dpdk数据加密处理方法、装置和网络设备
CN111147198B (zh) * 2020-01-02 2021-05-25 中科驭数（北京）科技有限公司 数据重传方法和装置
US11283722B2 (en) 2020-04-14 2022-03-22 Charter Communications Operating, Llc Packet prioritization for frame generation
US11394650B2 (en) * 2020-04-14 2022-07-19 Charter Communications Operating, Llc Modificationless packet prioritization for frame generation
US11290380B2 (en) * 2020-07-30 2022-03-29 S.C Correct Networks S.R.L. Method for transferring information across a data center network
US11968115B2 (en) 2021-10-31 2024-04-23 Avago Technologies International Sales Pte. Limited Method for verifying data center network performance

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1969509A (zh) * 2004-07-08 2007-05-23 思科技术公司 用于集中分组处理的网络设备体系结构
CN101478476A (zh) * 2008-12-08 2009-07-08 华为技术有限公司 一种分组微波数据的传输处理方法、装置及系统
WO2011064112A1 (fr) * 2009-11-25 2011-06-03 St-Ericsson (France) Sas Gestion de niveau de priorité de paquet de données

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3348080B1 (ja) * 2000-07-07 2002-11-20 松下電器産業株式会社 データ送信装置とデータ受信装置及びデータ送受信方法
US7023841B2 (en) * 2000-12-15 2006-04-04 Agere Systems Inc. Three-stage switch fabric with buffered crossbar devices
US7702742B2 (en) * 2005-01-18 2010-04-20 Fortinet, Inc. Mechanism for enabling memory transactions to be conducted across a lossy network
US20070219816A1 (en) * 2005-10-14 2007-09-20 Leviathan Entertainment, Llc System and Method of Prioritizing Items in a Queue
CN101018191A (zh) * 2006-02-08 2007-08-15 华为技术有限公司 多队列分组数据传输方法及其系统
JP4815284B2 (ja) * 2006-07-06 2011-11-16 アラクサラネットワークス株式会社 パケット転送装置
CN100562117C (zh) * 2007-11-19 2009-11-18 华为技术有限公司 一种码流调度方法、装置和系统
CN101252536B (zh) * 2008-03-31 2010-06-02 清华大学 路由器多队列数据包缓存管理与输出队列调度系统
CN101291546B (zh) * 2008-06-11 2011-09-14 清华大学 核心路由器交换结构协处理器
US8625427B1 (en) * 2009-09-03 2014-01-07 Brocade Communications Systems, Inc. Multi-path switching with edge-to-edge flow control

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1969509A (zh) * 2004-07-08 2007-05-23 思科技术公司 用于集中分组处理的网络设备体系结构
CN101478476A (zh) * 2008-12-08 2009-07-08 华为技术有限公司 一种分组微波数据的传输处理方法、装置及系统
WO2011064112A1 (fr) * 2009-11-25 2011-06-03 St-Ericsson (France) Sas Gestion de niveau de priorité de paquet de données

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110808917A (zh) * 2019-10-22 2020-02-18 北京红云融通技术有限公司 多链路聚合数据重传方法及发送设备
CN110808917B (zh) * 2019-10-22 2023-04-18 北京红云融通技术有限公司 多链路聚合数据重传方法及发送设备

Also Published As

Publication number Publication date
GB2514060B (en) 2020-12-16
GB2514060A (en) 2014-11-12
CN104956637A (zh) 2015-09-30
US20140226663A1 (en) 2014-08-14
GB201415039D0 (en) 2014-10-08
CN104956637B (zh) 2018-04-24
US9369398B2 (en) 2016-06-14

Similar Documents

Publication Publication Date Title
US9369398B2 (en) Method, device, and system to prioritize encapsulating packets in a plurality of logical network connections
US10044628B2 (en) Methods and systems for receiving and transmitting packets based on priority levels
US10171369B2 (en) Systems and methods for buffer management
US10243865B2 (en) Combined hardware/software forwarding mechanism and method
US10432556B1 (en) Enhanced audio video bridging (AVB) methods and apparatus
US7953885B1 (en) Method and apparatus to apply aggregate access control list/quality of service features using a redirect cause
US7602809B2 (en) Reducing transmission time for data packets controlled by a link layer protocol comprising a fragmenting/defragmenting capability
EP3094053A1 (fr) Classification de paquets par sortie prédictive pour la qualité de service
BR112015004565B1 (pt) Método para classificação de tráfego em estágios entre terminal e nós de agregação de um sistema de comunicações de banda larga e aparelho para classificação de tráfego em estágios entre terminal e nós de agregação de um sistema de comunicações de banda larga
EP3836498B1 (fr) Combinaison de file d'attente d'entrée et de sortie de transfert de paquets dans des dispositifs de réseau
US10715441B2 (en) System and method of a high buffered high bandwidth network element
TW200820697A (en) Systems and methods for applying back-pressure for sequencing in quality of service
WO2021041622A1 (fr) Procédés, systèmes et dispositifs de classification de données de couche de niveau 4 à partir de files d'attente de données
US9143448B1 (en) Methods for reassembling fragmented data units
WO2021213711A1 (fr) Gestion de files d'attente actives (aqm) sans état de cœur de double file d'attente virtuelle de réseaux de communication
US11799802B2 (en) Data traffic control device, a residential router, an operator network device and a telecommunication system
CN114051008B (zh) 一种降低无线网络的游戏业务时延的系统
KR101445466B1 (ko) 라우팅 환경에서의 소스 기반 큐 선택 메커니즘
JP3560499B2 (ja) 上位層指示による品質制御可能なｓａｒ機能を有する装置、ｌｓｉ装置および上位層指示による品質制御方法
US7512132B2 (en) Quality of service (QoS) class reordering
JP3836807B2 (ja) パケット転送装置
CN118590452A (zh) 一种报文调度方法及网络设备
CN115915262A (zh) 无线通信系统中执行业务流管理的方法和设备

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 14003237

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12887038

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 1415039

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20121025

WWE Wipo information: entry into national phase

Ref document number: 1415039.5

Country of ref document: GB

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12887038

Country of ref document: EP

Kind code of ref document: A1