US20190268181A1 - Methods and systems for transmitting and receiving data through tunnel groups - Google Patents
Methods and systems for transmitting and receiving data through tunnel groups Download PDFInfo
- Publication number
- US20190268181A1 US20190268181A1 US16/411,057 US201916411057A US2019268181A1 US 20190268181 A1 US20190268181 A1 US 20190268181A1 US 201916411057 A US201916411057 A US 201916411057A US 2019268181 A1 US2019268181 A1 US 2019268181A1
- Authority
- US
- United States
- Prior art keywords
- node
- tunnel
- tunnels
- tunnel group
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4633—Interconnection of networks using encapsulation techniques, e.g. tunneling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
- H04L47/82—Miscellaneous aspects
- H04L47/825—Involving tunnels, e.g. MPLS
-
- H04L61/2007—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/50—Address allocation
- H04L61/5007—Internet protocol [IP] addresses
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/10—Architectures or entities
- H04L65/102—Gateways
- H04L65/1033—Signalling gateways
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W72/00—Local resource management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W76/00—Connection management
- H04W76/10—Connection setup
- H04W76/15—Setup of multiple wireless link connections
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W76/00—Connection management
- H04W76/10—Connection setup
- H04W76/12—Setup of transport tunnels
Definitions
- the present invention relates in general to the field of computer networks. More particularly, the present invention relates to a method carried out by a first node for transmitting data to a second node mainly through a first tunnel group and receiving data from the second node mainly through a second tunnel group.
- Internet service providers can provide satellite Internet service to users through geostationary satellites which may offer high data throughput.
- the coverage area of satellite networks is also very big compared to other wireless networks that may be available in recent times. Satellite networks often provide coverage in many rural areas where other wireless networks may not be available.
- satellite communications have higher latencies compared to other wireless communications due the data having to travel long distances to the geostationary satellite.
- satellite communications provide very high performance for downlink data, performance of communications through satellite may not be satisfactory for uplink data, especially if uplink is done from many devices at once, i.e. from customers of the Internet service providers.
- wireless networks other than a satellite network may provide lower latency and may be cost effective.
- techniques are required for automatically enabling a user to use a satellite network for downlink and another wireless or wired network for uplink, while taking into account the asymmetric bandwidth and asymmetric latency.
- the embodiments of the present invention describe, in general, a first node and a second node communicating with each other through one or more tunnels.
- the first node transmits data to the second node mainly through a first tunnel group, and receives data from the second node mainly through a second tunnel group.
- the at least one tunnel in the first tunnel group and the at least one tunnel in the second tunnel group are formed using at least two network interfaces of the first node.
- the at least one tunnel in the first tunnel group is established through at least one network which, in general, has a network latency less than 150 milliseconds.
- the at least one tunnel in the second tunnel group is established through at least one satellite connection.
- the first node when network performance of the first tunnel group is below a first threshold, transmits data through the second tunnel group. When network performance of the second tunnel group is below a second threshold, the first node receives data through the first tunnel group.
- the tunnels in the first tunnel group and the tunnels in the second tunnel group are aggregated together to form one aggregated tunnel.
- the first node transmits or receives one or more tunnel management message is transmitted to or received from the second node.
- the tunnel management message may be sent from one node to another node for informing the another node which tunnels should be used for transmitting data and which tunnels should be use for receiving data.
- the first node when the first node receives first IP packets from a host through at least one of its Local Area Network (LAN) interface, the first node encapsulates the first IP packets in first encapsulating IP packets.
- the first node then transmits the first encapsulating IP packets mainly through a first one or more Wide Area Network (WAN) interfaces to a second node.
- the first node receives second encapsulating IP packets mainly through a second one or more WAN interfaces from the second node, and then decapsulates the second encapsulating IP packets to retrieve second IP packets.
- the second IP packets are transmitted through at least one corresponding LAN interface.
- the first one or more WAN interfaces are connected to a wireless network and the second one or more WAN interfaces are connected to a satellite connection.
- the first node further sends the second node information of the first one or more WAN interfaces and the second one or more WAN interfaces.
- FIG. 1 illustrates a network configuration according to various embodiments of the present invention.
- FIG. 2 illustrates tunnels established between a first node and a second node according to various embodiments of the present invention.
- FIG. 3 is an illustrative block diagram of a computer system or network node according to one of the embodiments of the present invention.
- FIG. 4 is an illustrative block diagram of a computer system or network node according to one of the embodiments of the present invention.
- FIG. 5 is an illustrative block diagram of a computer system or network node according to one of the embodiments of the present invention.
- FIG. 6A is a flowchart illustrating a process for a first node to transmit and receive data to and from a second node according to one of the embodiments of the present invention.
- FIG. 6B is a flowchart illustrating a process for a first node to transmit and receive data to and from a second node according to one of the embodiments of the present invention.
- FIG. 6C is a flowchart illustrating a process for a first node to transmit and receive data to and from a second node according to one of the embodiments of the present invention.
- FIG. 7 illustrates a network configuration according to various embodiments of the present invention.
- FIG. 8 illustrates the communication steps of a node communicating with a host according to one of the embodiments of the present invention.
- FIG. 9 illustrates the structure of IP packet being transmitted at each communication step according to one of the embodiments of the present invention.
- FIG. 10 illustrates a webpage used to configure a first node according to one of the embodiments of the present invention.
- FIG. 11 illustrates a webpage used to configure a first node according to one of the embodiments of the present invention.
- the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged.
- a process is terminated when its operations are completed, but could have additional steps not included in the figure.
- a process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
- Embodiments, or portions thereof, may be embodied in program instructions operable upon a processing unit for performing functions and operations as described herein.
- the program instructions making up the various embodiments may be stored in a computer readable storage medium.
- the term “computer readable storage medium”, “main memory”, or “secondary storage” may represent one or more devices for storing data, including read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), random access memory (RAM), magnetic RAM, core memory, floppy disk, flexible disk, hard disk, magnetic tape, CD-ROM, flash memory devices, a memory card and/or other machine readable mediums for storing information.
- the term “computer readable storage media” may also include, but is not limited to portable or fixed storage devices, optical storage mediums, magnetic mediums, memory chips or cartridges, wireless channels and various other mediums capable of storing, containing or carrying instruction(s) and/or data.
- a computer readable storage medium can be realized by virtualization, and can be a virtual computer readable storage medium including a virtual computer readable storage medium in a cloud-based instance.
- main memory or “secondary storage”, as used herein refers to any medium that participates in providing instructions to a processing unit for execution.
- the main memory or secondary storage is just one example of a machine-readable medium, which may carry instructions for implementing any of the methods and/or techniques described herein.
- Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media.
- Non-volatile media includes, for example, optical or magnetic disks.
- Volatile media includes dynamic memory.
- Transmission media includes coaxial cables, copper wire and fiber optics. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
- a volatile storage may be used for storing temporary variables or other intermediate information during execution of instructions by the processing unit.
- a non-volatile storage or static storage may be used for storing static information and instructions for the processing unit, as well as various system configuration parameters.
- the computer readable storage medium may include a number of software modules that may be implemented as software code to be executed by the processing unit using any suitable computer instruction type.
- the software code may be stored as a series of instructions or commands, or as a program in the computer readable storage medium.
- Various forms of computer readable storage media may be involved in carrying one or more sequences of one or more instructions to the processing unit for execution.
- the instructions may initially be carried on a magnetic disk from a remote computer.
- a remote computer can load the instructions into its dynamic memory and send the instructions to the system that runs the one or more sequences of one or more instructions.
- a processing unit may be a microprocessor, a microcontroller, a digital signal processor (DSP), any combination of those devices, or any other circuitry configured to process information.
- DSP digital signal processor
- a processing unit executes program instructions or code segments for implementing embodiments of the present invention. Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program instructions to perform the necessary tasks may be stored in a computer readable storage medium.
- a processing unit(s) can be realized by virtualization, and can be a virtual processing unit(s) including a virtual processing unit in a cloud-based instance.
- Embodiments of the present invention are related to the use of a computer system for implementing the techniques described herein.
- the inventive processing units may reside on a machine such as a computer platform.
- the techniques described herein are performed by computer system in response to the processing unit executing one or more sequences of one or more instructions contained in the volatile memory. Such instructions may be read into the volatile memory from another computer readable storage medium. Execution of the sequences of instructions contained in the volatile memory causes the processing unit to perform the process steps described herein.
- hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention.
- embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
- a code segment may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements.
- a code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents.
- Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
- a network interface that may be provided by a node is an Ethernet interface, a frame relay interface, a fibre optic interface, a cable interface, a DSL interface, a token ring interface, a serial bus interface, an universal serial bus (USB) interface, Firewire interface, Peripheral Component Interconnect (PCI) interface, etc.
- Ethernet interface a frame relay interface
- fibre optic interface a cable interface
- DSL interface a DSL interface
- a token ring interface a serial bus interface
- USB universal serial bus
- Firewire interface Firewire interface
- PCI Peripheral Component Interconnect
- a network interface may be implemented by an standalone electronic component or may be integrated with other electronic components.
- a network interface may have no network connection or at least one network connection depending on the configuration.
- a network interface may be an Ethernet interface, a frame relay interface, a fibre optic interface, a cable interface, a Digital Subscriber Line (DSL) interface, a token ring interface, a serial bus interface, an universal serial bus (USB) interface, Firewire interface, Peripheral Component Interconnect (PCI) interface, etc.
- a presently preferred embodiment of the present invention may utilize a gateway.
- a gateway is a device which performs protocol conversion between different types of networks or applications.
- the term gateway is not meant to be limited to a single type of device, as any device, hardware or software, that may act as a bridge between the user and the networks may be considered a gateway for purposes of this application.
- the gateway may couple with a plurality of multiple networks.
- a router, an access point or a wireless access point may all be considered a gateway for purposes of this invention.
- Embodiments, or portions thereof, may be embodied in a computer data signal, which may be in any suitable form for communication over a transmission medium such that it is readable for execution by a functional device (e.g., processing unit) for performing the operations described herein.
- the computer data signal may include any binary digital electronic signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic media, radio frequency (RF) links, and the like, and thus the data signal may be in the form of an electrical signal, optical signal, radio frequency or other wireless communication signal, etc.
- the code segments may, in certain embodiments, be downloaded via computer networks such as the Internet, an intranet, LAN, MAN, WAN, the PSTN, a satellite communication system, a cable transmission system, and/or the like.
- FIG. 3 is an illustrative block diagram of a computer system or network node, such as node 101 , according to one of the embodiments of the present invention.
- Node 101 comprises processing unit 301 , main memory 302 , system bus 303 , secondary storage 304 , local area network (LAN) interfaces 122 a and 122 b , and wide area network (WAN) interfaces 121 a , 121 b , and 121 c .
- Secondary storage 304 and main memory 302 are computer readable storage media.
- Processing unit 301 and main memory 302 are connected to each other directly.
- System bus 303 connects processing unit 301 directly or indirectly to secondary storage 304 , LAN interfaces 122 a and 122 b , and WAN interfaces 121 a , 121 b , 121 c , and 121 d .
- Using system bus 303 allows node 101 to have increased modularity.
- System bus 303 couples processing unit 301 to secondary storage 304 , and LAN interfaces 122 a and 122 b , and WAN interfaces 121 a , 121 b , 121 c , and 121 d .
- System bus 303 can be any of several types of bus structures including a memory bus, a peripheral bus, and a local bus using any of a variety of bus architectures.
- Secondary storage 304 stores program instructions for execution by processing unit 301 .
- the scope of the invention is not limited to node 101 having five network interfaces, such that node 101 may have more network interfaces.
- LAN interfaces 122 a and 122 b , and WAN interfaces 121 a , 121 b , 121 c , and 121 d are specified for illustration purposes only.
- FIG. 4 is illustrative block diagram of a computer system or network node, such as node 102 , according to one of the embodiments of the present invention.
- Node 102 comprises processing unit 401 , main memory 402 , system bus 403 , secondary storage 404 , LAN interface 132 , WAN interfaces 131 a , 131 b , and 131 c .
- the components of node 102 are connected to each other in a similar way as the components of node 101 .
- FIG. 5 is illustrative block diagram of a computer system or network node, such as node 702 , according to one of the embodiments of the present invention.
- Node 702 comprises processing unit 501 , main memory 502 , system bus 503 , secondary storage 504 , and WAN interface 703 .
- the components of node 702 are connected to each other in a similar way as the components of node 101 .
- node 702 may also be represented by the block diagram in FIG. 3 or FIG. 4 , such that node 702 may have one or more WAN interfaces, and one or more LAN interfaces.
- FIG. 1 illustrates a network configuration according to various embodiments of the present invention.
- Node 101 is connected to node 102 through four tunnels: tunnels 201 a , 201 b , 201 c and 201 d illustrated in FIG. 2 .
- LAN interface 122 a connects node 101 to host 103 a through interconnected networks 117 using connections 115 a and 115 c .
- LAN interface 122 b connects node 101 to host 103 b using connection 115 b.
- node 101 connects to a satellite network through WAN interface 121 a for establishing tunnel 201 a with node 102 .
- WAN interface 121 a communicates with satellite network 150 by using satellite modem 161 ;
- node 102 connects to satellite network 150 through WAN interface 131 a and satellite modem 160 for establishing tunnel 201 a with node 101 .
- connection between WAN interface 121 a and satellite network 150 and the connection between WAN interface 131 a and satellite network 150 are using satellite modems 160 and 161 respectively.
- Satellite modems 160 and 161 may have embedded antenna or external antenna used for communicating with satellite network 150 through connections 112 a and 110 a respectively.
- WAN interfaces 131 a and 121 a connect to satellite modems 160 and 161 respectively through wired or wireless connection medium 114 and 113 respectively.
- connection medium 114 and 113 are fast-speed wired connection medium, such as a computer bus, Universal Serial Bus (USB), serial bus, parallel bus, fiber optics, FireWire, Thunderbolt, and etc.
- Satellite modem 160 can also be included within a housing that contains node 102
- satellite modem 161 can be included within a housing that contains node 101 .
- node 102 can include an auxiliary port into which satellite modem 160 is plugged.
- Satellite modems 160 and 161 are configured to transmit and receive communications to and from satellite network 150 .
- Satellite modems 160 and 161 may be selected from any number of available satellite modems, including but not limited to: Quake Global; Comtech EF Data; Datum Systems; Hughes Network Systems; Newtec; Paradise Datacom; Radyne ComStream; Shiron Satellite Communications; AdvantechAMT (formerly Signal Processors, SPL/ACT, ACT Wireless); and TSI Technology.
- node 101 establishes tunnel 201 b with node 102 through WAN interface 121 b , wireless network 180 and WAN interface 131 b .
- WAN interface 121 b communicates with wireless network 180 through a wireless communication channel 110 b .
- WAN interface 131 b communicates with wireless network 180 through wireless communication channel 112 b .
- Communication through wireless communication channel 110 b and 112 b can be realized through a wireless modem or a cellular modem.
- the size of wireless network 180 is not limited.
- wireless network 180 may be a cellular network operated by Verizon, ATT, China Mobile, or Vodafone.
- a cellular network can also be a mobile virtual network operator (MVNO) or mobile other licensed operator (MOLO) that provides a wireless communications services but does not own the wireless or cellular network infrastructure over which the MVNO provides services to its customers.
- MVNO mobile virtual network operator
- MOLO mobile other licensed operator
- node 101 establishes tunnel 201 c with node 102 through WAN interface 121 c , wireless network 181 , interconnected network 171 and WAN interface 131 c .
- WAN interface 121 c communicates with wireless network 181 through wireless communication channel 110 c .
- Wireless network 181 communicates with WAN interface 131 c through interconnected networks 171 .
- the size of wireless network 181 is also not limited.
- wireless network 181 may be a cellular network operated by Verizon, AT&T, China Mobile, or Vodafone.
- Wireless network 181 and interconnected networks 171 communicate with each other through connection 111 a .
- Node 102 communicates with interconnected networks 171 through connection 112 c.
- node 101 establishes tunnel 201 d with node 102 through WAN interface 121 d , interconnected networks 171 and WAN interface 131 c .
- WAN interface 121 d communicates with interconnected networks 171 through connection 111 b .
- Node 102 uses WAN interface 131 c to communicate with interconnected networks 171 through connection 112 c.
- Wireless networks 180 and 181 are preferably configured for wireless digital transfer using a suitable digital data transfer protocol such as High Speed Downlink Packet Access (HSDPA) and High-Speed.
- HSDPA High Speed Downlink Packet Access
- High-Speed High Speed Downlink Packet Access
- wireless networks 180 and 181 are configured for wireless digital transfer using Worldwide Interoperability for Microwave Access (WiMAX), Local Multipoint Distribution Service (LMDS), Multichannel Multipoint Distribution Service (MMDS), IEEE 802.11 technologies, such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE 802.11ac, and/or IEEE 802.11ad.
- WiMAX Worldwide Interoperability for Microwave Access
- LMDS Local Multipoint Distribution Service
- MMDS Multichannel Multipoint Distribution Service
- IEEE 802.11 technologies such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE 802.11ac, and/or IEEE 802.11ad.
- WAN interfaces 121 b , 121 c and 131 b can be realized by using a connection through cellular phones, cellular modems, and/or wireless communication modems.
- a cellular modem such as a 3GPP LTE modem, is capable of communicating with one or more cellular networks.
- a cellular modem can be an embedded cellular modem or an external cellular modem.
- Connections 110 a , 112 a , 112 c , 111 a , 111 b , 115 a , 115 b , and 116 can be realized by using any type of wired or wireless connection medium.
- wired connection mediums may include Ethernet, fiber channel, digital subscriber loop, cable modem, frame relay, token ring, serial bus, USB, Firewire, PCI, etc.
- Wireless connection mediums may include a wireless link such as Wi-FiTM, a wireless connection using a wireless communication protocol such as IEEE 802.11 (wireless Ethernet), Bluetooth, etc.
- Interconnected networks 117 , 171 , and 172 can be public interconnected network, such as the Internet, private interconnected network or hybrid public and private interconnected networks.
- both interconnected networks 117 and 172 can be local area networks (LAN) connected together through tunnels 201 a , 201 b , 201 c and 201 d .
- interconnected networks 117 is a LAN and interconnected networks 172 is the Internet, such that node 102 performs as a gateway for interconnected networks 117 to communicate with the Internet through at least of the tunnel of tunnels 201 a , 201 b , 201 c and 201 d.
- a tunnel such as tunnels 201 a , 201 b , 201 c and 201 d , is established between two networks.
- hosts from two different networks can communicate as in the same network, such as in the same IP subnet.
- the tunnels can be implemented using Secure Sockets Layer (SSL), L2TP, Internet Protocol Security (IPSec) and SSL, IPSec and Layer 2 Tunneling Protocol (L2TP) or Point-to-Point Tunneling Protocol (PPTP).
- SSL Secure Sockets Layer
- IPSec Internet Protocol Security
- L2TP Layer 2 Tunneling Protocol
- PPTP Point-to-Point Tunneling Protocol
- One or more tunnels can be established between two network interfaces, such as WAN interface 121 c and 131 c . Therefore, there could be more than the four tunnels, i.e.
- tunnels 201 a , 201 b , 201 c and 201 d to be established between node 101 and node 102 .
- WAN interface 121 b may also establish another tunnel with WAN interface 131 c if wireless networks 180 can communicate with interconnected networks 171 .
- Data can be in stored in payload of data packets.
- Node 101 encapsulates data packets in encapsulating packets, and then transmit encapsulating packets to node 102 through one or more tunnels.
- node 102 receives the encapsulating packets, node 102 decapsulates the encapsulating packets to retrieve the data packets.
- Encapsulating packets may be distributed among the plurality of tunnels, i.e. tunnels 201 a , 201 b , 201 c and 201 d.
- node 102 when node 102 needs to transmit data packets to node 101 , node 102 first encapsulates data packets in encapsulating packets, and then transmits encapsulating packets to node 101 through the one or more tunnels. When node 101 receives the encapsulating packets, node 101 decapsulates the encapsulating packets to retrieve the data packets.
- tunnels enables the encapsulation of data from one type of protocol within the datagram of the same or different protocol.
- tunnels can be use to implement virtual private networks (VPN).
- the tunnels between node 101 and 102 are aggregated to form an aggregated tunnel or an aggregated VPN connection.
- tunnels 201 a , 201 b and 201 c are aggregated to form one aggregated VPN connection.
- the aggregated VPN connection allows, for example, data packets belonging to one TCP session are transmitted and received through tunnels 201 a , 201 b and 201 c to take advantage of the availability of the networks connecting to WAN interfaces 121 a , 121 b and 121 c.
- a first node forms at least two tunnel groups with a second node
- one of the tunnel groups is mainly used to transmit data packets from the first node to the second node and another tunnel group is mainly used to receive data packets from the second node.
- each tunnel group there can be one or more tunnels.
- there are at least two tunnel groups and each tunnel group has at least one tunnel there are at least two tunnels established between the first node and the second node.
- the at least two tunnels are established through two of network interfaces of the first node and at least one network interface of the second node.
- the second node may perform as a gateway for the first node.
- a tunnel group when a tunnel group is mainly used to transmit data packets from the first node to the second node, there is no data packet being transmitted from the second node to the first node through the tunnel group.
- a tunnel group when a tunnel group is mainly used to transmit data packets from the first node to the second node, there is a small number of packets being transmitted from the second node to the first node through the tunnel group as those packets are used for managing the tunnel group, such as transmitting health check packets, establishing one of the tunnels in the tunnel group and sending status information.
- a tunnel group when a tunnel group is mainly used to receive data packets by the first node from the second node, there is no data packet being transmitted by the first node to the second node through the tunnel group.
- a tunnel group when a tunnel group is mainly used to receive data packets by the first node from the second node, there is a small number of packets being transmitted from the first node to the second node through the tunnel group as those packets are used for managing the tunnel group, such as transmitting health check packets, establishing one of the tunnels in the tunnel group and sending status information.
- the first node is node 101 and the second node is node 102 ; a first tunnel group comprises tunnels 201 b and 201 c ; and a second tunnel group comprises tunnels 201 a and 201 d .
- the first tunnel group is used to transmit data packets from node 101 to node 102 while the second tunnel group is used to transmit data packets from node 102 to node 101 Therefore node 102 may perform as a gateway for node 101 and the networks connected to the node 101 , such as network 117 , host 103 a and hosts 103 b , to communicate with interconnected network 172 .
- node 101 may use a plurality of tunnels in the first tunnel group to transmit data packets to node 102 .
- node 102 receives the data packets through one or more tunnels of the first tunnel group, node 102 reorders the packets before transmitting the packets to interconnected network 172 if necessary.
- node 102 may use a plurality of tunnels in the second tunnel group to transmit data packets to node 101 .
- node 102 When a host in interconnected networks 172 sends a packet to host 103 a , node 102 sends the packet through one or more tunnels of the second tunnel group to node 101 . Then node 101 sends the packet to node 103 a through interconnected networks 117 .
- node 102 may use a plurality of tunnels in the second tunnel group to transmit data packets.
- the data packets belong to one session and node 101 reorders the packets before transmitting the packets to interconnected network 117 or host 103 b .
- node 101 reorders the packets before transmitting the packets to interconnected network 117 or host 103 b .
- tunnel 201 b when a tunnel is used to transmit data packets from node 101 to node 102 , the same tunnel is not used to transmit data packets from node 102 to node 101 .
- tunnel 201 b when tunnel 201 b is used to transmit data packets from node 101 to node 102 , tunnel 201 b is not used to transmit data packets from node 102 to node 101 .
- one or more of other tunnels such as tunnel 201 a , tunnel 201 c or/and 201 d , have to be used.
- a tunnel management message is sent by node 101 to instruct node 102 not to use tunnel 201 b for transmitting data packets to node 101 .
- node 102 can overrule the instruction sent by node 101 . The ability of overruling the instruction allows administrators of node 101 and 102 to have more flexibility to adapt to changing network environment.
- node 102 can still transmit a portion of data packets to node 101 through tunnel 201 b and the portion can be preconfigured by the administrator of node 102 , administrator of node 102 and/or negotiated between node 101 and node 102 .
- the corresponding WAN interface establishing the tunnel at node 101 is not used to receive data packets from node 102 .
- WAN interface 121 b is not used to receive data packets from node 102 .
- the corresponding WAN interface establishing the tunnel at node 101 is not used to transmit data packets to node 102 .
- tunnel 201 a when tunnel 201 a is used to receive data packets from node 102 to node 101 , WAN interface 121 a is not used to transmit data packets to node 101 .
- WAN interface 121 a There could be many reason why to have a WAN interface for transmitting or receiving data packets only, including the WAN interface is half-duplex, the network performance of the connection connected to the WAN interface is asymmetric, the cost of transmitting or receiving data packets through the connection connected to the WAN interface is asymmetric, and etc.
- node 101 transmits tunnel management messages to node 102 for managing one or more tunnels.
- the usage of tunnel management message includes: informing node 102 that which group a tunnel belongs to, informing node 102 how to use a tunnel, informing node 102 when to transmit data packets through a particular tunnel or tunnel group.
- node 101 receives tunnel management messages from node 102 for managing one or more tunnels. Therefore, the usage of tunnel management message also includes: informing node 101 that which group a tunnel belongs to, informing node 101 how to use a tunnel, informing node 101 when to transmit data packets through a particular tunnel or tunnel group.
- a tunnel management message comprises a tunnel identity field and an instruction field.
- a tunnel identity field comprises the identities of the first tunnel, the second tunnel, and the third tunnel and the instruction field comprises information that the first tunnel and the third tunnel belong to the first tunnel group while a second tunnel belongs the second tunnel group. Therefore, after receiving the tunnel management message, node 102 can expect data packets to arrive through the first and the third tunnels and does not use the first and the third tunnels to transmit data packets to node 101 while only transmits data packets to node 101 through the second tunnel.
- Tunnel management messages may be sent when tunnels are about to be established, are being established or after tunnels have been established.
- a tunnel management message comprises information that assists node 102 to identify the tunnels and the corresponding tunnel groups in the instruction field and the identities of the corresponding tunnels in the tunnel identity field.
- the tunnel message can only be sent by node 101 , such that node 102 can also send the tunnel message.
- a tunnel message may comprise information that tunnels 201 a and 201 b belong to a first tunnel group and tunnels 201 c and 201 d belongs to a second tunnel group.
- the tunnel management messages also comprises an indicator to indicate whether a tunnel group is for both transmission and receiving, transmission only or receiving only. For example, an indicator indicates that a first tunnel group is for transmission only and a second tunnel group is for both transmission and receiving.
- An indicator can be represented by a bit, a plurality of bits, a byte, a plurality of bytes, a string, a plurality of strings, XML messages, etc. Those who are skilled in the arts would appreciate that there are myriads ways to represent the indicator.
- one or more tunnels of the first tunnel group and one or more tunnels of the second tunnel group are aggregated to form one aggregated tunnel.
- node 101 transmits data packets through the aggregated tunnel to node 102
- node 101 uses the first tunnel group to transmit data packets.
- tunnels 201 a and 201 b belong to the first tunnel group
- node 101 transmits data packets through tunnels 201 a and 201 b to node 102 .
- node 102 uses the second tunnel group to transmit data packets as instructed by the tunnel management messages.
- node 102 transmits data packets through tunnels 201 c and 201 d to node 101 .
- the tunnel management message can also be used to manage the aggregated tunnel. For example, when a tunnel is added to the first tunnel group, node 101 sends a tunnel management message to node 102 that there is one more tunnel in the aggregated tunnel.
- a tunnel management message comprises an indicator in the instruction field to indicate whether a tunnel is for both transmission and receiving, transmission only or receiving only.
- tunnel 201 a is used for node 101 to receive data packets from node 102 only; tunnels 201 b , 201 c and 201 d are used for node 101 to transmit data packets to node 102 only. Therefore the indicator indicates that tunnel 201 a is for receiving data packets only and tunnels 201 b , 201 c and 201 d are used for transmitting data packets only.
- Node 102 then transmits data packets to node 101 through tunnel 201 a and receives data from node 102 through tunnels 201 a , 201 b and 201 c according to the information in the tunnel management message.
- a tunnel management message comprise status information of a tunnel.
- the status information can be part of information that is stored in the instruction field.
- a tunnel management message also comprises a status field.
- the status field is used to hold information of status of the tunnel. For example, when node 101 updates node 102 about the status of a tunnel, node 101 sends a tunnel management message with the tunnel status in the status field to node 102 .
- the tunnel identity field holds the identity of the corresponding tunnel.
- the instruction field in this case may be empty as the tunnel management message is for status reporting purpose.
- a tunnel can be used by node 101 to transmit and receive data packets to and from node 102 . Therefore the indicator in the tunnel management messages shows that the tunnel can be used for both transmission and receiving.
- a tunnel is preferably used by node 101 to transmit data packets to node 102 and is not preferably used by node 101 to receive data packets from node 102 even though the tunnel is capable for transmission and receiving. Therefore the indicator in the tunnel management messages shows the corresponding preference.
- another tunnel is preferably used by node 102 to transmit data packets to node 101 and is not preferably used by node 102 to receive data packets from node 101 even though the tunnel is capable for transmission and receiving. Therefore the indicator in the tunnel management messages shows the corresponding preference of the another tunnel.
- the first tunnel group comprises one or more tunnels connecting to one or more networks and each network should have latency less than 150 milliseconds.
- network latency is limited to less than 150 milliseconds, the network is considered as a fast network. More importantly, 150 milliseconds network latency is lower than the usual network latency of satellite communications.
- network latency of the first tunnel group is more than 150 milliseconds, the advantage for node 101 to use the first tunnel group to transmit data packets to node 102 becomes not obvious as the network latency of the first tunnel group is not significantly better than the network latency of the second tunnel group if the second tunnel group is connected to one or more satellite connections.
- a tunnel management message is sent by either or both of node 101 and node 102 to other node that a tunnel group that was originally being used for receiving only, can then be used for both receiving and transmission, or a tunnel group that was originally being used for transmission only, can then be used for both receiving and transmission.
- the value of latency can be determined based on the average, maximum, minimum or other statistical calculations of latency observed for each tunnel in the tunnel group.
- node 101 uses only one tunnel of the first tunnel group at a time to transmit data to node 102 .
- the tunnel is selected from tunnels of the first tunnel group according to predefined criteria, such as network latency, packet drop rate, bandwidth, price, signal strength and etc.
- the predefine criteria is entered by the administrator of node 101 through a web interface, command line, system console, and/or any other data input mechanism.
- the predefine criteria is sent to node 101 through an Internet connection, an intranet connection or SMS from a remote server.
- the predefined criteria can be stored in a storage medium of node 101 , such as secondary storage 304 or main memory 302 .
- the predefined criteria are first retrieved from a remote host and then stored in a storage medium of node 101 such as secondary storage 304 or main memory 302 .
- node 101 uses link load-balancing technique to transmit data packet to node 102 .
- node 101 uses the second tunnel group for transmitting data when a first condition(s) is met. For example, when node 101 is unable to use the first tunnel group for transmitting data to node 102 or the performance of the first tunnel group becomes unacceptable, the first condition(s) is met, and node 101 uses the second tunnel group for transmitting data. In one of the embodiments, node 101 uses the first tunnel group for receiving data when a second condition(s) is met. For example, when node 101 is unable to use the second tunnel group for receiving data from node 102 or the performance of the second tunnel group becomes unacceptable, the second condition(s) are met, and node 101 uses the first tunnel group for receiving data.
- condition(s) allows flexibility that a tunnel group can be reconfigured when status or network performance of the tunnel group or another tunnel group changes.
- the first and second conditions are based on at least one of the following criteria: coverage, performance, traffic congestion, and latency sensitivity.
- coverage when the first condition is based on coverage, the first condition is met when the location of node 101 is out of the coverage of one or more networks that the first tunnel group connects to.
- the condition when the condition is met, node 101 cannot use the first tunnel group for transmitting data, and therefore node 101 uses the second tunnel group to transmit data.
- the second condition is based on coverage and is met when the location of node 101 is out of the coverage of one or more networks that the second tunnel group connects to. When the second condition is met, node 101 cannot use the second tunnel group for receiving data, and therefore node 101 uses the first tunnel group to receive data.
- node 101 is out of coverage of a network. For example, when one or more tunnels of the first tunnel group connects to a cellular network which does not provide coverage outside a specific country or region, node 101 becomes out of the coverage of the cellular network when node 101 moves out of the specific country or region. Using roaming services may not be cost effective in some scenarios, and hence node 101 starts using the second tunnel group for transmitting data. In another example, node 101 may lose coverage of one or more networks that the second tunnel group connects to, depending on its location, such as when it moves outdoors, or when it moves indoors.
- the second tunnel group comprises tunnel 201 a which is established through satellite network 150 .
- node 101 may not be able to receive data through tunnel 201 a belonging to the second tunnel group from the satellite, then node 101 uses the first tunnel group to receive data from node 102 .
- the first condition is based on performance and is met when the performance experienced by data transmitted or received through the first tunnel group becomes very poor and unsatisfactory. For example, node 101 starts using the second tunnel group for transmitting data when the first condition is met.
- the second condition is based on performance and is met when the performance experienced by data received through the second tunnel group becomes very poor. For example, node 101 starts using the first tunnel group for receiving data when the second condition is met.
- the first condition when the first condition is based on latency sensitivity, the first condition is met when a data transfer is latency insensitive.
- a data transfer When a data transfer is latency insensitive, it can use tunnels with high latency. Therefore, when the first condition is met and the data transfer is latency insensitive, node 101 can use the second tunnel group for transmitting data since the high latency of the second tunnel group is acceptable for the latency insensitive data transfer.
- the second tunnel group may have high latency while transmitting data when tunnel 201 a is in the second tunnel group because, for example, tunnel 201 a is established through satellite network 150 .
- node 101 uses the first tunnel group for receiving data from node 102 .
- a data transfer is latency sensitive, it is preferable to use tunnels with low latency.
- the data being transferred is generated by a real-time audio conversion application and any delay in transmitting or receiving the data will make the audio conversion difficult to be listened to.
- the second tunnel group comprising tunnel 201 a may have higher latency than the first tunnel group. Therefore the first tunnel group is used by node 101 to receive data belonging to a latency sensitive data transfer from node 102 .
- first conditions There can be one or more first conditions and one or more second conditions.
- the first condition(s) and second condition(s) may or may not be based on the same criteria.
- a condition can be based on one or more criteria.
- the network latency of the first tunnel group is the average latency, the maximum latency, the minimum latency, or the latency variance of the latencies of the more than one tunnels.
- the network latency of the first tunnel group is used to determine whether the first tunnel group should be used for transmitting data, then the determination is based on the average latency, the maximum latency, the minimum latency, or the latency variance of the latencies of the more than one tunnels belonging to the first tunnel group.
- the determination is based on the average latency, the maximum latency, the minimum latency, or the latency variance of the latencies of the more than one tunnels belonging to the second tunnel group.
- the administrator of node 101 and/or node 102 can determine which kind of latency suits their needs.
- processing unit 301 determines whether the first tunnel group should be used for transmitting data, based on performance criterion/criteria.
- the performance criterion/criteria is/are selected from a group comprising packet delay, bandwidth, throughput, packet loss, packet drop, power consumption, and signal noise ratio, round-trip time, interference level, error rate, quality of service, queuing delay, and packet jitter.
- a threshold is predefined for the selected performance criteria, and when the threshold is not satisfied, processing unit 301 determines to use the second tunnel group for transmitting data. Processing unit 301 keeps monitoring the performance of the tunnels of the first tunnel group in order to determine whether the threshold is satisfied or not.
- processing unit 301 determines whether the second tunnel group should be used for receiving data, based on performance criterion/criteria.
- the performance criterion/criteria is/are selected from a group comprising network latency, response time, packet delay, bandwidth, throughput, packet loss, packet drop, power consumption, and signal noise ratio, round-trip time, interference level, error rate, quality of service, queuing delay, and packet jitter.
- a threshold is predefined for the selected performance criteria, and when the threshold is not satisfied, processing unit 301 determines to use the first tunnel group for receiving data. Processing unit 301 keeps monitoring the performance of the tunnels of the second tunnel group in order to determine whether the threshold is satisfied or not.
- processing unit 301 selects tunnels for the first tunnel group and the second tunnel group based on one or more performance criteria.
- the performance criteria are selected from a group comprising network latency, response time, packet delay, bandwidth, throughput, packet loss, packet drop, power consumption, signal noise ratio, round-trip time, interference level, error rate, quality of service, queuing delay, usage price, location and packet jitter.
- thresholds are predefined for the selected performance criteria corresponding to the first tunnel group and the selected performance criteria corresponding to the second tunnel group. For example, processing unit 301 determines whether tunnel 201 b satisfies the threshold for the selected performance criteria corresponding to the first tunnel group.
- tunnel 201 b satisfies the threshold for the selected performance criteria corresponding to the first tunnel group
- tunnel 201 b belongs to the first tunnel group.
- processing unit 301 determines whether tunnel 201 a satisfies the threshold for the selected performance criteria corresponding to the second tunnel group. If tunnel 201 a satisfies the threshold for the selected performance criteria corresponding to the second tunnel group, tunnel 201 a belongs to the second tunnel group.
- the performance criteria for selecting tunnels for the first tunnel group and the performance criteria for selecting tunnels for the second tunnel group may or may not be the same.
- a tunnel does not belong to any tunnel group if its performance does not meet the performance criterion/criteria of the tunnel group.
- processing unit 301 keeps monitoring performance of each of the tunnels belonging to the first tunnel group and the second tunnel group.
- tunnel 201 b and tunnel 201 c belong to the first tunnel group.
- the selected performance criteria corresponding to the first tunnel group are network latency and throughput.
- the threshold defined for network latency is 150 milliseconds, such that the network latency of a network that a tunnel connects to must be less than 150 milliseconds.
- the threshold for throughput is 20 Mbps, such that the throughput of data transmission through a tunnel should in general equal to or more than 20 Mbps.
- Processing unit 301 periodically determines whether tunnel 201 b satisfies the thresholds for network latency and throughput.
- processing unit 301 removes tunnel 201 b from the first tunnel group because tunnel 201 b does not satisfy the threshold. Similarly, if the throughput of data transmission through tunnel 201 b becomes in general less than 20 Mbps, processing unit 301 removes tunnel 201 b from the first tunnel group because tunnel 201 b does not satisfy the threshold.
- tunnel 201 b should satisfy thresholds of both the performance criteria for remaining in the first tunnel group. Alternatively, tunnel 201 b should satisfy threshold of at least one of the performance criteria for remaining in the first tunnel group.
- processing unit 301 determines to removes tunnel 201 b from the first tunnel group only after tunnel 201 b has not been satisfying the thresholds of the performance criteria for a predefined time period. For example, processing unit 301 determines periodically whether tunnel 201 b should remain in the first tunnel group. If tunnel 201 b does not satisfy the threshold of the performance criteria during three consecutive determinations of processing unit 301 , then processing unit 301 removes tunnel 201 b from the first tunnel group. In another example, if processing unit 301 determines that network latency of a network that tunnel 201 b connects to has been more than 150 milliseconds for more than fifteen minutes, processing unit 301 removes tunnel 201 b from the first tunnel group. When tunnel 201 b is removed from the first tunnel group, processing unit 301 uses tunnel 201 c of the first tunnel group to transmit data.
- Throughput of a tunnel can be determined by processing unit 301 by transmitting data packets or testing packets through the tunnel.
- the throughput estimated can be average throughput, minimum throughput maximum throughput.
- the format of the testing packets is preferably the same as the format of data packets that are transmitted using tunnel 201 b.
- tunnel 201 a and 201 d belong to the second tunnel group.
- the selected performance criteria corresponding the second tunnel group are usage price and packet drop rate. As some carriers may charge higher usage price after a tunnel connects to a network for a certain time period, the usage price may increase while a tunnel is established.
- Processing unit 301 periodically determines whether the usage price of tunnel 201 a and 201 d is under a threshold. If the usage price of a tunnel, such as tunnel 201 a , becomes above the threshold, processing unit 301 removes tunnel 201 a from the second tunnel group. When tunnel 201 a is removed from the second tunnel group, node 101 uses tunnel 201 d for receiving data.
- Processing unit 301 also monitors packet drop rate through tunnels 201 a and 201 d .
- packet drop rate of a tunnel such as tunnel 201 d
- processing unit 301 removes tunnel 201 d from the second tunnel group.
- tunnel 201 a should satisfy thresholds of both the performance criteria for being kept in the second tunnel group.
- tunnel 201 a should satisfy threshold of at least one of the performance criteria for being kept in the second tunnel group.
- a tunnel belongs to a tunnel group but is not being utilized.
- tunnels 201 b , 201 b and 201 c belong to the first tunnel group and tunnel 201 a belongs to the second tunnel group.
- the first tunnel group is used to transmit data from node 101 to node 102 and the second tunnel group is used to transmit data from node 102 to node 101 . It is possible that not all tunnels 201 b , 201 b and 201 c are used at the same time to transmit data from node 101 to node 102 .
- only one of the tunnels 201 b , 201 b and 201 c is used to transmit data while the other two tunnels are used for backup or hot-failover purpose.
- two of the tunnels 201 b , 201 b and 201 c are used to transmit data while the remaining one is not used.
- the decision of selecting tunnel(s) for utilization and how to use the tunnels can be predefined by administrator of node 101 , administrator of node 102 or the system operator operating both nodes 101 and 102 .
- the decision of how data is being distributed among the utilized tunnels can be defined before the tunnels are set, during the establishment of the tunnels and/or anytime after the tunnels are established.
- a priority is assigned to each tunnel of a tunnel group by node 101 .
- tunnels 201 b , 201 c and 201 d belong to the first tunnel group.
- Node 101 assigns highest priority to tunnel 201 b , second-highest priority to tunnel 201 c and lowest priority to tunnel 201 d .
- Node 101 uses any two tunnels belonging to the first tunnel group at a given time for transmitting data. Therefore, since tunnels 201 b and 201 c have a higher priority than tunnel 201 d , node 101 uses tunnels 201 b and 201 c for transmitting data when possible.
- connection through tunnel 201 b fails, node 101 then starts using tunnel 201 c and 201 d for transmitting data.
- the connection through tunnel 201 b may fail for various reasons, such as traffic congestion, tunnel 201 b not having enough capacity, no network coverage, or equipment malfunctioning.
- performance of data transmission through tunnel 201 b deteriorates significantly, node 101 stops using tunnel 201 b and starts using tunnels 201 c and 201 d for transmitting data.
- tunnels 201 a and 201 b belong to the second tunnel group.
- Node 101 assigns highest priority to tunnel 201 a and lowest priority to tunnel 201 b .
- node 101 is configured with a predefined configuration to assign highest priority to tunnel 201 a in the second tunnel group because tunnel 201 a is established through satellite network 150 .
- node 101 assigns the highest priority to tunnel 201 a in the second tunnel group because tunnel 201 a has the highest throughput and largest coverage area. If a connection through tunnel 201 a fails, node 101 starts using tunnel 201 b for receiving data. The connection through tunnel 201 a may fail for various reasons.
- the connection through tunnel 201 a may fail.
- the network latency for receiving data through tunnel 201 a may become very high and the connection may fail because of high network latency.
- Priority is assigned to tunnels in a tunnel group according to one or more performance criteria or can be assigned according to a predefined configuration.
- a predefined configuration may be entered by an administrator or a user of node 101 through a web interface, an application programming interface (API), a command line interface or a console.
- API application programming interface
- node 101 determines performance of each tunnel in a tunnel group. For example, processing unit 301 may assign highest priority to tunnel 201 b in the first tunnel group because tunnel 201 b has the highest throughput and the lowest packet drop rate. Alternatively, processing unit 301 may assign highest priority to tunnel 201 b in the first tunnel group because tunnel 201 b has the lowest usage price.
- node 101 when performance of a tunnel in a tunnel group deteriorates, the tunnel is not removed from the tunnel group, but node 101 determines not to use the tunnel. For example, when performance of a tunnel, such as tunnel 201 b , belonging to the first tunnel group, deteriorates and becomes unsatisfactory, node 101 stops using tunnel 201 b for transmitting data until the performance of tunnel 201 b becomes satisfactory again. Tunnel 201 b is marked as inactive and node 101 uses other tunnel(s) in the first tunnel group, such as tunnel 201 c , to transmit data. Node 101 may determine that performance of tunnel 201 b has deteriorated based on one or more performance criteria. For example, when packet drop rate of data transmission through tunnel 201 b has increased significantly, node 101 stops using tunnel 201 b for transmitting data.
- tunnel 201 a when performance of a tunnel, such as tunnel 201 a , belonging to the second tunnel group, deteriorates and becomes unsatisfactory, node 101 stops using tunnel 201 a for receiving data until the performance of tunnel 201 a becomes satisfactory again.
- Tunnel 201 a is marked as inactive and node 101 uses other tunnel(s) in the second tunnel group, such as tunnel 201 d , to receive data.
- Node 101 may determine that performance of tunnel 201 a has deteriorated based on one or more performance criteria. For example, when latency of tunnel 201 a increases significantly, node 101 stops using tunnel 201 a for receiving data.
- the benefits of not removing a tunnel whose performance has deteriorated from a tunnel group includes that node 101 may be to be able to start using the tunnel again as soon as the performance of the tunnel becomes satisfactory. Processing unit 301 does not need to determine again as to whether the tunnel should belong to the tunnel group or not.
- a tunnel may belong to both the first tunnel group and the second tunnel group.
- tunnel 201 b belongs to both the first tunnel group and the second tunnel group.
- Tunnel 201 b is then used by node 101 for transmitting and receiving data to and from node 102 respectively.
- the first tunnel group comprises tunnels 201 b and 201 c
- the second tunnel group comprises tunnels 201 a and 201 b .
- Node 101 uses tunnels 201 b and 201 c for transmitting data, and tunnels 201 a and 201 b for receiving data.
- node 101 uses tunnel 201 c only for transmitting data, tunnel 201 a only for receiving data, and tunnel 201 b for both transmitting and receiving data.
- Processing unit 301 may select tunnel 201 b for belonging to both the first tunnel group and the second tunnel group because tunnel 201 b has very good performance, such as high throughput, low packet drop rate, low latency, low usage price, or good performance based on other performance criteria.
- processing unit 301 of node 101 or processing unit 401 of node 102 determines that a tunnel leaves or joins a tunnel group, new tunnel management messages are sent by the processing unit to inform the other processing unit that the tunnel should be used or not and how to use the tunnel.
- tunnel management messages indicate that the first tunnel group is only used by node 101 to transmit data to node 102
- the first tunnel group is not allowed to be used for receiving data from node 101 to node 102 even if a corresponding condition has been met.
- the tunnel management message indicates that the second tunnel group is only used by node 101 to receive data from node 102
- the second tunnel group is not allowed to be used for transmitting data to node 102 even if a corresponding condition has been met.
- new tunnel management messages are sent by node 101 or node 102 to update that the first tunnel group and/or the second tunnel group can be used for bidirectional communications.
- new tunnel management messages are sent by node 101 or node 102 to update that the first tunnel group and/or the second tunnel group are no longer used for bidirectional communications and can only used for transmitting data from node 101 and/or receiving data by node 101 respectively.
- new tunnel management messages allow the flexibility to change how the first tunnel group and the second tunnel group is used for transmitting and receiving data after the tunnel have been established.
- the new tunnel management messages are used to manage one or more specific tunnel, instead of a tunnel group. This allows more granular management of tunnel.
- node 101 in the case that node 101 is no longer able to or determines not to receive data through the second tunnel group or one of the tunnels of the second tunnel group, such as in the situation of being out of coverage, node 101 will inform node 102 , by using tunnel management message, not to further transmit data through the second tunnel group or the one of the tunnels of the second tunnel group.
- node 102 in the case that node 102 is no longer able to or determines not to receive data through the first tunnel group or one of the tunnels of the first tunnel group, such as in the situation of heavy packet loss, node 102 will inform node 101 , by using tunnel management message, not to further transmit data through the first tunnel group or the one of the tunnels of the first tunnel group.
- FIG. 6A and FIG. 6B are two flowcharts illustrating one of the embodiments of the present invention that node 101 transmits and receives data packets to and from node 102 respectively.
- node 101 first selects a first one or more WAN interfaces of node 101 , such as WAN interface 121 a , 121 b , 121 c and 121 d , at step 601 .
- node 101 receives a data packet from a host or node that connects to one of its LAN interfaces 122 a and 122 b at step 602 , it encapsulates the data packet into one or more encapsulating packets at step 603 .
- the step of encapsulating a data packet allows the data packet be sent through a tunnel.
- step 601 and step 602 can be swapped such that the selection of WAN interface is performed after a data packet has been received.
- the first one or more WAN interfaces of node 101 selected are mainly used for transmitting data packets from node 101 to node 102 .
- node 102 does not transmit data packets to node 101 through the first one or more WAN interfaces of node 101 .
- node 102 transmits a small number of packets to node 101 through the first one or more WAN interfaces of node 101 , and the packets are used for managing purpose, such as status information of the connections connected to the first one or more WAN interfaces.
- node 101 transmits the one or more encapsulating packets through the first one or more WAN interfaces selected at step 601 .
- the first one or more WAN interfaces may connect to a wired or wireless network.
- Node 101 also selects a second one or more WAN interfaces mainly for receiving data packets from node 102 .
- the data packets from node 102 may be encapsulating packets.
- node 102 only transmits data packets to node 101 through the second one or more WAN interfaces of node 101 , such that node 101 only receives data packets from node 102 through the second one or more WAN interfaces of node 101 .
- node 102 transmits a small number of packets to node 101 through WAN interface(s) of node 101 that do(does) not belong to the second one or more WAN interfaces of node 101 ; the packets are used for managing purpose, such as transmitting health check packets, status information of the connections connected to the one or more WAN interfaces.
- node 101 only receives the small number of packets from node 102 through the WAN interfaces of node 101 not belonging to the second one or more WAN interfaces of node 101 .
- node 101 sends node 102 information of the second one or more WAN interfaces by using a WAN interface management message.
- the information includes the identities of WAN interface(s) of node 101 that is(are) preferred for node 102 for transmitting data packets to node 101 . Therefore node 102 can set the destination address of the encapsulating packet to be the address of the preferred WAN interface(s) of node 101 .
- the information is used as an instruction for node 102 .
- node 102 also determines by itself which WAN interface(s) of node 101 should be used to receive data packets from node 102 to node 101 .
- the usage of WAN interface management message includes: informing node 102 how to use a WAN interface of node 101 , informing node 102 when to transmit data packets through a particular WAN interface of node 101 .
- node 101 receives WAN interface management messages from node 102 for managing one or more WAN interfaces. Therefore, the usage of WAN interface management message also includes: informing node 101 how to use a WAN interface, informing node 101 when to transmit data packets through a particular WAN interface.
- node 101 when node 101 receives an encapsulating packet from node 102 through one of WAN interfaces 121 a , 121 b , 121 c and 121 d at step 611 , node 101 decapsulates the encapsulating packet to retrieve a data packet at step 612 . Then node 101 transmits the data packet to a host or node through one of its LAN interfaces 122 a and 122 b according to the destination address of the data packet at step 613 .
- node 101 selects the first one or more WAN interface that connects to a network with network latency less than 150 milliseconds at step 601 . This allows the encapsulating packets to reach node 102 quickly. Therefore, it is preferred not to choose a WAN interface that connects to a satellite network because the satellite round trip time is usually more than 150 milliseconds.
- the encapsulation and decapsulation performed by node 101 at step 603 and step 612 respectively particularly allows data to be transmitted to and received from node 102 through one or more tunnels.
- the one or more tunnels can be established via a pair of WAN interfaces, such as WAN interface 121 b and WAN interface 131 b.
- node 102 transmits encapsulating packets to node 101 through a satellite network, for example through tunnel 201 a .
- a satellite network for example through tunnel 201 a .
- the satellite round trip time may be larger than the round trip time of other networks, the bandwidth offered by a satellite network could be larger than that of other networks.
- processing unit 401 of node 102 determines which WAN interface of node 101 is used to transmit the encapsulating packets from node 101 to node 102 .
- node 102 may select WAN interface 121 b of node 101 .
- node 102 When node 102 has made the WAN interface selection decision, it sends a WAN interface management message to node 101 about the decision. Without the WAN interface management message, node 101 will not be able to recognize the need to use WAN interface 121 b to transmit encapsulating packets to node 102 .
- the WAN interface selection decision at step 601 and the WAN interface selection decision for node 102 to determine which WAN interface for transmitting and receiving encapsulating packets can be determined according to: network performance of network connecting to WAN interfaces, conditions, network coverage, geographical location, network usage price, etc.
- FIG. 6C is a flowchart illustrating various embodiments according to the present invention.
- Node 101 transmits data packets to and receives data packets from node 102 respectively.
- Node 101 and node 102 together perform as a proxy/gateway for the host or node that connects to one of the LAN interfaces 122 a and 122 b of node 101 .
- Those who are skilled in the arts would appreciate the techniques of updating source address, updating destination address, and restoring the original destination address are described below, in general, deployed by proxies or gateways.
- node 101 first selects one or more WAN interfaces, such as WAN interface 121 a , 121 b , 121 c and 121 d , at step 631 .
- node 101 modifies the destination address of the data packet received to be the address of one of WAN interfaces of node 102 at step 633 .
- the original destination address of the data packet is recorded and is sent to node 102 before the data packet is transmitted.
- the original destination address is used for node 102 to restore the destination address.
- node 101 transmits the data packet with the modified destination address through the one or more WAN interfaces selected at step 634 .
- node 102 restores the destination address to the original destination address and modifies the source address of the data packet to be the address of one of the WAN interfaces of node 102 , and then transmits the data packet to the original destination address.
- the purpose of restoring the original destination address is to allow the data packet be transmitted to the original destination address.
- the purpose of updating the source address of the data packet to be the address of one of the WAN interfaces of node 102 is to allow the host or node of the original destination address be able to respond to the transmitted data packet using the address of one of the WAN interfaces of node 102 , instead of using one of the WAN interfaces of node 101 .
- node 102 receives the response, it forwards the response to one of the WAN interfaces of node 101 by modifying the destination address of the IP packets holding the response to the address of the WAN interface of node 101 .
- node 101 modifies the source address of the IP packets to the address of the original destination address.
- the WAN interface of node 101 selected by node 101 for transmitting the data packets to node 102 and the WAN interface of node 101 selected by node 102 for receiving data packets from node 102 can be the same or different.
- node 101 selects WAN interface 121 b to transmit encapsulating packets and node 102 selects WAN interface 121 a to receive encapsulated response to the data packets encapsulated in the encapsulating packets.
- node 101 selects WAN interface 121 c to transmit data packets and node 102 selects WAN interface 131 c to transmit response to the data packets to node 101 .
- FIG. 7 illustrates a network configuration according to various embodiments of the present invention.
- Node 702 can perform as a gateway, VPN gateway, or proxy for node 101 such that, for example, node 101 , host 103 b , hosts and nodes in interconnected networks 117 and host 103 a can transmit and receive data packets through node 702 .
- node 101 is configured to use a first one or more WAN interface to send packets to node 702 and a second one or more WAN interface to receive packets from node 702 .
- node 101 uses WAN interfaces 121 b and 121 c to send data packets to node 702 and uses WAN interface 121 a receive data packets from node 702 .
- Node 101 can be configured by an administrator or a network operator, such as the operator of satellite network 150 and the operator of wireless network 180 . In one example, as node 101 is already configured which WAN interface(s) is (are) used, there is no need to perform step 601 as there is no WAN interface selection being performed.
- FIG. 8 illustrates the communication steps of an example that node 702 performs as a gateway for host 103 b to communicate with a host in interconnected networks 171 .
- FIG. 9 illustrates the structure of IP packet being transmitted at each communication step of FIG. 8 .
- host 103 b transmits IP packet 901 to a host in interconnected network 171
- host 103 b first transmits IP packet 901 to node 101 at step 801 .
- IP packet 901 could be for service request, data transfer or any other purposes.
- the structure of IP packet 901 is illustrated in FIG. 9 .
- the payload field is used to hold data.
- the header section has a destination address field and a source address field.
- the destination address field is the IP address of the host in interconnected networks 171 .
- the source address field is the IP address of host 103 b.
- node 101 After node 101 has received IP packet 901 at communication step 801 , node 101 encapsulates IP packet 901 in IP packet 902 . As node 101 has a plurality of WAN interfaces, node 101 can choose any of the WAN interfaces to transmit IP packet 902 . Alternatively, node 101 can select a WAN interface that has been determined or configured before the IP packet 901 has arrived. For example, node 101 selects a WAN interface based on: a configuration entered by an administrator of node 101 ; latency of the connection that a WAN interface connected to; a condition entered by an administrator of node 101 ; and instructions received from a remote server or node.
- node 101 selects WAN interface 121 b , then transmits IP packet 902 to wireless network 180 through a wireless modem connecting to WAN interface 121 b at communication step 802 .
- the payload field of IP packet 902 stores IP packet 901 .
- the header section of IP packet 902 has a destination address field and a source address field.
- the destination address field of IP packet 902 is the IP address of WAN interface 703 of node 702 .
- the source address field of IP packet 902 is IP address of WAN interface 121 b of node 101 .
- wireless network 180 routes IP packet 902 to router 704 at communication step 803 .
- Wireless network 180 may connect to router 704 through private interconnected networks or public interconnected networks. For example, if node 702 and wireless network 180 are operated by the same administrator, wireless network 180 can route IP packet 902 through a direct connection to router 704 . On the other hand, if node 702 and wireless network 180 are not operated by the same administrator, in general, IP packet 902 will be routed to router 704 through public interconnected networks.
- router 704 When router 704 receives IP packet 902 , it routes IP packet 902 to WAN interface 703 at communication step 804 . In one variant, router 704 performs as a gateway or a proxy with network address translation (NAT) capability and the destination address of IP packet 902 is then updated accordingly.
- NAT network address translation
- node 702 When node 702 receives IP packet 902 , it decapsulates IP packet 902 to retrieve IP packet 901 . After examining the destination address of IP packet 901 , node 702 performs network address translation (NAT) on the source address of IP packet 901 . The modified IP packet 901 is IP packet 903 . Therefore IP packet 903 is based on IP packet 901 . The source address of IP packet 903 is the IP address of WAN interface 703 . Therefore if the host in interconnected network 171 replies, WAN interface 703 can receive the reply. The destination address of IP packet 903 is the same as the destination address of IP packet 901 , which is the IP address of the host in interconnected networks 171 that host 103 b tries to send IP packet 901 to. Node 702 then transmits IP packet 903 to router 704 at communication step 805 .
- NAT network address translation
- router 704 When router 704 receives IP packet 903 , it routes IP packet 903 to the host in interconnected networks 171 at communication step 806 . In one variant, router 704 performs as a gateway or a proxy with NAT capability and the source address of IP packet 903 is then updated accordingly.
- IP packet 903 When the host in interconnected networks 171 receives IP packet 903 , the host may or may not reply. If the host replies, it should reply with at least one IP packet, for example IP packet 904 , at communication step 807 .
- the destination address and source address of IP packet 904 is the IP address of WAN interface 703 and the IP address of the host in interconnected networks 171 respectively.
- router 704 When router 704 receives IP packet 904 , it routes IP packet 904 to node 702 at communication step 808 . In one variant, router 704 performs as a gateway or a proxy with NAT capability and the destination address of IP packet 904 is then updated accordingly.
- node 702 When node 702 receives IP packet 904 , it performs NAT on IP packet 904 . Such that IP packet 904 has been modified and the destination address of the modified IP packet 904 becomes the IP address of host 103 b . Then node 702 encapsulates modified IP packet 904 in IP packet 905 and then transmits IP packet 905 to router 704 at communication step 809 .
- the payload field of IP packet 905 stores modified IP packet 904 . Therefore IP packet 905 is based on IP packet 904 .
- the header section of IP packet 905 has a destination address field and a source address field.
- the destination address field of IP packet 905 is the IP address of one of WAN interfaces of node 101 . For illustration purpose, node 702 selects WAN interface 121 a . Therefore, the destination address field IP packet 905 is IP address of WAN interface 121 a .
- the source address field of IP packet 905 is IP address of WAN interface 703 .
- node 702 can select one of the WAN interfaces of node 101 for transmitting IP packet 905 to node 101 .
- node 702 can select a WAN interface that has been determined or configured before the IP packet 904 has arrived. For example, node 702 selects a WAN interface of node 101 based on: a configuration entered by an administrator of node 702 ; latency of the connection that a WAN interface connects to; a condition entered by an administrator of node 702 ; and instructions received from a remote server or node.
- router 704 When router 704 receives IP packet 905 , it routes IP packet 905 to satellite network 150 through satellite modem 160 at communication step 810 . In one variant, router 704 performs as a gateway or a proxy with NAT capability and the destination address of IP packet 905 is then updated accordingly. In communication step 811 , satellite modem 160 transmits IP packet 905 to satellite network 150 , which then transmits IP packet 905 to satellite modem 161 . Satellite modem 161 forwards IP packet 905 to node 101 in communication step 812 through connection medium 113 .
- node 101 When node 101 receives IP packet 905 , it decapsulates IP packet 905 to retrieve modified IP packet 904 , which is the same as IP packet 906 and then transmits IP packet 906 to host 103 b according to the destination address in the header of IP packet 906 in communication step 813 .
- the communication steps shown in FIG. 8 illustrates that different WAN interfaces of node 101 are used for transmitting and receiving IP packets, even if the IP packets belong to the same session.
- router 704 may need to use a state database to record the translation and the state database is in general stored in a storage unit of router 704 .
- node 702 when node 702 receives IP packet 904 after step 808 , it does not performs NAT on IP packet 904 . Therefore IP packet 904 is not modified and IP packet 905 encapsulates IP packet 904 , instead of modified IP packet 904 . The destination of address of IP packet 904 is changed by node 101 to IP packet 906 after step 812 . This allows NAT being performed by node 101 , instead of by node 702 . This may reduce the computing resources requirement of node 702 .
- node 101 sends a WAN interface management message to inform node 702 which WAN interface of node 101 should node 702 transmit IP packets to.
- WAN interface management message comprises a WAN interface identity field and an instruction field.
- WAN interfaces 121 a and 121 d can be used by node 101 to receive IP packets from node 702 while WAN interfaces 121 b and 121 c cannot be used by node 101 to receive IP packets from node 702 .
- the WAN interface management message comprises the identities of WAN interfaces 121 a and 121 d in the WAN interface identity field and the instruction field, which holds the information that WAN interfaces 121 a and 121 d can be used by node 101 to receive IP packets from node 702 while WAN interfaces 121 b and 121 c cannot be used by node 101 to receive IP packets from node 702 .
- the WAN interface management messages may be sent anytime before IP packet 905 is transmitted by node 702 in step 809 .
- identity(identities) of WAN interface(s) of node 101 that is(are) used for node 101 to transmit IP packets to node 702 is(are) stored in the WAN interface field and the instruction that this (these) WAN interface(s) is(are) used for transmission only is stored in the instruction field.
- a WAN interface management message comprises an indicator in the instruction field to indicate whether a WAN is for both transmission and receiving, transmission only or receiving only.
- the indicator can represented by a bit, a plurality of bits, a byte, a plurality of bytes, a string, a plurality of strings, XML messages, etc.
- WAN interface management messages is determined and transmitted by one of node 101 or node 702 . Therefore, the decision of which WAN interface of node 101 to use for transmitting and/or receiving IP packets is determined by node 101 or node 702 . For example, an administrator of node 101 can determine which WAN interface(s) of node 101 and node 702 is(are) used for transmitting and/or receiving IP packets. Therefore, a WAN interface can be set as unidirectional transmission, unidirectional receiving or bidirectional.
- a WAN interface management message comprises status information of a WAN interface.
- the status information can be part of information stored in the instruction field.
- the status is stored in a status field of WAN interface management message. For example, when node 101 updates node 102 about the status of a WAN interface, node 101 sends a WAN interface management message with the WAN interface status in the status field to node 102 .
- the WAN interface identity field holds the identity of the corresponding WAN interface.
- the instruction field in this case may be empty as the WAN interface management message is for status reporting purpose.
- one or more tunnels are established between two WAN interfaces of two nodes and when a WAN interface is selected for transmission and receiving, transmission only or receiving only, the corresponding one or more tunnels that are established through the selected WAN interface will be affected by WAN interface management message.
- WAN interface management message For example, when WAN interface 121 a can only be used for receiving IP packets from node 702 and cannot be used for transmitting IP packets to node 702 , the one or more tunnels that established through WAN interface 121 a to WAN interface 705 can also only be used for receiving IP packets from node 702 and cannot be used for transmitting IP packets to node 702 .
- an administrator of a node determines WAN interface pair stating a WAN interface of the node and a WAN interface of another node, such that the WAN interfaces stated in the WAN interface pair are used for transmitting and receiving, transmitting only or receiving only IP packet between the two nodes.
- the administrator determines a WAN interface pair, WAN interface 121 a and WAN interface 703 , which is used for receiving IP packets from WAN interface 703 only.
- the administrator determines that a WAN interface pair, WAN interface 121 d and WAN interface 703 , is bidirectional, such that WAN interface 121 and WAN interface 703 can be used for transmitting and receiving IP packets to and from each other.
- a node such as node 102 intermediates access by a host, such as host 103 a or 103 b , to interconnected networks 172 .
- node 102 terminates incoming access requests and connections at the application layer of the Open System Interconnection (OSI) reference model or of the TCP/IP model.
- OSI Open System Interconnection
- node 102 operates as an application-layer proxy to protect resources in interconnected networks 172 from direct exposure to hosts connected to node 101 .
- Node 102 receives incoming access requests encapsulated in a packet, decapsulates the access requests to reach the underlying application data, and sends the application data comprising the access requests to interconnected networks 172 .
- node 102 allows direct connections between layers of the OSI reference model or of the TCP/IP model.
- node 102 exchanges data using a secure channel negotiated with the requesting host, such as host 103 a or 103 b .
- Node 102 receives a secure request via the one of tunnels 201 a , 201 b , 201 c or 201 d , and makes requests to interconnected networks 172 on behalf of the requesting host, i.e. host 103 a or 103 b , to establish a data connection between the requesting host and interconnected networks 172 .
- FIG. 10 illustrates a webpage used to configure node 101 .
- Webpage 1001 can be shown at a display of a laptop computer, desktop computer, handheld computing device, mobile phone or any device capable of displaying webpage 1001 .
- Webpage 1001 has two parts: VPN Profile 1011 and WAN Connection Priority 1012 .
- An administrator can enter the information required to setup a VPN using VPN Profile 1011 and configure priority, transmission and receiving of data of WAN interfaces in WAN Connection Priority 1012 .
- the information is used to create one or more tunnels belonging to the VPN.
- the information is used to create multiple tunnels and the multiple tunnels are aggregated to form one aggregated tunnel for the VPN.
- the information required includes security information, identity information and encryption information.
- the information required is retrieved from a remote management server, a secured device coupled to node 101 or a preconfigured database. This releases the administrator the burden to enter required information manually through the web page.
- WAN Connection Priority 1012 has three main sections: WAN interface identity 1021 , priority selection 1022 and direction selections 1031 to 1033 .
- WAN interface identity 1021 has four WAN interfaces shown.
- “WAN 1”, “WAN 2”, “WAN 3” and “WAN 4” in WAN interface identity 1021 corresponds to WAN interfaces 121 a , 121 b , 121 c and 121 d respectively.
- Priority selection 1022 is used to configure priority to WAN interfaces.
- the priority of “WAN 1” is configured to be “OFF” that processing unit 301 will not use WAN interface 121 a to transmit or receive data traffic.
- processing unit 301 uses WAN interfaces 121 b , 121 c and 121 d with the same priority.
- Direction selection 1031 is configured to be “Up Only”, such that processing unit 301 only uses WAN interface 121 b to transmit data.
- Direction selection 1032 is configured to be “Down Only”, such that processing unit 301 only uses WAN interface 121 c to receive data.
- Direction selection 1033 is configured to be “Up/Down Only”, such that processing unit 301 uses WAN interface 121 d to both transmit and receive data.
- the information used to configure WAN Connection Priority 1012 is retrieved from a remote management server, a secured device coupled to node 101 or a preconfigured database. This releases the administrator the burden to enter required information manually through the web page.
- the web page for configuring node 102 is similar and the number of WAN interfaces in WAN interface identity 1021 will be three as node 102 has three WAN interfaces.
- FIG. 11 illustrates a webpage used to configure node 101 .
- the design of webpage 1101 is similar to the design of webpage 1001 .
- Webpage 1101 has two parts: VPN Profile 1111 and Tunnel Priority 1112 .
- An administrator can enter the information required to setup a VPN using VPN Profile 1111 and configure priority, transmission and receiving of data of tunnels in Tunnel Priority 1112 .
- VPN Profile 1111 can be identical to VPN Profile 1011 if the information required to setup VPN is the same.
- Tunnel Priority 1112 has three main sections: tunnel identity 1121 , priority selection 1122 and direction selections 1131 to 1133 .
- tunnel identity 1121 has four tunnels shown.
- “Tunnel A”, “Tunnel B”, “Tunnel C” and “Tunnel D” in tunnel identity 1121 corresponds to tunnels 201 a , 201 b , 201 c and 201 d respectively.
- Priority selection 1122 is used to configure priority to tunnel.
- the priority of “Tunnel A” is configured to be “OFF” that node 101 will not use tunnel 201 a to transmit or receive data traffic.
- processing unit 301 uses tunnels 201 b , 201 c and 201 d with the same priority.
- Direction selection 1131 is configured to be “Up Only”, such that node 101 only uses tunnel 201 b to transmit data.
- Direction selection 1132 is configured to be “Down Only”, such that node 101 only uses tunnel 201 c to receive data.
- Direction selection 1133 is configured to be “Up/Down Only”, such that node 101 uses tunnel 201 d to transmit and receive data.
- the information used to configure Tunnel Priority 1112 is retrieved from a remote management server, a secured device coupled to node 101 or a preconfigured database. This releases the administrator the burden to enter required information manually through the web page.
- the web page for configuring node 102 is similar and the number tunnels in tunnel identity 1121 will be the same as the number of tunnels established between node 101 and 102 are also four.
- tunnel identity 1121 instead of shown tunnel identities in tunnel identity 1121 , WAN interfaces of node 101 and node 102 are shown in tunnel identity 1121 . As a tunnel can be established between one or more WAN interfaces of node 101 and one or more WAN interfaces of node 102 , a tunnel identity can be replaced with the WAN interfaces used for establishing the tunnel.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Mobile Radio Communication Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
- The present application is a Continuation application which claims the benefits of and is based on U.S. application Ser. No. 14/396,747 filed on Oct. 24, 2014, which is a National Stage Application and further claims the benefits of and is based on International Application No. PCT/M2014/059378 filed on Mar. 3, 2014, the disclosures of which are hereby incorporated by specific reference thereto.
- The present invention relates in general to the field of computer networks. More particularly, the present invention relates to a method carried out by a first node for transmitting data to a second node mainly through a first tunnel group and receiving data from the second node mainly through a second tunnel group.
- Internet service providers can provide satellite Internet service to users through geostationary satellites which may offer high data throughput. In addition to the high throughput, the coverage area of satellite networks is also very big compared to other wireless networks that may be available in recent times. Satellite networks often provide coverage in many rural areas where other wireless networks may not be available. However, satellite communications have higher latencies compared to other wireless communications due the data having to travel long distances to the geostationary satellite. Although satellite communications provide very high performance for downlink data, performance of communications through satellite may not be satisfactory for uplink data, especially if uplink is done from many devices at once, i.e. from customers of the Internet service providers.
- The paper titled “Networking using Direct Broadcast Satellite” (Venkata Padmanabhan, Hari Balakrishnan, Keith Sklower, Elan Amir, and Randy H. Katz. Networking using Direct Broadcast Satellite. In Proc. of the 1st international Workshop on Satellite-based Services, Rye, N.Y., November 1996. University of California at Berkeley) presents a networking architecture where a geostationary satellite broadcasts directly to user premises, while outgoing traffic from the user premises to the Internet are sent over an SLIP/PPP line. This paper explains in detail an asymmetric routing technology. There are two options for sending packets from the users to the Internet. In the first option, packets are encapsulated and sent over the SLIP line using DBS source address. In the second option, home agent-based routing is used.
- Users may enjoy high downlink bandwidth when accessing the Internet through a satellite network. However, for transmitting packets to a host or node accessible through the internet, using wireless networks other than a satellite network may provide lower latency and may be cost effective. For a data session using a connection-oriented protocol, techniques are required for automatically enabling a user to use a satellite network for downlink and another wireless or wired network for uplink, while taking into account the asymmetric bandwidth and asymmetric latency.
- The embodiments of the present invention describe, in general, a first node and a second node communicating with each other through one or more tunnels.
- According to one of the embodiments of the present invention, the first node transmits data to the second node mainly through a first tunnel group, and receives data from the second node mainly through a second tunnel group. There is at least one tunnel in the first tunnel group and at least one tunnel in the second tunnel group. The at least one tunnel in the first tunnel group and the at least one tunnel in the second tunnel group are formed using at least two network interfaces of the first node.
- In a preferred embodiment, the at least one tunnel in the first tunnel group is established through at least one network which, in general, has a network latency less than 150 milliseconds. The at least one tunnel in the second tunnel group is established through at least one satellite connection.
- According to one of the embodiments, when network performance of the first tunnel group is below a first threshold, the first node transmits data through the second tunnel group. When network performance of the second tunnel group is below a second threshold, the first node receives data through the first tunnel group.
- According to one of the embodiments of the present invention, the tunnels in the first tunnel group and the tunnels in the second tunnel group are aggregated together to form one aggregated tunnel.
- According to one of the embodiments of the present invention, the first node transmits or receives one or more tunnel management message is transmitted to or received from the second node. The tunnel management message may be sent from one node to another node for informing the another node which tunnels should be used for transmitting data and which tunnels should be use for receiving data.
- According to one of the embodiments of the present invention, when the first node receives first IP packets from a host through at least one of its Local Area Network (LAN) interface, the first node encapsulates the first IP packets in first encapsulating IP packets. The first node then transmits the first encapsulating IP packets mainly through a first one or more Wide Area Network (WAN) interfaces to a second node. The first node receives second encapsulating IP packets mainly through a second one or more WAN interfaces from the second node, and then decapsulates the second encapsulating IP packets to retrieve second IP packets. The second IP packets are transmitted through at least one corresponding LAN interface. The first one or more WAN interfaces are connected to a wireless network and the second one or more WAN interfaces are connected to a satellite connection. The first node further sends the second node information of the first one or more WAN interfaces and the second one or more WAN interfaces.
- The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.
-
FIG. 1 illustrates a network configuration according to various embodiments of the present invention. -
FIG. 2 illustrates tunnels established between a first node and a second node according to various embodiments of the present invention. -
FIG. 3 is an illustrative block diagram of a computer system or network node according to one of the embodiments of the present invention. -
FIG. 4 is an illustrative block diagram of a computer system or network node according to one of the embodiments of the present invention. -
FIG. 5 is an illustrative block diagram of a computer system or network node according to one of the embodiments of the present invention. -
FIG. 6A is a flowchart illustrating a process for a first node to transmit and receive data to and from a second node according to one of the embodiments of the present invention. -
FIG. 6B is a flowchart illustrating a process for a first node to transmit and receive data to and from a second node according to one of the embodiments of the present invention. -
FIG. 6C is a flowchart illustrating a process for a first node to transmit and receive data to and from a second node according to one of the embodiments of the present invention. -
FIG. 7 illustrates a network configuration according to various embodiments of the present invention. -
FIG. 8 illustrates the communication steps of a node communicating with a host according to one of the embodiments of the present invention. -
FIG. 9 illustrates the structure of IP packet being transmitted at each communication step according to one of the embodiments of the present invention. -
FIG. 10 illustrates a webpage used to configure a first node according to one of the embodiments of the present invention. -
FIG. 11 illustrates a webpage used to configure a first node according to one of the embodiments of the present invention. - The ensuing description provides preferred exemplary embodiment(s) only, and is not intended to limit the scope, applicability or configuration of the invention. Rather, the ensuing description of the preferred exemplary embodiment(s) will provide those skilled in the art with an enabling description for implementing a preferred exemplary embodiment of the invention. It being understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the invention as set forth in the appended claims.
- Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, circuits may be shown in block diagrams in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.
- Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
- Embodiments, or portions thereof, may be embodied in program instructions operable upon a processing unit for performing functions and operations as described herein. The program instructions making up the various embodiments may be stored in a computer readable storage medium.
- Moreover, as disclosed herein, the term “computer readable storage medium”, “main memory”, or “secondary storage” may represent one or more devices for storing data, including read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), random access memory (RAM), magnetic RAM, core memory, floppy disk, flexible disk, hard disk, magnetic tape, CD-ROM, flash memory devices, a memory card and/or other machine readable mediums for storing information. The term “computer readable storage media” may also include, but is not limited to portable or fixed storage devices, optical storage mediums, magnetic mediums, memory chips or cartridges, wireless channels and various other mediums capable of storing, containing or carrying instruction(s) and/or data. A computer readable storage medium can be realized by virtualization, and can be a virtual computer readable storage medium including a virtual computer readable storage medium in a cloud-based instance.
- The term “computer readable storage media”, “main memory”, or “secondary storage”, as used herein refers to any medium that participates in providing instructions to a processing unit for execution. The main memory or secondary storage is just one example of a machine-readable medium, which may carry instructions for implementing any of the methods and/or techniques described herein. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks. Volatile media includes dynamic memory. Transmission media includes coaxial cables, copper wire and fiber optics. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
- A volatile storage may be used for storing temporary variables or other intermediate information during execution of instructions by the processing unit. A non-volatile storage or static storage may be used for storing static information and instructions for the processing unit, as well as various system configuration parameters.
- The computer readable storage medium may include a number of software modules that may be implemented as software code to be executed by the processing unit using any suitable computer instruction type. The software code may be stored as a series of instructions or commands, or as a program in the computer readable storage medium.
- Various forms of computer readable storage media may be involved in carrying one or more sequences of one or more instructions to the processing unit for execution. For example, the instructions may initially be carried on a magnetic disk from a remote computer. Alternatively, a remote computer can load the instructions into its dynamic memory and send the instructions to the system that runs the one or more sequences of one or more instructions.
- A processing unit may be a microprocessor, a microcontroller, a digital signal processor (DSP), any combination of those devices, or any other circuitry configured to process information.
- A processing unit executes program instructions or code segments for implementing embodiments of the present invention. Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program instructions to perform the necessary tasks may be stored in a computer readable storage medium. A processing unit(s) can be realized by virtualization, and can be a virtual processing unit(s) including a virtual processing unit in a cloud-based instance.
- Embodiments of the present invention are related to the use of a computer system for implementing the techniques described herein. In an embodiment, the inventive processing units may reside on a machine such as a computer platform. According to one embodiment of the invention, the techniques described herein are performed by computer system in response to the processing unit executing one or more sequences of one or more instructions contained in the volatile memory. Such instructions may be read into the volatile memory from another computer readable storage medium. Execution of the sequences of instructions contained in the volatile memory causes the processing unit to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
- A code segment may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
- A network interface that may be provided by a node is an Ethernet interface, a frame relay interface, a fibre optic interface, a cable interface, a DSL interface, a token ring interface, a serial bus interface, an universal serial bus (USB) interface, Firewire interface, Peripheral Component Interconnect (PCI) interface, etc.
- A network interface may be implemented by an standalone electronic component or may be integrated with other electronic components. A network interface may have no network connection or at least one network connection depending on the configuration. A network interface may be an Ethernet interface, a frame relay interface, a fibre optic interface, a cable interface, a Digital Subscriber Line (DSL) interface, a token ring interface, a serial bus interface, an universal serial bus (USB) interface, Firewire interface, Peripheral Component Interconnect (PCI) interface, etc.
- A presently preferred embodiment of the present invention may utilize a gateway. A gateway is a device which performs protocol conversion between different types of networks or applications. The term gateway is not meant to be limited to a single type of device, as any device, hardware or software, that may act as a bridge between the user and the networks may be considered a gateway for purposes of this application. The gateway may couple with a plurality of multiple networks. A router, an access point or a wireless access point may all be considered a gateway for purposes of this invention.
- Embodiments, or portions thereof, may be embodied in a computer data signal, which may be in any suitable form for communication over a transmission medium such that it is readable for execution by a functional device (e.g., processing unit) for performing the operations described herein. The computer data signal may include any binary digital electronic signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic media, radio frequency (RF) links, and the like, and thus the data signal may be in the form of an electrical signal, optical signal, radio frequency or other wireless communication signal, etc. The code segments may, in certain embodiments, be downloaded via computer networks such as the Internet, an intranet, LAN, MAN, WAN, the PSTN, a satellite communication system, a cable transmission system, and/or the like.
-
FIG. 3 is an illustrative block diagram of a computer system or network node, such asnode 101, according to one of the embodiments of the present invention.Node 101 comprises processingunit 301,main memory 302,system bus 303,secondary storage 304, local area network (LAN) interfaces 122 a and 122 b, and wide area network (WAN) interfaces 121 a, 121 b, and 121 c.Secondary storage 304 andmain memory 302 are computer readable storage media.Processing unit 301 andmain memory 302 are connected to each other directly.System bus 303 connects processingunit 301 directly or indirectly tosecondary storage 304, LAN interfaces 122 a and 122 b, andWAN interfaces system bus 303 allowsnode 101 to have increased modularity.System bus 303couples processing unit 301 tosecondary storage 304, andLAN interfaces WAN interfaces System bus 303 can be any of several types of bus structures including a memory bus, a peripheral bus, and a local bus using any of a variety of bus architectures.Secondary storage 304 stores program instructions for execution by processingunit 301. The scope of the invention is not limited tonode 101 having five network interfaces, such thatnode 101 may have more network interfaces. LAN interfaces 122 a and 122 b, andWAN interfaces -
FIG. 4 is illustrative block diagram of a computer system or network node, such asnode 102, according to one of the embodiments of the present invention.Node 102 comprises processingunit 401,main memory 402,system bus 403,secondary storage 404,LAN interface 132, WAN interfaces 131 a, 131 b, and 131 c. The components ofnode 102 are connected to each other in a similar way as the components ofnode 101. -
FIG. 5 is illustrative block diagram of a computer system or network node, such asnode 702, according to one of the embodiments of the present invention.Node 702 comprises processingunit 501,main memory 502,system bus 503,secondary storage 504, andWAN interface 703. The components ofnode 702 are connected to each other in a similar way as the components ofnode 101. Alternatively,node 702 may also be represented by the block diagram inFIG. 3 orFIG. 4 , such thatnode 702 may have one or more WAN interfaces, and one or more LAN interfaces. -
FIG. 1 illustrates a network configuration according to various embodiments of the present invention.Node 101 is connected tonode 102 through four tunnels:tunnels FIG. 2 . LAN interface 122 a connectsnode 101 to host 103 a throughinterconnected networks 117 usingconnections LAN interface 122 b connectsnode 101 to host 103b using connection 115 b. - For illustration purpose,
node 101 connects to a satellite network throughWAN interface 121 a for establishingtunnel 201 a withnode 102.WAN interface 121 a communicates withsatellite network 150 by usingsatellite modem 161;node 102 connects tosatellite network 150 throughWAN interface 131 a andsatellite modem 160 for establishingtunnel 201 a withnode 101. - The connection between
WAN interface 121 a andsatellite network 150 and the connection betweenWAN interface 131 a andsatellite network 150 are usingsatellite modems Satellite modems satellite network 150 throughconnections satellite modems wireless connection medium connection medium Satellite modem 160 can also be included within a housing that containsnode 102, andsatellite modem 161 can be included within a housing that containsnode 101. Alternatively,node 102 can include an auxiliary port into whichsatellite modem 160 is plugged.Satellite modems satellite network 150.Satellite modems - Also for illustration purpose,
node 101 establishestunnel 201 b withnode 102 throughWAN interface 121 b,wireless network 180 andWAN interface 131 b.WAN interface 121 b communicates withwireless network 180 through awireless communication channel 110 b. SimilarlyWAN interface 131 b communicates withwireless network 180 throughwireless communication channel 112 b. Communication throughwireless communication channel wireless network 180 is not limited. For example,wireless network 180 may be a cellular network operated by Verizon, ATT, China Mobile, or Vodafone. A cellular network can also be a mobile virtual network operator (MVNO) or mobile other licensed operator (MOLO) that provides a wireless communications services but does not own the wireless or cellular network infrastructure over which the MVNO provides services to its customers. - Also for illustration purpose,
node 101 establishes tunnel 201 c withnode 102 throughWAN interface 121 c,wireless network 181,interconnected network 171 andWAN interface 131 c.WAN interface 121 c communicates withwireless network 181 throughwireless communication channel 110 c.Wireless network 181 communicates withWAN interface 131 c throughinterconnected networks 171. The size ofwireless network 181 is also not limited. For example,wireless network 181 may be a cellular network operated by Verizon, AT&T, China Mobile, or Vodafone.Wireless network 181 andinterconnected networks 171 communicate with each other throughconnection 111 a.Node 102 communicates withinterconnected networks 171 throughconnection 112 c. - Also for illustration purpose,
node 101 establishestunnel 201 d withnode 102 throughWAN interface 121 d,interconnected networks 171 andWAN interface 131 c.WAN interface 121 d communicates withinterconnected networks 171 throughconnection 111 b.Node 102 usesWAN interface 131 c to communicate withinterconnected networks 171 throughconnection 112 c. -
Wireless networks - Uplink Packet Access (HSUPA), Code Division Multiple Access (CDMA), Evolution-Data Optimized (EVDO), Enhanced Data Rates for GSM Evolution (EDGE), General Packet Radio Service (GPRS) Third Generation Partnership Program Long Term Evolution (3GPP LTE) or any other digital data transfer protocol for wireless data transfer over radio, microwave or frequency bands used in wireless networks. In one variant,
wireless networks - WAN interfaces 121 b, 121 c and 131 b can be realized by using a connection through cellular phones, cellular modems, and/or wireless communication modems. A cellular modem, such as a 3GPP LTE modem, is capable of communicating with one or more cellular networks. A cellular modem can be an embedded cellular modem or an external cellular modem.
-
Connections -
Interconnected networks interconnected networks tunnels interconnected networks 117 is a LAN andinterconnected networks 172 is the Internet, such thatnode 102 performs as a gateway forinterconnected networks 117 to communicate with the Internet through at least of the tunnel oftunnels - A tunnel, such as
tunnels Layer 2 Tunneling Protocol (L2TP) or Point-to-Point Tunneling Protocol (PPTP). One or more tunnels can be established between two network interfaces, such asWAN interface tunnels node 101 andnode 102. For example,WAN interface 121 b may also establish another tunnel withWAN interface 131 c ifwireless networks 180 can communicate withinterconnected networks 171. - Data can be in stored in payload of data packets.
Node 101 encapsulates data packets in encapsulating packets, and then transmit encapsulating packets tonode 102 through one or more tunnels. Whennode 102 receives the encapsulating packets,node 102 decapsulates the encapsulating packets to retrieve the data packets. Encapsulating packets may be distributed among the plurality of tunnels, i.e.tunnels - Similarly, when
node 102 needs to transmit data packets tonode 101,node 102 first encapsulates data packets in encapsulating packets, and then transmits encapsulating packets tonode 101 through the one or more tunnels. Whennode 101 receives the encapsulating packets,node 101 decapsulates the encapsulating packets to retrieve the data packets. - Those who are skilled in the arts would appreciate that a tunnel enables the encapsulation of data from one type of protocol within the datagram of the same or different protocol. Those who are skilled in the arts would also appreciate that tunnels can be use to implement virtual private networks (VPN). In one variant, the tunnels between
node tunnels tunnels WAN interfaces - Those who are skilled in the arts would appreciate that there are many methods to aggregate a plurality of tunnels to form one aggregated tunnel or connection. One of the methods is disclosed in the U.S. patent application Ser. No. 12/646,774, Filed Dec. 23, 2009, entitled “THROUGHPUT OPTIMIZATION FOR BONDED VARIABLE BANDWIDTH CONNECTIONS”.
- According to one of the embodiments of the present invention, when a first node forms at least two tunnel groups with a second node, one of the tunnel groups is mainly used to transmit data packets from the first node to the second node and another tunnel group is mainly used to receive data packets from the second node. In each tunnel group, there can be one or more tunnels. As there are at least two tunnel groups and each tunnel group has at least one tunnel, there are at least two tunnels established between the first node and the second node. The at least two tunnels are established through two of network interfaces of the first node and at least one network interface of the second node. In one variant, the second node may perform as a gateway for the first node. In one variant, when a tunnel group is mainly used to transmit data packets from the first node to the second node, there is no data packet being transmitted from the second node to the first node through the tunnel group. In one variant, when a tunnel group is mainly used to transmit data packets from the first node to the second node, there is a small number of packets being transmitted from the second node to the first node through the tunnel group as those packets are used for managing the tunnel group, such as transmitting health check packets, establishing one of the tunnels in the tunnel group and sending status information. In one variant, when a tunnel group is mainly used to receive data packets by the first node from the second node, there is no data packet being transmitted by the first node to the second node through the tunnel group. In one variant, when a tunnel group is mainly used to receive data packets by the first node from the second node, there is a small number of packets being transmitted from the first node to the second node through the tunnel group as those packets are used for managing the tunnel group, such as transmitting health check packets, establishing one of the tunnels in the tunnel group and sending status information.
- For illustration purpose, the first node is
node 101 and the second node isnode 102; a first tunnel group comprisestunnels 201 b and 201 c; and a second tunnel group comprisestunnels node 101 tonode 102 while the second tunnel group is used to transmit data packets fromnode 102 tonode 101 Thereforenode 102 may perform as a gateway fornode 101 and the networks connected to thenode 101, such asnetwork 117, host 103 a and hosts 103 b, to communicate withinterconnected network 172. Furthermore, as the number of tunnels in the first tunnel group can be more than one,node 101 may use a plurality of tunnels in the first tunnel group to transmit data packets tonode 102. Whennode 102 receives the data packets through one or more tunnels of the first tunnel group,node 102 reorders the packets before transmitting the packets tointerconnected network 172 if necessary. Similarly, in another example, as the number of tunnels in the second tunnel group can be more than one,node 102 may use a plurality of tunnels in the second tunnel group to transmit data packets tonode 101. When a host ininterconnected networks 172 sends a packet to host 103 a,node 102 sends the packet through one or more tunnels of the second tunnel group tonode 101. Thennode 101 sends the packet tonode 103 a throughinterconnected networks 117. - Similarly, as the number of tunnels in the second tunnel group can be more than one,
node 102 may use a plurality of tunnels in the second tunnel group to transmit data packets. In one variant, the data packets belong to one session andnode 101 reorders the packets before transmitting the packets tointerconnected network 117 or host 103 b. Those skilled in the art would appreciate that there are myriad ways to use multiple network interfaces to transmit data packets belonging to the same session. - According to one of the embodiments of the present invention, when a tunnel is used to transmit data packets from
node 101 tonode 102, the same tunnel is not used to transmit data packets fromnode 102 tonode 101. For example, whentunnel 201 b is used to transmit data packets fromnode 101 tonode 102,tunnel 201 b is not used to transmit data packets fromnode 102 tonode 101. In order fornode 102 to transmit data packets to 101, one or more of other tunnels, such astunnel 201 a, tunnel 201 c or/and 201 d, have to be used. A tunnel management message is sent bynode 101 to instructnode 102 not to usetunnel 201 b for transmitting data packets tonode 101. In one variant,node 102 can overrule the instruction sent bynode 101. The ability of overruling the instruction allows administrators ofnode node 102 can still transmit a portion of data packets tonode 101 throughtunnel 201 b and the portion can be preconfigured by the administrator ofnode 102, administrator ofnode 102 and/or negotiated betweennode 101 andnode 102. - In one variant, when a tunnel is used to transmit data packets from
node 101 tonode 102, the corresponding WAN interface establishing the tunnel atnode 101 is not used to receive data packets fromnode 102. For example, whentunnel 201 b is used to transmit data packets fromnode 101 tonode 102,WAN interface 121 b is not used to receive data packets fromnode 102. In one variant, when a tunnel is used to receive data packets fromnode 102 tonode 101, the corresponding WAN interface establishing the tunnel atnode 101 is not used to transmit data packets tonode 102. For example, whentunnel 201 a is used to receive data packets fromnode 102 tonode 101,WAN interface 121 a is not used to transmit data packets tonode 101. There could be many reason why to have a WAN interface for transmitting or receiving data packets only, including the WAN interface is half-duplex, the network performance of the connection connected to the WAN interface is asymmetric, the cost of transmitting or receiving data packets through the connection connected to the WAN interface is asymmetric, and etc. - According to one of the embodiments of the present invention,
node 101 transmits tunnel management messages tonode 102 for managing one or more tunnels. The usage of tunnel management message includes: informingnode 102 that which group a tunnel belongs to, informingnode 102 how to use a tunnel, informingnode 102 when to transmit data packets through a particular tunnel or tunnel group. In one variant,node 101 receives tunnel management messages fromnode 102 for managing one or more tunnels. Therefore, the usage of tunnel management message also includes: informingnode 101 that which group a tunnel belongs to, informingnode 101 how to use a tunnel, informingnode 101 when to transmit data packets through a particular tunnel or tunnel group. - A tunnel management message comprises a tunnel identity field and an instruction field. For example, for illustration purpose only, a first tunnel and a third tunnel belong to the first tunnel group while a second tunnel belongs the second tunnel group. In such case, the tunnel identity field comprises the identities of the first tunnel, the second tunnel, and the third tunnel and the instruction field comprises information that the first tunnel and the third tunnel belong to the first tunnel group while a second tunnel belongs the second tunnel group. Therefore, after receiving the tunnel management message,
node 102 can expect data packets to arrive through the first and the third tunnels and does not use the first and the third tunnels to transmit data packets tonode 101 while only transmits data packets tonode 101 through the second tunnel. Tunnel management messages may be sent when tunnels are about to be established, are being established or after tunnels have been established. In another example, a tunnel management message comprises information that assistsnode 102 to identify the tunnels and the corresponding tunnel groups in the instruction field and the identities of the corresponding tunnels in the tunnel identity field. There is no limitation that the tunnel message can only be sent bynode 101, such thatnode 102 can also send the tunnel message. For example, a tunnel message may comprise information thattunnels tunnels 201 c and 201 d belongs to a second tunnel group. In one variant, the tunnel management messages also comprises an indicator to indicate whether a tunnel group is for both transmission and receiving, transmission only or receiving only. For example, an indicator indicates that a first tunnel group is for transmission only and a second tunnel group is for both transmission and receiving. - An indicator can be represented by a bit, a plurality of bits, a byte, a plurality of bytes, a string, a plurality of strings, XML messages, etc. Those who are skilled in the arts would appreciate that there are myriads ways to represent the indicator.
- In one variant, one or more tunnels of the first tunnel group and one or more tunnels of the second tunnel group are aggregated to form one aggregated tunnel. When
node 101 transmits data packets through the aggregated tunnel tonode 102,node 101 uses the first tunnel group to transmit data packets. Astunnels node 101 transmits data packets throughtunnels node 102. Whennode 102 transmits data packets through the aggregated tunnel tonode 101,node 102 uses the second tunnel group to transmit data packets as instructed by the tunnel management messages. Astunnels 201 c and 201 d belong to the second tunnel group,node 102 transmits data packets throughtunnels 201 c and 201 d tonode 101. The tunnel management message can also be used to manage the aggregated tunnel. For example, when a tunnel is added to the first tunnel group,node 101 sends a tunnel management message tonode 102 that there is one more tunnel in the aggregated tunnel. - According to one of the embodiments of the present invention, a tunnel management message comprises an indicator in the instruction field to indicate whether a tunnel is for both transmission and receiving, transmission only or receiving only. For example, for illustration purpose,
tunnel 201 a is used fornode 101 to receive data packets fromnode 102 only;tunnels node 101 to transmit data packets tonode 102 only. Therefore the indicator indicates thattunnel 201 a is for receiving data packets only andtunnels Node 102 then transmits data packets tonode 101 throughtunnel 201 a and receives data fromnode 102 throughtunnels - In one variant, a tunnel management message comprise status information of a tunnel. The status information can be part of information that is stored in the instruction field. In one variant, a tunnel management message also comprises a status field. The status field is used to hold information of status of the tunnel. For example, when
node 101updates node 102 about the status of a tunnel,node 101 sends a tunnel management message with the tunnel status in the status field tonode 102. The tunnel identity field holds the identity of the corresponding tunnel. The instruction field in this case may be empty as the tunnel management message is for status reporting purpose. - In one variant, a tunnel can be used by
node 101 to transmit and receive data packets to and fromnode 102. Therefore the indicator in the tunnel management messages shows that the tunnel can be used for both transmission and receiving. In one variant, for illustration purpose, a tunnel is preferably used bynode 101 to transmit data packets tonode 102 and is not preferably used bynode 101 to receive data packets fromnode 102 even though the tunnel is capable for transmission and receiving. Therefore the indicator in the tunnel management messages shows the corresponding preference. Similarly, for illustration purpose, another tunnel is preferably used bynode 102 to transmit data packets tonode 101 and is not preferably used bynode 102 to receive data packets fromnode 101 even though the tunnel is capable for transmission and receiving. Therefore the indicator in the tunnel management messages shows the corresponding preference of the another tunnel. - In one variant, the first tunnel group comprises one or more tunnels connecting to one or more networks and each network should have latency less than 150 milliseconds. When network latency is limited to less than 150 milliseconds, the network is considered as a fast network. More importantly, 150 milliseconds network latency is lower than the usual network latency of satellite communications. When network latency of the first tunnel group is more than 150 milliseconds, the advantage for
node 101 to use the first tunnel group to transmit data packets tonode 102 becomes not obvious as the network latency of the first tunnel group is not significantly better than the network latency of the second tunnel group if the second tunnel group is connected to one or more satellite connections. In such case, a tunnel management message is sent by either or both ofnode 101 andnode 102 to other node that a tunnel group that was originally being used for receiving only, can then be used for both receiving and transmission, or a tunnel group that was originally being used for transmission only, can then be used for both receiving and transmission. The value of latency can be determined based on the average, maximum, minimum or other statistical calculations of latency observed for each tunnel in the tunnel group. - In one variant, when the first tunnel group has more than one tunnel,
node 101 uses only one tunnel of the first tunnel group at a time to transmit data tonode 102. The tunnel is selected from tunnels of the first tunnel group according to predefined criteria, such as network latency, packet drop rate, bandwidth, price, signal strength and etc. The predefine criteria is entered by the administrator ofnode 101 through a web interface, command line, system console, and/or any other data input mechanism. Alternatively, the predefine criteria is sent tonode 101 through an Internet connection, an intranet connection or SMS from a remote server. The predefined criteria can be stored in a storage medium ofnode 101, such assecondary storage 304 ormain memory 302. In one variant, the predefined criteria are first retrieved from a remote host and then stored in a storage medium ofnode 101 such assecondary storage 304 ormain memory 302. In one variant, when the first tunnel group has more than one tunnel,node 101 uses link load-balancing technique to transmit data packet tonode 102. - In one of the embodiments,
node 101 uses the second tunnel group for transmitting data when a first condition(s) is met. For example, whennode 101 is unable to use the first tunnel group for transmitting data tonode 102 or the performance of the first tunnel group becomes unacceptable, the first condition(s) is met, andnode 101 uses the second tunnel group for transmitting data. In one of the embodiments,node 101 uses the first tunnel group for receiving data when a second condition(s) is met. For example, whennode 101 is unable to use the second tunnel group for receiving data fromnode 102 or the performance of the second tunnel group becomes unacceptable, the second condition(s) are met, andnode 101 uses the first tunnel group for receiving data. The use of condition(s) allows flexibility that a tunnel group can be reconfigured when status or network performance of the tunnel group or another tunnel group changes. - The first and second conditions are based on at least one of the following criteria: coverage, performance, traffic congestion, and latency sensitivity. For example, when the first condition is based on coverage, the first condition is met when the location of
node 101 is out of the coverage of one or more networks that the first tunnel group connects to. When the condition is met,node 101 cannot use the first tunnel group for transmitting data, and thereforenode 101 uses the second tunnel group to transmit data. In another example, the second condition is based on coverage and is met when the location ofnode 101 is out of the coverage of one or more networks that the second tunnel group connects to. When the second condition is met,node 101 cannot use the second tunnel group for receiving data, and thereforenode 101 uses the first tunnel group to receive data. - There are many reasons for
node 101 being out of coverage of a network. For example, when one or more tunnels of the first tunnel group connects to a cellular network which does not provide coverage outside a specific country or region,node 101 becomes out of the coverage of the cellular network whennode 101 moves out of the specific country or region. Using roaming services may not be cost effective in some scenarios, and hencenode 101 starts using the second tunnel group for transmitting data. In another example,node 101 may lose coverage of one or more networks that the second tunnel group connects to, depending on its location, such as when it moves outdoors, or when it moves indoors. The second tunnel group comprisestunnel 201 a which is established throughsatellite network 150. Whensatellite modem 161 moves indoors, and is not pointing toward the sky,node 101 may not be able to receive data throughtunnel 201 a belonging to the second tunnel group from the satellite, thennode 101 uses the first tunnel group to receive data fromnode 102. - In another example, the first condition is based on performance and is met when the performance experienced by data transmitted or received through the first tunnel group becomes very poor and unsatisfactory. For example,
node 101 starts using the second tunnel group for transmitting data when the first condition is met. In another example, the second condition is based on performance and is met when the performance experienced by data received through the second tunnel group becomes very poor. For example,node 101 starts using the first tunnel group for receiving data when the second condition is met. - In another example, when the first condition is based on latency sensitivity, the first condition is met when a data transfer is latency insensitive. When a data transfer is latency insensitive, it can use tunnels with high latency. Therefore, when the first condition is met and the data transfer is latency insensitive,
node 101 can use the second tunnel group for transmitting data since the high latency of the second tunnel group is acceptable for the latency insensitive data transfer. The second tunnel group may have high latency while transmitting data whentunnel 201 a is in the second tunnel group because, for example,tunnel 201 a is established throughsatellite network 150. Alternatively, when the second condition is based on latency sensitivity, and a data transfer fromnode 102 tonode 101 is latency sensitive,node 101 uses the first tunnel group for receiving data fromnode 102. When a data transfer is latency sensitive, it is preferable to use tunnels with low latency. For example, the data being transferred is generated by a real-time audio conversion application and any delay in transmitting or receiving the data will make the audio conversion difficult to be listened to. The second tunnelgroup comprising tunnel 201 a may have higher latency than the first tunnel group. Therefore the first tunnel group is used bynode 101 to receive data belonging to a latency sensitive data transfer fromnode 102. - There can be one or more first conditions and one or more second conditions. The first condition(s) and second condition(s) may or may not be based on the same criteria. A condition can be based on one or more criteria.
- According to one of the embodiments, when a tunnel group has more than one tunnel, the network latency of the first tunnel group is the average latency, the maximum latency, the minimum latency, or the latency variance of the latencies of the more than one tunnels. For example, when the network latency of the first tunnel group is used to determine whether the first tunnel group should be used for transmitting data, then the determination is based on the average latency, the maximum latency, the minimum latency, or the latency variance of the latencies of the more than one tunnels belonging to the first tunnel group.
- In another example, when network latency of the second tunnel group is used to determine whether the second tunnel group should be used for receiving data, then the determination is based on the average latency, the maximum latency, the minimum latency, or the latency variance of the latencies of the more than one tunnels belonging to the second tunnel group. The administrator of
node 101 and/ornode 102 can determine which kind of latency suits their needs. - In one of the embodiments of the present invention, processing
unit 301 determines whether the first tunnel group should be used for transmitting data, based on performance criterion/criteria. The performance criterion/criteria is/are selected from a group comprising packet delay, bandwidth, throughput, packet loss, packet drop, power consumption, and signal noise ratio, round-trip time, interference level, error rate, quality of service, queuing delay, and packet jitter. In one variant, a threshold is predefined for the selected performance criteria, and when the threshold is not satisfied, processingunit 301 determines to use the second tunnel group for transmitting data.Processing unit 301 keeps monitoring the performance of the tunnels of the first tunnel group in order to determine whether the threshold is satisfied or not. - In one of the embodiments, processing
unit 301 determines whether the second tunnel group should be used for receiving data, based on performance criterion/criteria. The performance criterion/criteria is/are selected from a group comprising network latency, response time, packet delay, bandwidth, throughput, packet loss, packet drop, power consumption, and signal noise ratio, round-trip time, interference level, error rate, quality of service, queuing delay, and packet jitter. In one variant, a threshold is predefined for the selected performance criteria, and when the threshold is not satisfied, processingunit 301 determines to use the first tunnel group for receiving data.Processing unit 301 keeps monitoring the performance of the tunnels of the second tunnel group in order to determine whether the threshold is satisfied or not. - In one of the embodiments of the present invention, processing
unit 301 selects tunnels for the first tunnel group and the second tunnel group based on one or more performance criteria. The performance criteria are selected from a group comprising network latency, response time, packet delay, bandwidth, throughput, packet loss, packet drop, power consumption, signal noise ratio, round-trip time, interference level, error rate, quality of service, queuing delay, usage price, location and packet jitter. In one variant, thresholds are predefined for the selected performance criteria corresponding to the first tunnel group and the selected performance criteria corresponding to the second tunnel group. For example, processingunit 301 determines whethertunnel 201 b satisfies the threshold for the selected performance criteria corresponding to the first tunnel group. Iftunnel 201 b satisfies the threshold for the selected performance criteria corresponding to the first tunnel group,tunnel 201 b belongs to the first tunnel group. Similarly, processingunit 301 determines whethertunnel 201 a satisfies the threshold for the selected performance criteria corresponding to the second tunnel group. Iftunnel 201 a satisfies the threshold for the selected performance criteria corresponding to the second tunnel group,tunnel 201 a belongs to the second tunnel group. The performance criteria for selecting tunnels for the first tunnel group and the performance criteria for selecting tunnels for the second tunnel group may or may not be the same. - It is possible that a tunnel does not belong to any tunnel group if its performance does not meet the performance criterion/criteria of the tunnel group.
- In one of the embodiments of the present invention, processing
unit 301 keeps monitoring performance of each of the tunnels belonging to the first tunnel group and the second tunnel group. For example,tunnel 201 b and tunnel 201 c belong to the first tunnel group. The selected performance criteria corresponding to the first tunnel group are network latency and throughput. The threshold defined for network latency is 150 milliseconds, such that the network latency of a network that a tunnel connects to must be less than 150 milliseconds. The threshold for throughput is 20 Mbps, such that the throughput of data transmission through a tunnel should in general equal to or more than 20 Mbps.Processing unit 301 periodically determines whethertunnel 201 b satisfies the thresholds for network latency and throughput. If the network latency of a network thattunnel 201 b connects to become significantly more than 150 milliseconds, processingunit 301 removestunnel 201 b from the first tunnel group becausetunnel 201 b does not satisfy the threshold. Similarly, if the throughput of data transmission throughtunnel 201 b becomes in general less than 20 Mbps, processingunit 301 removestunnel 201 b from the first tunnel group becausetunnel 201 b does not satisfy the threshold. In one variant,tunnel 201 b should satisfy thresholds of both the performance criteria for remaining in the first tunnel group. Alternatively,tunnel 201 b should satisfy threshold of at least one of the performance criteria for remaining in the first tunnel group. In one variant, processingunit 301 determines to removestunnel 201 b from the first tunnel group only aftertunnel 201 b has not been satisfying the thresholds of the performance criteria for a predefined time period. For example, processingunit 301 determines periodically whethertunnel 201 b should remain in the first tunnel group. Iftunnel 201 b does not satisfy the threshold of the performance criteria during three consecutive determinations ofprocessing unit 301, then processingunit 301 removestunnel 201 b from the first tunnel group. In another example, if processingunit 301 determines that network latency of a network thattunnel 201 b connects to has been more than 150 milliseconds for more than fifteen minutes, processingunit 301 removestunnel 201 b from the first tunnel group. Whentunnel 201 b is removed from the first tunnel group, processingunit 301 uses tunnel 201 c of the first tunnel group to transmit data. - Throughput of a tunnel can be determined by processing
unit 301 by transmitting data packets or testing packets through the tunnel. The throughput estimated can be average throughput, minimum throughput maximum throughput. The format of the testing packets is preferably the same as the format of data packets that are transmitted usingtunnel 201 b. - In another example,
tunnel Processing unit 301 periodically determines whether the usage price oftunnel tunnel 201 a, becomes above the threshold, processingunit 301 removestunnel 201 a from the second tunnel group. Whentunnel 201 a is removed from the second tunnel group,node 101 usestunnel 201 d for receiving data.Processing unit 301 also monitors packet drop rate throughtunnels tunnel 201 d, becomes higher than a threshold, processingunit 301 removestunnel 201 d from the second tunnel group. In one variant,tunnel 201 a should satisfy thresholds of both the performance criteria for being kept in the second tunnel group. Alternatively,tunnel 201 a should satisfy threshold of at least one of the performance criteria for being kept in the second tunnel group. - According to one of the embodiments of the present invention, a tunnel belongs to a tunnel group but is not being utilized. For illustration purpose only,
tunnels tunnel 201 a belongs to the second tunnel group. The first tunnel group is used to transmit data fromnode 101 tonode 102 and the second tunnel group is used to transmit data fromnode 102 tonode 101. It is possible that not alltunnels node 101 tonode 102. In one variant, only one of thetunnels tunnels node 101, administrator ofnode 102 or the system operator operating bothnodes - In one of the embodiments of the present invention, a priority is assigned to each tunnel of a tunnel group by
node 101. For example,tunnels Node 101 assigns highest priority totunnel 201 b, second-highest priority to tunnel 201 c and lowest priority totunnel 201 d.Node 101 uses any two tunnels belonging to the first tunnel group at a given time for transmitting data. Therefore, sincetunnels 201 b and 201 c have a higher priority thantunnel 201 d,node 101 usestunnels 201 b and 201 c for transmitting data when possible. If a connection throughtunnel 201 b fails,node 101 then starts usingtunnel 201 c and 201 d for transmitting data. The connection throughtunnel 201 b may fail for various reasons, such as traffic congestion,tunnel 201 b not having enough capacity, no network coverage, or equipment malfunctioning. Alternatively, if performance of data transmission throughtunnel 201 b deteriorates significantly,node 101stops using tunnel 201 b and starts usingtunnels 201 c and 201 d for transmitting data. - In another example,
tunnels Node 101 assigns highest priority totunnel 201 a and lowest priority totunnel 201 b. For example,node 101 is configured with a predefined configuration to assign highest priority totunnel 201 a in the second tunnel group becausetunnel 201 a is established throughsatellite network 150. In another example,node 101 assigns the highest priority totunnel 201 a in the second tunnel group becausetunnel 201 a has the highest throughput and largest coverage area. If a connection throughtunnel 201 a fails,node 101starts using tunnel 201 b for receiving data. The connection throughtunnel 201 a may fail for various reasons. For example, whensatellite modem 161 is moved indoors, then it may not point directly atsatellite network 150, and thus the connection throughtunnel 201 a may fail. In another example, the network latency for receiving data throughtunnel 201 a may become very high and the connection may fail because of high network latency. - Priority is assigned to tunnels in a tunnel group according to one or more performance criteria or can be assigned according to a predefined configuration. A predefined configuration may be entered by an administrator or a user of
node 101 through a web interface, an application programming interface (API), a command line interface or a console. - When the priority is assigned according to performance criteria,
node 101 determines performance of each tunnel in a tunnel group. For example, processingunit 301 may assign highest priority totunnel 201 b in the first tunnel group becausetunnel 201 b has the highest throughput and the lowest packet drop rate. Alternatively, processingunit 301 may assign highest priority totunnel 201 b in the first tunnel group becausetunnel 201 b has the lowest usage price. - In one of the embodiments of the present invention, when performance of a tunnel in a tunnel group deteriorates, the tunnel is not removed from the tunnel group, but
node 101 determines not to use the tunnel. For example, when performance of a tunnel, such astunnel 201 b, belonging to the first tunnel group, deteriorates and becomes unsatisfactory,node 101stops using tunnel 201 b for transmitting data until the performance oftunnel 201 b becomes satisfactory again.Tunnel 201 b is marked as inactive andnode 101 uses other tunnel(s) in the first tunnel group, such as tunnel 201 c, to transmit data.Node 101 may determine that performance oftunnel 201 b has deteriorated based on one or more performance criteria. For example, when packet drop rate of data transmission throughtunnel 201 b has increased significantly,node 101stops using tunnel 201 b for transmitting data. - In another example, when performance of a tunnel, such as
tunnel 201 a, belonging to the second tunnel group, deteriorates and becomes unsatisfactory,node 101stops using tunnel 201 a for receiving data until the performance oftunnel 201 a becomes satisfactory again.Tunnel 201 a is marked as inactive andnode 101 uses other tunnel(s) in the second tunnel group, such astunnel 201 d, to receive data.Node 101 may determine that performance oftunnel 201 a has deteriorated based on one or more performance criteria. For example, when latency oftunnel 201 a increases significantly,node 101stops using tunnel 201 a for receiving data. - The benefits of not removing a tunnel whose performance has deteriorated from a tunnel group includes that
node 101 may be to be able to start using the tunnel again as soon as the performance of the tunnel becomes satisfactory.Processing unit 301 does not need to determine again as to whether the tunnel should belong to the tunnel group or not. - In one of the embodiments, a tunnel may belong to both the first tunnel group and the second tunnel group. For example,
tunnel 201 b belongs to both the first tunnel group and the second tunnel group.Tunnel 201 b is then used bynode 101 for transmitting and receiving data to and fromnode 102 respectively. The first tunnel group comprisestunnels 201 b and 201 c, and the second tunnel group comprisestunnels Node 101 usestunnels 201 b and 201 c for transmitting data, andtunnels node 101 uses tunnel 201 c only for transmitting data,tunnel 201 a only for receiving data, andtunnel 201 b for both transmitting and receiving data.Processing unit 301 may selecttunnel 201 b for belonging to both the first tunnel group and the second tunnel group becausetunnel 201 b has very good performance, such as high throughput, low packet drop rate, low latency, low usage price, or good performance based on other performance criteria. - When processing
unit 301 ofnode 101 orprocessing unit 401 ofnode 102 determines that a tunnel leaves or joins a tunnel group, new tunnel management messages are sent by the processing unit to inform the other processing unit that the tunnel should be used or not and how to use the tunnel. - In one variant, when tunnel management messages indicate that the first tunnel group is only used by
node 101 to transmit data tonode 102, the first tunnel group is not allowed to be used for receiving data fromnode 101 tonode 102 even if a corresponding condition has been met. Similarly when the tunnel management message indicates that the second tunnel group is only used bynode 101 to receive data fromnode 102, the second tunnel group is not allowed to be used for transmitting data tonode 102 even if a corresponding condition has been met. Alternatively, new tunnel management messages are sent bynode 101 ornode 102 to update that the first tunnel group and/or the second tunnel group can be used for bidirectional communications. Alternatively, new tunnel management messages are sent bynode 101 ornode 102 to update that the first tunnel group and/or the second tunnel group are no longer used for bidirectional communications and can only used for transmitting data fromnode 101 and/or receiving data bynode 101 respectively. - The use of new tunnel management messages allow the flexibility to change how the first tunnel group and the second tunnel group is used for transmitting and receiving data after the tunnel have been established. In one variant, the new tunnel management messages are used to manage one or more specific tunnel, instead of a tunnel group. This allows more granular management of tunnel.
- In one variant, there is no need to transmit tunnel management message when the first tunnel group has already been used to transmit data from
node 102 tonode 101. This is because of the fact that when data has arrived from the first tunnel group tonode 101,node 101 is able to determine thatnode 102 has determined to use the first tunnel group to transmit data without the assistance of tunnel management message. Similarly, there is no need to transmit tunnel management message when the second tunnel group has already been used to transmit data fromnode 101 tonode 102. - In one variant, in the case that
node 101 is no longer able to or determines not to receive data through the second tunnel group or one of the tunnels of the second tunnel group, such as in the situation of being out of coverage,node 101 will informnode 102, by using tunnel management message, not to further transmit data through the second tunnel group or the one of the tunnels of the second tunnel group. Similarly, in the case thatnode 102 is no longer able to or determines not to receive data through the first tunnel group or one of the tunnels of the first tunnel group, such as in the situation of heavy packet loss,node 102 will informnode 101, by using tunnel management message, not to further transmit data through the first tunnel group or the one of the tunnels of the first tunnel group. -
FIG. 6A andFIG. 6B are two flowcharts illustrating one of the embodiments of the present invention thatnode 101 transmits and receives data packets to and fromnode 102 respectively. - Referring to
FIG. 1 ,node 101 first selects a first one or more WAN interfaces ofnode 101, such asWAN interface step 601. Whennode 101 receives a data packet from a host or node that connects to one of itsLAN interfaces step 602, it encapsulates the data packet into one or more encapsulating packets atstep 603. Those who are skilled in the art would appreciate that the step of encapsulating a data packet allows the data packet be sent through a tunnel. In one variant,step 601 and step 602 can be swapped such that the selection of WAN interface is performed after a data packet has been received. The first one or more WAN interfaces ofnode 101 selected are mainly used for transmitting data packets fromnode 101 tonode 102. In one variant,node 102 does not transmit data packets tonode 101 through the first one or more WAN interfaces ofnode 101. In one variant,node 102 transmits a small number of packets tonode 101 through the first one or more WAN interfaces ofnode 101, and the packets are used for managing purpose, such as status information of the connections connected to the first one or more WAN interfaces. - Then, at
step 604,node 101 transmits the one or more encapsulating packets through the first one or more WAN interfaces selected atstep 601. The first one or more WAN interfaces may connect to a wired or wireless network. -
Node 101 also selects a second one or more WAN interfaces mainly for receiving data packets fromnode 102. The data packets fromnode 102 may be encapsulating packets. In one variant,node 102 only transmits data packets tonode 101 through the second one or more WAN interfaces ofnode 101, such thatnode 101 only receives data packets fromnode 102 through the second one or more WAN interfaces ofnode 101. In one variant,node 102 transmits a small number of packets tonode 101 through WAN interface(s) ofnode 101 that do(does) not belong to the second one or more WAN interfaces ofnode 101; the packets are used for managing purpose, such as transmitting health check packets, status information of the connections connected to the one or more WAN interfaces. Such that,node 101 only receives the small number of packets fromnode 102 through the WAN interfaces ofnode 101 not belonging to the second one or more WAN interfaces ofnode 101. - In one variant,
node 101 sendsnode 102 information of the second one or more WAN interfaces by using a WAN interface management message. In such case, the information includes the identities of WAN interface(s) ofnode 101 that is(are) preferred fornode 102 for transmitting data packets tonode 101. Thereforenode 102 can set the destination address of the encapsulating packet to be the address of the preferred WAN interface(s) ofnode 101. In one variant, the information is used as an instruction fornode 102. In one variant,node 102 also determines by itself which WAN interface(s) ofnode 101 should be used to receive data packets fromnode 102 tonode 101. - The usage of WAN interface management message includes: informing
node 102 how to use a WAN interface ofnode 101, informingnode 102 when to transmit data packets through a particular WAN interface ofnode 101. In one variant,node 101 receives WAN interface management messages fromnode 102 for managing one or more WAN interfaces. Therefore, the usage of WAN interface management message also includes: informingnode 101 how to use a WAN interface, informingnode 101 when to transmit data packets through a particular WAN interface. - As shown in
FIG. 6B , whennode 101 receives an encapsulating packet fromnode 102 through one ofWAN interfaces step 611,node 101 decapsulates the encapsulating packet to retrieve a data packet atstep 612. Thennode 101 transmits the data packet to a host or node through one of itsLAN interfaces step 613. - It is preferred that
node 101 selects the first one or more WAN interface that connects to a network with network latency less than 150 milliseconds atstep 601. This allows the encapsulating packets to reachnode 102 quickly. Therefore, it is preferred not to choose a WAN interface that connects to a satellite network because the satellite round trip time is usually more than 150 milliseconds. - In one variant, the encapsulation and decapsulation performed by
node 101 atstep 603 and step 612 respectively particularly allows data to be transmitted to and received fromnode 102 through one or more tunnels. The one or more tunnels can be established via a pair of WAN interfaces, such asWAN interface 121 b andWAN interface 131 b. - In one preferred embodiment,
node 102 transmits encapsulating packets tonode 101 through a satellite network, for example throughtunnel 201 a. Although the satellite round trip time may be larger than the round trip time of other networks, the bandwidth offered by a satellite network could be larger than that of other networks. Further, it may be more economical for the administrator ofnode 102 to transmit data packets, and/or encapsulating packets tonode 101 through a satellite network. - In one variant, processing
unit 401 ofnode 102 determines which WAN interface ofnode 101 is used to transmit the encapsulating packets fromnode 101 tonode 102. For example,node 102 may selectWAN interface 121 b ofnode 101. Whennode 102 has made the WAN interface selection decision, it sends a WAN interface management message tonode 101 about the decision. Without the WAN interface management message,node 101 will not be able to recognize the need to useWAN interface 121 b to transmit encapsulating packets tonode 102. - According to one of the embodiments of the present invention, the WAN interface selection decision at
step 601 and the WAN interface selection decision fornode 102 to determine which WAN interface for transmitting and receiving encapsulating packets can be determined according to: network performance of network connecting to WAN interfaces, conditions, network coverage, geographical location, network usage price, etc. -
FIG. 6C is a flowchart illustrating various embodiments according to the present invention.Node 101 transmits data packets to and receives data packets fromnode 102 respectively.Node 101 andnode 102 together perform as a proxy/gateway for the host or node that connects to one of the LAN interfaces 122 a and 122 b ofnode 101. Those who are skilled in the arts would appreciate the techniques of updating source address, updating destination address, and restoring the original destination address are described below, in general, deployed by proxies or gateways. - Similar to the flowchart illustrated in
FIG. 6A ,node 101 first selects one or more WAN interfaces, such asWAN interface step 631. Whennode 101 receives a data packet from a host or node that connects to one of itsLAN interfaces step 632,node 101 modifies the destination address of the data packet received to be the address of one of WAN interfaces ofnode 102 atstep 633. The original destination address of the data packet is recorded and is sent tonode 102 before the data packet is transmitted. The original destination address is used fornode 102 to restore the destination address. - Then
node 101 transmits the data packet with the modified destination address through the one or more WAN interfaces selected atstep 634. Whennode 102 receives the packet,node 102 restores the destination address to the original destination address and modifies the source address of the data packet to be the address of one of the WAN interfaces ofnode 102, and then transmits the data packet to the original destination address. The purpose of restoring the original destination address is to allow the data packet be transmitted to the original destination address. The purpose of updating the source address of the data packet to be the address of one of the WAN interfaces ofnode 102 is to allow the host or node of the original destination address be able to respond to the transmitted data packet using the address of one of the WAN interfaces ofnode 102, instead of using one of the WAN interfaces ofnode 101. Whennode 102 receives the response, it forwards the response to one of the WAN interfaces ofnode 101 by modifying the destination address of the IP packets holding the response to the address of the WAN interface ofnode 101. Whennode 101 receives the IP packets holding the response,node 101 modifies the source address of the IP packets to the address of the original destination address. - In one variant, the WAN interface of
node 101 selected bynode 101 for transmitting the data packets tonode 102 and the WAN interface ofnode 101 selected bynode 102 for receiving data packets fromnode 102 can be the same or different. For example, atstep 601node 101 selectsWAN interface 121 b to transmit encapsulating packets andnode 102 selectsWAN interface 121 a to receive encapsulated response to the data packets encapsulated in the encapsulating packets. Similarly, for example, atstep 631,node 101 selectsWAN interface 121 c to transmit data packets andnode 102 selectsWAN interface 131 c to transmit response to the data packets tonode 101. -
FIG. 7 illustrates a network configuration according to various embodiments of the present invention. -
Node 702 can perform as a gateway, VPN gateway, or proxy fornode 101 such that, for example,node 101, host 103 b, hosts and nodes ininterconnected networks 117 and host 103 a can transmit and receive data packets throughnode 702. - According to one of the embodiments of the present invention,
node 101 is configured to use a first one or more WAN interface to send packets tonode 702 and a second one or more WAN interface to receive packets fromnode 702. For example,node 101 usesWAN interfaces node 702 and usesWAN interface 121 a receive data packets fromnode 702.Node 101 can be configured by an administrator or a network operator, such as the operator ofsatellite network 150 and the operator ofwireless network 180. In one example, asnode 101 is already configured which WAN interface(s) is (are) used, there is no need to performstep 601 as there is no WAN interface selection being performed. -
FIG. 8 illustrates the communication steps of an example thatnode 702 performs as a gateway forhost 103 b to communicate with a host ininterconnected networks 171. -
FIG. 9 illustrates the structure of IP packet being transmitted at each communication step ofFIG. 8 . Whenhost 103 b transmitsIP packet 901 to a host ininterconnected network 171, host 103 b first transmitsIP packet 901 tonode 101 atstep 801.IP packet 901 could be for service request, data transfer or any other purposes. The structure ofIP packet 901 is illustrated inFIG. 9 . InIP packet 901, the payload field is used to hold data. The header section has a destination address field and a source address field. The destination address field is the IP address of the host ininterconnected networks 171. The source address field is the IP address ofhost 103 b. - After
node 101 has receivedIP packet 901 atcommunication step 801,node 101 encapsulatesIP packet 901 inIP packet 902. Asnode 101 has a plurality of WAN interfaces,node 101 can choose any of the WAN interfaces to transmitIP packet 902. Alternatively,node 101 can select a WAN interface that has been determined or configured before theIP packet 901 has arrived. For example,node 101 selects a WAN interface based on: a configuration entered by an administrator ofnode 101; latency of the connection that a WAN interface connected to; a condition entered by an administrator ofnode 101; and instructions received from a remote server or node. - For illustration purpose,
node 101 selectsWAN interface 121 b, then transmitsIP packet 902 towireless network 180 through a wireless modem connecting toWAN interface 121 b atcommunication step 802. The payload field ofIP packet 902stores IP packet 901. The header section ofIP packet 902 has a destination address field and a source address field. The destination address field ofIP packet 902 is the IP address ofWAN interface 703 ofnode 702. The source address field ofIP packet 902 is IP address ofWAN interface 121 b ofnode 101. - As
WAN interface 703 is reachable throughrouter 704,wireless network 180routes IP packet 902 torouter 704 atcommunication step 803.Wireless network 180 may connect torouter 704 through private interconnected networks or public interconnected networks. For example, ifnode 702 andwireless network 180 are operated by the same administrator,wireless network 180 can routeIP packet 902 through a direct connection torouter 704. On the other hand, ifnode 702 andwireless network 180 are not operated by the same administrator, in general,IP packet 902 will be routed torouter 704 through public interconnected networks. - When
router 704 receivesIP packet 902, itroutes IP packet 902 toWAN interface 703 atcommunication step 804. In one variant,router 704 performs as a gateway or a proxy with network address translation (NAT) capability and the destination address ofIP packet 902 is then updated accordingly. - When
node 702 receivesIP packet 902, it decapsulatesIP packet 902 to retrieveIP packet 901. After examining the destination address ofIP packet 901,node 702 performs network address translation (NAT) on the source address ofIP packet 901. The modifiedIP packet 901 isIP packet 903. ThereforeIP packet 903 is based onIP packet 901. The source address ofIP packet 903 is the IP address ofWAN interface 703. Therefore if the host ininterconnected network 171 replies,WAN interface 703 can receive the reply. The destination address ofIP packet 903 is the same as the destination address ofIP packet 901, which is the IP address of the host ininterconnected networks 171 that host 103 b tries to sendIP packet 901 to.Node 702 then transmitsIP packet 903 torouter 704 atcommunication step 805. - When
router 704 receivesIP packet 903, itroutes IP packet 903 to the host ininterconnected networks 171 atcommunication step 806. In one variant,router 704 performs as a gateway or a proxy with NAT capability and the source address ofIP packet 903 is then updated accordingly. - When the host in
interconnected networks 171 receivesIP packet 903, the host may or may not reply. If the host replies, it should reply with at least one IP packet, forexample IP packet 904, atcommunication step 807. The destination address and source address ofIP packet 904 is the IP address ofWAN interface 703 and the IP address of the host ininterconnected networks 171 respectively. - When
router 704 receivesIP packet 904, itroutes IP packet 904 tonode 702 atcommunication step 808. In one variant,router 704 performs as a gateway or a proxy with NAT capability and the destination address ofIP packet 904 is then updated accordingly. - When
node 702 receivesIP packet 904, it performs NAT onIP packet 904. Such thatIP packet 904 has been modified and the destination address of the modifiedIP packet 904 becomes the IP address ofhost 103 b. Thennode 702 encapsulates modifiedIP packet 904 inIP packet 905 and then transmitsIP packet 905 torouter 704 atcommunication step 809. The payload field ofIP packet 905 stores modifiedIP packet 904. ThereforeIP packet 905 is based onIP packet 904. The header section ofIP packet 905 has a destination address field and a source address field. The destination address field ofIP packet 905 is the IP address of one of WAN interfaces ofnode 101. For illustration purpose,node 702 selectsWAN interface 121 a. Therefore, the destination addressfield IP packet 905 is IP address ofWAN interface 121 a. The source address field ofIP packet 905 is IP address ofWAN interface 703. - As there can be more than one connections connecting to the plurality of WAN interfaces of
node 101,node 702 can select one of the WAN interfaces ofnode 101 for transmittingIP packet 905 tonode 101. Alternatively,node 702 can select a WAN interface that has been determined or configured before theIP packet 904 has arrived. For example,node 702 selects a WAN interface ofnode 101 based on: a configuration entered by an administrator ofnode 702; latency of the connection that a WAN interface connects to; a condition entered by an administrator ofnode 702; and instructions received from a remote server or node. - When
router 704 receivesIP packet 905, itroutes IP packet 905 tosatellite network 150 throughsatellite modem 160 atcommunication step 810. In one variant,router 704 performs as a gateway or a proxy with NAT capability and the destination address ofIP packet 905 is then updated accordingly. Incommunication step 811,satellite modem 160 transmitsIP packet 905 tosatellite network 150, which then transmitsIP packet 905 tosatellite modem 161.Satellite modem 161forwards IP packet 905 tonode 101 incommunication step 812 throughconnection medium 113. - When
node 101 receivesIP packet 905, it decapsulatesIP packet 905 to retrieve modifiedIP packet 904, which is the same asIP packet 906 and then transmitsIP packet 906 to host 103 b according to the destination address in the header ofIP packet 906 incommunication step 813. - The communication steps shown in
FIG. 8 illustrates that different WAN interfaces ofnode 101 are used for transmitting and receiving IP packets, even if the IP packets belong to the same session. - Those who are skilled in the arts would appreciate that when
router 704 performs NAT,router 704 may need to use a state database to record the translation and the state database is in general stored in a storage unit ofrouter 704. - In one variant, when
node 702 receivesIP packet 904 afterstep 808, it does not performs NAT onIP packet 904. ThereforeIP packet 904 is not modified andIP packet 905 encapsulatesIP packet 904, instead of modifiedIP packet 904. The destination of address ofIP packet 904 is changed bynode 101 toIP packet 906 afterstep 812. This allows NAT being performed bynode 101, instead of bynode 702. This may reduce the computing resources requirement ofnode 702. - According to one of the embodiments of the present invention,
node 101 sends a WAN interface management message to informnode 702 which WAN interface ofnode 101 shouldnode 702 transmit IP packets to. WAN interface management message comprises a WAN interface identity field and an instruction field. For example, for illustration purpose only, WAN interfaces 121 a and 121 d can be used bynode 101 to receive IP packets fromnode 702 while WAN interfaces 121 b and 121 c cannot be used bynode 101 to receive IP packets fromnode 702. The WAN interface management message comprises the identities ofWAN interfaces node 101 to receive IP packets fromnode 702 while WAN interfaces 121 b and 121 c cannot be used bynode 101 to receive IP packets fromnode 702. The WAN interface management messages may be sent anytime beforeIP packet 905 is transmitted bynode 702 instep 809. In another example, identity(identities) of WAN interface(s) ofnode 101 that is(are) used fornode 101 to transmit IP packets tonode 702 is(are) stored in the WAN interface field and the instruction that this (these) WAN interface(s) is(are) used for transmission only is stored in the instruction field. - In one variant, a WAN interface management message comprises an indicator in the instruction field to indicate whether a WAN is for both transmission and receiving, transmission only or receiving only. For example, for illustration purpose, The indicator can represented by a bit, a plurality of bits, a byte, a plurality of bytes, a string, a plurality of strings, XML messages, etc. Those who are skilled in the arts would appreciate that there are myriads ways to represent the indicator.
- In one variant, WAN interface management messages is determined and transmitted by one of
node 101 ornode 702. Therefore, the decision of which WAN interface ofnode 101 to use for transmitting and/or receiving IP packets is determined bynode 101 ornode 702. For example, an administrator ofnode 101 can determine which WAN interface(s) ofnode 101 andnode 702 is(are) used for transmitting and/or receiving IP packets. Therefore, a WAN interface can be set as unidirectional transmission, unidirectional receiving or bidirectional. - In one variant, a WAN interface management message comprises status information of a WAN interface. The status information can be part of information stored in the instruction field. In one variant, the status is stored in a status field of WAN interface management message. For example, when
node 101updates node 102 about the status of a WAN interface,node 101 sends a WAN interface management message with the WAN interface status in the status field tonode 102. The WAN interface identity field holds the identity of the corresponding WAN interface. The instruction field in this case may be empty as the WAN interface management message is for status reporting purpose. - According to one of the embodiments of the present invention, one or more tunnels are established between two WAN interfaces of two nodes and when a WAN interface is selected for transmission and receiving, transmission only or receiving only, the corresponding one or more tunnels that are established through the selected WAN interface will be affected by WAN interface management message. For example, when WAN interface 121 a can only be used for receiving IP packets from
node 702 and cannot be used for transmitting IP packets tonode 702, the one or more tunnels that established throughWAN interface 121 a toWAN interface 705 can also only be used for receiving IP packets fromnode 702 and cannot be used for transmitting IP packets tonode 702. - According to one of the embodiments of the present invention, an administrator of a node determines WAN interface pair stating a WAN interface of the node and a WAN interface of another node, such that the WAN interfaces stated in the WAN interface pair are used for transmitting and receiving, transmitting only or receiving only IP packet between the two nodes. For example, the administrator determines a WAN interface pair,
WAN interface 121 a andWAN interface 703, which is used for receiving IP packets fromWAN interface 703 only. In another example, the administrator determines that a WAN interface pair,WAN interface 121 d andWAN interface 703, is bidirectional, such that WAN interface 121 andWAN interface 703 can be used for transmitting and receiving IP packets to and from each other. - According to one of the embodiments, a node, such as
node 102, intermediates access by a host, such ashost interconnected networks 172. In one example,node 102 terminates incoming access requests and connections at the application layer of the Open System Interconnection (OSI) reference model or of the TCP/IP model. In this example,node 102 operates as an application-layer proxy to protect resources ininterconnected networks 172 from direct exposure to hosts connected tonode 101.Node 102 receives incoming access requests encapsulated in a packet, decapsulates the access requests to reach the underlying application data, and sends the application data comprising the access requests tointerconnected networks 172. - In another example,
node 102 allows direct connections between layers of the OSI reference model or of the TCP/IP model. In this example,node 102 exchanges data using a secure channel negotiated with the requesting host, such ashost Node 102 receives a secure request via the one oftunnels interconnected networks 172 on behalf of the requesting host, i.e.host interconnected networks 172. -
FIG. 10 illustrates a webpage used to configurenode 101.Webpage 1001 can be shown at a display of a laptop computer, desktop computer, handheld computing device, mobile phone or any device capable of displayingwebpage 1001.Webpage 1001 has two parts:VPN Profile 1011 andWAN Connection Priority 1012. An administrator can enter the information required to setup a VPN usingVPN Profile 1011 and configure priority, transmission and receiving of data of WAN interfaces inWAN Connection Priority 1012. The information is used to create one or more tunnels belonging to the VPN. For example, the information is used to create multiple tunnels and the multiple tunnels are aggregated to form one aggregated tunnel for the VPN. Those who are skilled in the art would appreciate that the information required includes security information, identity information and encryption information. In one variant, the information required is retrieved from a remote management server, a secured device coupled tonode 101 or a preconfigured database. This releases the administrator the burden to enter required information manually through the web page. -
WAN Connection Priority 1012 has three main sections:WAN interface identity 1021,priority selection 1022 anddirection selections 1031 to 1033. Asnode 101 has four WAN interfaces,WAN interface identity 1021 has four WAN interfaces shown. For example, “WAN 1”, “WAN 2”, “WAN 3” and “WAN 4” inWAN interface identity 1021 corresponds toWAN interfaces Priority selection 1022 is used to configure priority to WAN interfaces. For example, the priority of “WAN 1” is configured to be “OFF” thatprocessing unit 301 will not useWAN interface 121 a to transmit or receive data traffic. As the priority of “WAN 2”, “WAN 3”, “WAN 4” are configured to be “highest”, processingunit 301 usesWAN interfaces Direction selection 1031 is configured to be “Up Only”, such thatprocessing unit 301 only usesWAN interface 121 b to transmit data.Direction selection 1032 is configured to be “Down Only”, such thatprocessing unit 301 only usesWAN interface 121 c to receive data.Direction selection 1033 is configured to be “Up/Down Only”, such thatprocessing unit 301 usesWAN interface 121 d to both transmit and receive data. - In one variant, the information used to configure
WAN Connection Priority 1012 is retrieved from a remote management server, a secured device coupled tonode 101 or a preconfigured database. This releases the administrator the burden to enter required information manually through the web page. - The web page for configuring
node 102 is similar and the number of WAN interfaces inWAN interface identity 1021 will be three asnode 102 has three WAN interfaces. -
FIG. 11 illustrates a webpage used to configurenode 101. The design ofwebpage 1101 is similar to the design ofwebpage 1001.Webpage 1101 has two parts:VPN Profile 1111 andTunnel Priority 1112. An administrator can enter the information required to setup a VPN usingVPN Profile 1111 and configure priority, transmission and receiving of data of tunnels inTunnel Priority 1112.VPN Profile 1111 can be identical toVPN Profile 1011 if the information required to setup VPN is the same. -
Tunnel Priority 1112 has three main sections:tunnel identity 1121,priority selection 1122 anddirection selections 1131 to 1133. Asnode 101 has established four tunnels withnode 102,tunnel identity 1121 has four tunnels shown. For example, “Tunnel A”, “Tunnel B”, “Tunnel C” and “Tunnel D” intunnel identity 1121 corresponds totunnels Priority selection 1122 is used to configure priority to tunnel. For example, the priority of “Tunnel A” is configured to be “OFF” thatnode 101 will not usetunnel 201 a to transmit or receive data traffic. As the priority of “Tunnel B”, “Tunnel C”, “Tunnel D” are configured to be “highest”, processingunit 301 usestunnels Direction selection 1131 is configured to be “Up Only”, such thatnode 101 only usestunnel 201 b to transmit data.Direction selection 1132 is configured to be “Down Only”, such thatnode 101 only uses tunnel 201 c to receive data.Direction selection 1133 is configured to be “Up/Down Only”, such thatnode 101 usestunnel 201 d to transmit and receive data. - In one variant, the information used to configure
Tunnel Priority 1112 is retrieved from a remote management server, a secured device coupled tonode 101 or a preconfigured database. This releases the administrator the burden to enter required information manually through the web page. - The web page for configuring
node 102 is similar and the number tunnels intunnel identity 1121 will be the same as the number of tunnels established betweennode - In one variant, instead of shown tunnel identities in
tunnel identity 1121, WAN interfaces ofnode 101 andnode 102 are shown intunnel identity 1121. As a tunnel can be established between one or more WAN interfaces ofnode 101 and one or more WAN interfaces ofnode 102, a tunnel identity can be replaced with the WAN interfaces used for establishing the tunnel. - The foregoing description of preferred embodiments of the present invention provides illustration and description, but is not intended to be exhaustive or to limit the invention to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from the practice of the invention.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/411,057 US20190268181A1 (en) | 2014-03-03 | 2019-05-13 | Methods and systems for transmitting and receiving data through tunnel groups |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/IB2014/059378 WO2015132624A1 (en) | 2014-03-03 | 2014-03-03 | Methods and systems for transmitting and receiving data through tunnel groups |
US14/396,747 US10291431B2 (en) | 2014-03-03 | 2014-03-03 | Methods and systems for transmitting and receiving data through tunnel groups |
US16/411,057 US20190268181A1 (en) | 2014-03-03 | 2019-05-13 | Methods and systems for transmitting and receiving data through tunnel groups |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2014/059378 Continuation WO2015132624A1 (en) | 2014-03-03 | 2014-03-03 | Methods and systems for transmitting and receiving data through tunnel groups |
US14/396,747 Continuation US10291431B2 (en) | 2014-03-03 | 2014-03-03 | Methods and systems for transmitting and receiving data through tunnel groups |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190268181A1 true US20190268181A1 (en) | 2019-08-29 |
Family
ID=52001433
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/396,747 Active 2034-07-03 US10291431B2 (en) | 2014-03-03 | 2014-03-03 | Methods and systems for transmitting and receiving data through tunnel groups |
US16/411,057 Abandoned US20190268181A1 (en) | 2014-03-03 | 2019-05-13 | Methods and systems for transmitting and receiving data through tunnel groups |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/396,747 Active 2034-07-03 US10291431B2 (en) | 2014-03-03 | 2014-03-03 | Methods and systems for transmitting and receiving data through tunnel groups |
Country Status (4)
Country | Link |
---|---|
US (2) | US10291431B2 (en) |
CN (3) | CN112383962B (en) |
GB (1) | GB2525693B (en) |
WO (1) | WO2015132624A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3863238A1 (en) * | 2020-02-08 | 2021-08-11 | Ihor Yartsev | System, method and use of software-defined virtual multi-service tunnel networking |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10637782B2 (en) | 2011-03-23 | 2020-04-28 | Hughes Network Systems, Llc | System and method for policy-based multipath WAN transports for improved quality of service over broadband networks |
FR3028371B1 (en) * | 2014-11-06 | 2016-11-18 | Bull Sas | METHOD FOR MONITORING AND CONTROLLING DEPORTS OF A CLUSTER USING AN INFINIBAND-TYPE COMMUNICATION NETWORK AND COMPUTER PROGRAM USING SAID METHOD |
US10225105B2 (en) * | 2015-07-08 | 2019-03-05 | Openvpn Technologies, Inc. | Network address translation |
US10397283B2 (en) | 2015-07-15 | 2019-08-27 | Oracle International Corporation | Using symmetric and asymmetric flow response paths from an autonomous system |
US11252199B2 (en) | 2015-07-15 | 2022-02-15 | Oracle International Corporation | Redirecting packets in an autonomous system |
US10719341B2 (en) | 2015-12-02 | 2020-07-21 | Nicira, Inc. | Learning of tunnel endpoint selections |
EP4383657A3 (en) * | 2015-12-02 | 2024-09-18 | Nicira Inc. | Load balancing over multiple tunnel endpoints |
CA3014178C (en) * | 2016-02-10 | 2022-08-02 | Hughes Network Systems, Llc | System and method for policy-based multipath wan transports for improved quality of service over broadband networks |
KR20190020082A (en) * | 2016-06-18 | 2019-02-27 | 클레버넷 인코퍼레이티드 | An intelligent adaptive transport layer that uses multiple channels to improve performance |
US10511530B2 (en) * | 2016-12-13 | 2019-12-17 | Viasat, Inc. | Return-link routing in a hybrid network |
US10833891B2 (en) | 2017-02-11 | 2020-11-10 | Pismo Labs Technology Limited | Methods and systems for transmitting information packets through tunnel groups at a network node |
EP3454520B1 (en) * | 2017-09-12 | 2020-11-04 | Cisco Technology, Inc. | Virtual private networks without software requirements |
CN112073285B (en) * | 2019-06-10 | 2022-04-22 | 华为技术有限公司 | Error code notification method and related equipment |
US11528299B2 (en) | 2020-05-06 | 2022-12-13 | Hewlett Packard Enterprise Development Lp | Method device and system for policy based packet processing |
US20220385637A1 (en) * | 2021-05-27 | 2022-12-01 | Microsoft Technology Licensing, Llc | Selecting a vpn connection using negotiated cryptographic algorithms to improve throughput |
US20220385498A1 (en) * | 2021-06-01 | 2022-12-01 | Cisco Technology, Inc. | On-demand and scalable tunnel management in a multi-cloud and on-premises environment |
WO2023149185A1 (en) * | 2022-02-02 | 2023-08-10 | ソニーグループ株式会社 | Communication device, communication method, and communication system |
US11743191B1 (en) | 2022-07-25 | 2023-08-29 | Vmware, Inc. | Load balancing over tunnel endpoint groups |
Family Cites Families (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6484210B1 (en) * | 1997-11-10 | 2002-11-19 | General Instrument Corporation | Packet processing relay agent to provide link layer forwarding in one-way cable/wireless/satellite modems |
EP1052812A3 (en) * | 1999-05-12 | 2002-10-02 | Hughes Electronics Corporation | Technique for load balancing internet traffic over a hybrid asymmetric satelliteterrestrial network |
US7002918B1 (en) * | 2000-05-22 | 2006-02-21 | Northrop Grumman Corporation | Method and apparatus for real time scheduling in a satellite communications network |
US6850732B2 (en) * | 2001-03-30 | 2005-02-01 | Wengen Wireless Llc | Scalable satellite data communication system that provides incremental global broadband service using earth-fixed cells |
US8212719B2 (en) * | 2001-06-06 | 2012-07-03 | Global Locate, Inc. | Method and apparatus for background decoding of a satellite navigation message to maintain integrity of long term orbit information in a remote receiver |
US20040081199A1 (en) * | 2002-10-29 | 2004-04-29 | Lopez Ricardo Jorge | Multi-channel communication system and method based on class of service requirements |
ATE370601T1 (en) * | 2002-11-27 | 2007-09-15 | Research In Motion Ltd | DATA TRANSFER FROM A HOST SERVER VIA TUNNEL SERVER TO A WIRELESS DEVICE AND ASSIGNMENT OF A TEMPORARY IPV6 ADDRESS TO A TEMPORARY IPV4 ADDRESS FOR COMMUNICATION IN AN IPV4 WIRELESS NETWORK WITH THE DEVICE |
US7360243B2 (en) | 2003-10-02 | 2008-04-15 | Adria Comm Llc | Standard based firewall adapter for communication systems and method |
JP2004147344A (en) * | 2003-12-15 | 2004-05-20 | Toshiba Corp | Communication equipment |
US7477597B2 (en) * | 2004-09-08 | 2009-01-13 | Alcatel Lucent | Rural broadband hybrid satellite/terrestrial solution |
FR2884997A1 (en) * | 2005-04-25 | 2006-10-27 | Thomson Licensing Sa | Multi-line connection establishing method for digital data packet e.g. video data, communication, involves establishing communication tunnel between modem and gateway using connection and connected to remote network |
CN100411451C (en) * | 2006-02-28 | 2008-08-13 | 华为技术有限公司 | Method for configurating radio link parameter |
CN101689914A (en) * | 2006-09-26 | 2010-03-31 | 维尔塞特公司 | Improved spot beam satellite ground systems |
CN101573889B (en) * | 2006-10-03 | 2014-05-14 | 维尔塞特公司 | Web/bulk transfer preallocation of upstream resources in satellite communication system |
CN101188589B (en) * | 2006-11-15 | 2012-06-06 | 上海瑞高信息技术有限公司 | Synchronization adaptation method and device for broadcast data in satellite synchronization distribution network |
CN201114131Y (en) * | 2007-08-29 | 2008-09-10 | 南京中网通信有限公司 | Device for accomplishing satellite net and ground local area network interconnecting and communicating |
US7873060B2 (en) | 2008-10-18 | 2011-01-18 | Fortinet, Inc. | Accelerating data communication using tunnels |
CN101527968B (en) * | 2009-04-09 | 2011-01-05 | 北京航空航天大学 | Interaction method between space network and ground network, and communication protocol gateway |
US9204336B2 (en) * | 2010-08-17 | 2015-12-01 | Telefonaktiebolaget L M Ericsson (Publ) | Technique of processing network traffic that has been sent on a tunnel |
CN102724055A (en) * | 2011-07-28 | 2012-10-10 | 北京天地互连信息技术有限公司 | Mobile IPv6 protocol-based multi-interface mobile node address selection system |
US8761787B2 (en) * | 2011-10-04 | 2014-06-24 | At&T Intellectual Property I, L.P. | Methods, systems and apparatus to facilitate ranked network priority |
CN102420676B (en) * | 2011-11-30 | 2014-03-26 | 中国人民解放军西安通信学院 | High efficient interactive transmission method applicable to deep space interplanetary satellite network |
US8966131B2 (en) * | 2012-01-06 | 2015-02-24 | Qualcomm Incorporated | System method for bi-directional tunneling via user input back channel (UIBC) for wireless displays |
US9264972B2 (en) * | 2012-06-11 | 2016-02-16 | Qualcomm Incorporated | Home networking with integrated cellular communication |
US8432808B1 (en) * | 2012-06-15 | 2013-04-30 | Viasat Inc. | Opportunistically delayed delivery in a satellite network |
US9725476B2 (en) * | 2012-07-09 | 2017-08-08 | Universal Display Corporation | Silylated metal complexes |
EP2901766A2 (en) * | 2012-09-27 | 2015-08-05 | Interdigital Patent Holdings, Inc. | End-to-end architecture, api framework, discovery, and access in a virtualized network |
-
2014
- 2014-03-03 CN CN202011377073.4A patent/CN112383962B/en active Active
- 2014-03-03 WO PCT/IB2014/059378 patent/WO2015132624A1/en active Application Filing
- 2014-03-03 US US14/396,747 patent/US10291431B2/en active Active
- 2014-03-03 CN CN201480043845.7A patent/CN105723784B/en active Active
- 2014-03-03 CN CN202011192420.6A patent/CN112584520B/en active Active
- 2014-03-03 GB GB1418215.8A patent/GB2525693B/en active Active
-
2019
- 2019-05-13 US US16/411,057 patent/US20190268181A1/en not_active Abandoned
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3863238A1 (en) * | 2020-02-08 | 2021-08-11 | Ihor Yartsev | System, method and use of software-defined virtual multi-service tunnel networking |
WO2021156715A1 (en) * | 2020-02-08 | 2021-08-12 | Ihor Yartsev | System, method and use of software-defined virtual multi-service tunnel networking |
US20210250202A1 (en) * | 2020-02-08 | 2021-08-12 | Ihor YARTSEV | System, method and use of software-defined virtual multi-service tunnel networking |
Also Published As
Publication number | Publication date |
---|---|
CN112383962B (en) | 2023-12-19 |
GB201418215D0 (en) | 2014-11-26 |
CN105723784B (en) | 2021-01-12 |
US10291431B2 (en) | 2019-05-14 |
CN112584520A (en) | 2021-03-30 |
US20160020922A1 (en) | 2016-01-21 |
WO2015132624A1 (en) | 2015-09-11 |
GB2525693B (en) | 2016-10-19 |
CN112584520B (en) | 2023-07-04 |
GB2525693A (en) | 2015-11-04 |
CN105723784A (en) | 2016-06-29 |
CN112383962A (en) | 2021-02-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11582814B2 (en) | Methods and systems for transmitting and receiving data through one or more tunnels for packets satisfying one or more conditions | |
US20190268181A1 (en) | Methods and systems for transmitting and receiving data through tunnel groups | |
KR102578298B1 (en) | Traffic flow migration in backhaul networks | |
US10462828B2 (en) | Policy and billing services in a cloud-based access solution for enterprise deployments | |
US10749752B2 (en) | Methods and systems for managing VPN tunnels | |
US11824685B2 (en) | Method for implementing GRE tunnel, access point and gateway | |
US20140192634A1 (en) | System and Method for Network Failover and Network Selection with Multi-Mode Modem in Remote Access Points | |
US10015733B2 (en) | Methods and systems for transmitting and receiving data | |
CN113225770A (en) | Data transmission method and device | |
JP5441178B2 (en) | Cognitive radio communication system and cognitive radio communication relay device | |
WO2015178139A1 (en) | Radio communication system, user station, and communication control method | |
KR20160061089A (en) | Backhaul system and apparatus and control method applied to the same |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: PISMO LABS TECHNOLOGY LIMITED, HONG KONG Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SUNG, PATRICK HO WAI;NG, KAM CHIU;REEL/FRAME:052033/0617 Effective date: 20200226 |
|
AS | Assignment |
Owner name: PISMO LABS TECHNOLOGY LIMITED, HONG KONG Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LEUNG, WAN CHUN;REEL/FRAME:052033/0643 Effective date: 20200301 Owner name: PISMO LABS TECHNOLOGY LIMITED, HONG KONG Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CHAU, KIT WAI;REEL/FRAME:052033/0634 Effective date: 20200226 Owner name: PISMO LABS TECHNOLOGY LIMITED, HONG KONG Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KWAN, YING;REEL/FRAME:052033/0652 Effective date: 20200226 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |