WO2021061581A1 - Methods, systems, and computer readable media for providing a multi-tenant software-defined wide area network (sd-wan) node - Google Patents
Methods, systems, and computer readable media for providing a multi-tenant software-defined wide area network (sd-wan) node Download PDFInfo
- Publication number
- WO2021061581A1 WO2021061581A1 PCT/US2020/051882 US2020051882W WO2021061581A1 WO 2021061581 A1 WO2021061581 A1 WO 2021061581A1 US 2020051882 W US2020051882 W US 2020051882W WO 2021061581 A1 WO2021061581 A1 WO 2021061581A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- conduit
- tenant
- service provider
- network
- wan
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2854—Wide area networks, e.g. public data networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4633—Interconnection of networks using encapsulation techniques, e.g. tunneling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/28—Restricting access to network management systems or functions, e.g. using authorisation function to access network configuration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/302—Route determination based on requested QoS
- H04L45/306—Route determination based on the nature of the carried application
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/64—Routing or path finding of packets in data switching networks using an overlay routing layer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0272—Virtual private networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/06—Network architectures or network communication protocols for network security for supporting key management in a packet data network
- H04L63/062—Network architectures or network communication protocols for network security for supporting key management in a packet data network for key distribution, e.g. centrally by trusted party
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Definitions
- the subject matter described herein relates to communications networks. More particularly, the subject matter described herein relates to methods, systems, and computer readable media for providing a multi-tenant software-defined wide area network (SD-WAN) node.
- SD-WAN software-defined wide area network
- a wide area network can be used to connect multiple sites, offices, and/or local area networks (LANs) for networking purposes.
- an enterprise e.g., a business or university
- LANs local area networks
- an enterprise e.g., a business or university
- the enterprise may use WAN links (e.g., leased telecommunications circuits and/or other techniques for physical and/or virtual connections like packet switching or circuit switching connections) to connect the various locations within the WAN and to maintain or achieve a desired service level (e.g., security, bandwidth, and/or latency requirements).
- WANs may use encryption and various network protocols when sending traffic via WAN links and, as such, various network nodes may be needed to route, process, and/or facilitate WAN related communications.
- a software-defined WAN involves using software-defined networking (SDN) concepts to create and manage a WAN.
- SD-WAN may include a number of physical and/or virtual nodes or appliances that are programmable by a network controller for handling or facilitating WAN related communications.
- An SD-WAN may also utilize lower-cost and commercially available Internet access and related network equipment in lieu of more expensive WAN connection technologies and specialized equipment.
- SD-WANs can be useful for connecting multiple sites in an effective and cost-efficient manner, issues can arise when attempting to connect WAN users to networks or services, such as cloud services, controlled by others. For example, since many cloud service providers are serving multiple customers, it is impractical for the service provider to allow each enterprise to place an SD-WAN appliance in their network.
- service provider is by default (e.g., via Internet connections) providing its cloud services at a service level desirable to all WAN-based users.
- service providers lack an effective and cost efficient way to provide SD-WANs reliable access to cloud services and, as such, SD-WAN users may not receive cloud services at a service level that they are accustomed.
- One method occurs at a first network node in a service provider network for providing at least one service to multiple tenants.
- the method includes generating, using input from an administrator of the service provider network, user configuration information for a first tenant, wherein the user configuration information includes a security key for allowing the first tenant to configure aspects of the first network node and a total bandwidth limit associated with the first tenant; sending, to the first tenant, at least some of the user configuration information; receiving, from the first tenant, first configuration information for configuring a first conduit for tunneling communications between the service provider network and a first site associated with the first tenant’s SD-WAN; configuring, using the first configuration information, the first conduit for tunneling communications between the service provider network and the first site, wherein the first network node is associated with a plurality of conduits, wherein a second conduit of the plurality of conduits is at least in part configured by a second tenant and not the first tenant; and tunnel
- One system includes a first network node in a service provider network for providing at least one service to multiple tenants.
- the first network node includes at least one processor and memory.
- the first network node is configured for: generating, using input from an administrator of the service provider network, user configuration information for a first tenant, wherein the user configuration information includes a security key for allowing the first tenant to configure aspects of the first network node and a total bandwidth limit associated with the first tenant; sending, to the first tenant, at least some of the user configuration information; receiving, from the first tenant, first configuration information for configuring a first conduit for tunneling communications between the service provider network and a first site associated with the first tenant’s SD-WAN; configuring, using the first configuration information, the first conduit for tunneling communications between the service provider network and the first site, wherein the first network node is associated with a plurality of conduits, wherein a second conduit of the plurality of conduits is at least in part configured by a second tenant and not the first tenant; and tunneling, via the first conduit, communications
- the subject matter described herein can be implemented in software in combination with hardware and/or firmware.
- the subject matter described herein can be implemented in software executed by a processor.
- the subject matter described herein may be implemented using a computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps.
- Example computer readable media suitable for implementing the subject matter described herein include non-transitory devices, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits.
- a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.
- node refers to at least one physical computing platform including one or more processors and memory.
- functions or ‘module’ refer to software in combination with hardware and/or firmware for implementing features described herein.
- FIG 1 is a diagram illustrating an example communications environment utilizing a multi-tenant software-defined wide area network (SD- WAN) node
- Figure 2 is a diagram illustrating an example SD-WAN node for providing cloud services to adaptive private networks (APNs);
- SD- WAN software-defined wide area network
- Figure 3 is a diagram illustrating example APN configuration information usable for configuring aspects of an SD-WAN node
- FIGS. 4A-4C are diagrams illustrating graphical user interfaces (GUIs) for configuring aspects of an SD-WAN node
- Figure 5 is a diagram illustrating example actions for providing APN configuration information to an SD-WAN node
- Figures 6A-6B are diagrams illustrating example messages traversing a cloud conduit that connects an adaptive private network and a service provider network;
- Figure 7 is a diagram illustrating an example process for providing a multi-tenant SD-WAN node.
- an SD-WAN may be managed as a single administrative domain.
- nodes of the SD-WAN may be managed by a network administrator and that network administrator may have full control of the configuration parameters of the SD-WAN.
- issues can arise when attempting to provide such cloud services to SD-WAN users at a service level consistent with the SD-WAN.
- One possible solution for connecting SD-WAN users and cloud services may involve placing a node of the SD-WAN in a service provider network.
- This local SD-WAN node may then be used to provide an improved (e.g., highly reliable) network path to Internet-based applications or services hosted by the service provider.
- the service provider may not want to manage hundreds or thousands of SD-WAN nodes for their customers.
- the service provider may not want to allow an outside network administrators unfettered access to the service provider network so that they can configure their own SD-WAN node.
- the service provider would prefer to control at least some aspects of any local SD-WAN node in its network.
- a network node e.g., an SD-WAN node
- service gateway in accordance with at least some aspects described herein may be in a service provider network and provide at least one service to one or more tenants (e.g., administrative domains, enterprises, or related SD-WANs).
- tenants e.g., administrative domains, enterprises, or related SD-WANs.
- the network node or service gateway or aspects therein may be safely administered from several different administrative domains with no possibility that the different administrative domains will interfere with each other.
- the different administrative domains may be prevented from interfering with each other by security measures (e.g., unique preshared keys) and/or access designs (e.g., tenant-specific bandwidth limits, tenant-specific and/or site-specific conduits, etc.).
- security measures e.g., unique preshared keys
- access designs e.g., tenant-specific bandwidth limits, tenant-specific and/or site-specific conduits, etc.
- a network node or service gateway in accordance with at least some aspects described herein may be configured to allow each tenant to configure aspects (e.g., encryption and protocol settings) of one or more conduits (e.g., communications tunnels using multiple WAN links for connecting a service provider network to an SD-WAN site), while allowing the service provider to configure various other aspects, e.g., how many sites each tenant can connect to the service provider network and a total bandwidth for each tenant).
- the configuration for each tenant may be performed independently and may occur at various times (e.g., configuration is performed without requiring coordination between any of the tenants).
- an SD-WAN node may send and receive traffic via multiple WAN links associated with different tenants, where each of the tenants maintain their own time synchronization mechanism for monitoring the state of their WAN links.
- a network node or service gateway in accordance with at least some aspects described herein may maintain and/or utilize a separate time synchronization mechanism (e.g., a master clock) for conduits associated with the network node or service gateway.
- a network node or service gateway in accordance with at least some aspects described herein may maintain and/or utilize a separate time synchronization mechanism for each conduit associated with the network node or service gateway.
- a network node or service gateway in accordance with at least some aspects described herein may maintain or store WAN link identifiers associated with each tenant along with unique tenant and/or site identifiers.
- the network node or service gateway may store the information using a data structure (e.g., a hash table that uses the tenant and/or site identifiers as input and corresponding WAN link identifiers as output) such that the tenant and/or site identifiers can identify corresponding WAN link identifiers, particularly when the WAN link identifiers are not unique on their own.
- a data structure e.g., a hash table that uses the tenant and/or site identifiers as input and corresponding WAN link identifiers as output
- a cloud conduit between an SDN-WAN node in a service provider network and a client appliance in an SDN-WAN site of a tenant may be established.
- a heartbeat process may be used to determine when the cloud conduit is no longer needed.
- the client appliance may periodical poll the SD-WAN node in the service provider network using connection status requests.
- the SD-WAN node may reply to the connection status requests and may track any missed connection status requests from the client appliance.
- the SD-WAN node may assume that the client appliance is either non-operational or no longer has a cloud conduit connection active. In response to determining the cloud conduit is no longer needed, the SD-WAN node may automatically clean up the resources used for the connection to the client appliance so that the resources are available for use by other client appliances.
- a multi-tenant SD-WAN node that can interact with or be a part of separate, independent SD-WANs
- service providers are able to provide cloud services (e.g., Internet-based applications, cloud-based storage, etc.) to SD-WAN users more reliably and in an effective and cost-efficient manner.
- a multi-tenant SD-WAN node e.g., an SD-WAN service gateway in a service provider network
- service providers can efficiently use resources since the SD-WAN node can detect inactivity of dynamic cloud conduits and can free up related resources associated with any inactive cloud conduit, thereby allowing the resources to be reallocated to another cloud conduit and/or tenant.
- FIG. 1 is a diagram illustrating an example communications environment 100 utilizing an SD-WAN node.
- communications environment 100 may include an adaptive private network (APN) 101 and a service provider network 102.
- APN 101 may represent various network nodes, equipment, and user devices associated with an administrative domain.
- APN 101 may also be referred to a SD-WAN and may use SD-WAN technology, WAN links, and related equipment to connect the sites.
- APN 101 may represent an enterprise network comprising multiple sites, e.g., site 110 and site 112, communicatively connected via SD-WAN nodes or appliances.
- the SD-WAN nodes or appliances may be configured to use WAN links and related equipment for tunneling communications between sites using encryption and encapsulation techniques.
- Each of site 110 and site 112 may represent a LAN, a sub-network, or a physical location (e.g., an office building) associated with APN 101, where each site may include various devices, equipment, and/or appliances.
- site 110 may include a network controller (NC) 113 and data storage 118 and site 112 may include CA 116 and data storage 120.
- NC network controller
- a site e.g., site 110
- NC 114 may include functionality similar to CA 116 and may facilitate cloud conduit connections between site 110 and other sites of APN 101 and/or cloud conduit connections between site 110 and service provider network 102.
- APN 101 may have an active network controller (e.g., NC 114) and a standby network controller that can become active when a failover occurs.
- the active network controller may be at the same site as the standby network controller or at a different site.
- NC 114 may represent any suitable entity or entities (e.g., software executing on a processor, an FPGA, an ASIC, or a combination of software, an FPGA, and/or an ASIC) for performing one or more aspects associated with controlling or managing an SD-WAN (e.g., APN 101) and related network nodes, e.g., CA 116.
- NC 114 may be implemented using one or more processors and/or memory.
- NC 114 may utilize one or more processors (e.g., executing software stored in memory) for configuring CA 116.
- NC 114 may also utilize one or more processors to send instructions or network information to various modules or entities in APN 101 or service provider network 102.
- NC 114 may include or interact with a master clock controller, a time synchronization module, or related functionality.
- a time synchronization module may be responsible for time synchronization with APN and may send sync signals (e.g., time synchronization information) periodically to keep common clocks or slave clocks synchronized.
- CA 116 may be any suitable entity or entities (e.g., software executing on a processor, an FPGA, an ASIC, or a combination of software, an FPGA, and/or an ASIC) for connecting sites in APN 101.
- CA 116 may be configured to use one or more WAN conduits, e.g., logical entities, for tunneling communications between locations.
- each conduit may use WAN links (e.g., a group of IP addresses connected using commercially available Internet access, cellular networks, leased lines, and/or other connection technologies) and communications that traverse a conduit may use one or more network protocols (e.g., encapsulation protocols) and/or encryption techniques.
- CA 116 may use network management protocols, link aggregation techniques, and/or congestion avoidance mechanisms to reduce latency, increase bandwidth, and otherwise maintain one or more reliable connections between sites 110 and 112.
- CA 116 and/or SN 104 may monitor paths (e.g., WAN links) associated with a conduit to track state and latency of the paths before sending packets via a viable (e.g., low latency) path. In this example, if a packet is lost, a packet may be retransmitted it via a different path.
- monitor paths associated with a conduit may involve periodically sending (e.g., every 50 milliseconds) status update messages or other messages to SN 104 and using the information obtained from these messages to select an appropriate link for sending packets from one location to another location.
- Each of data storages 118 and 120 may represent any suitable entity (e.g., a computer readable medium or memory) for storing APN configuration information, CA configuration information, and/or other data.
- each of data storages 118 and 120 may store network information sent from NC 114 and may store state information associated with communications traversing a WAN conduit.
- Service provider network 102 may represent various network nodes, equipment, and/or devices associated with providing cloud services 108.
- Service provider network 102 may include a service provider site 103 for interfacing with cloud services 108, e.g., Internet-based applications, cloud- based storage, Internet-based collaboration tools, etc.
- service provider site 103 may include Internet-facing hosts and servers for receiving user requests for cloud services 108 and for responding to the user requests.
- service provider site 103 may communicate with cloud services 108 via various connection technologies and/or network equipment.
- cloud services 108 may include a separate enterprise network, data centers, and network equipment controlled by the service provider and may be connected to service provider site 103 and/or the Internet via multiple, redundant connection technologies.
- Cloud services 108 may represent any suitable entities (e.g., networks, nodes, devices, equipment, etc.) associated with providing or hosting Internet- based or third-party hosted applications or services.
- cloud services 108 may include a voice over Internet protocol (VoIP) service, software as a service (SAAS), or an Internet backhaul service.
- VoIP voice over Internet protocol
- SAAS software as a service
- Internet backhaul service e.g., user responses
- communications e.g., user responses
- Service provider site 103 may include an SD-WAN node (SN) 104 and data storage 106.
- SN 104 may be any suitable entity or entities (e.g., software and/or VMs executing on one or more processors in a computing platform) for connecting APN 101 and service provider network 102.
- Data storage 106 may represent any suitable entity (e.g., a computer readable medium or memory) for storing APN configuration information, SN configuration information, and/or other data.
- data storage 106 may store configuration information provided by tenants and by the service provider and may also store state information associated with communications traversing a cloud conduit, e.g., between service provider site 103 and site 110 or 112.
- SN 104 may represent a multi-tenant SD-WAN node and may be configurable for providing cloud services 108 to APNs or sites thereof controlled by different tenants (e.g., administrative domains) by utilizing configurable cloud conduits.
- SN 104 may be configured to use one or more cloud conduits (e.g., IP addresses of nodes in different networks or site connected via the Internet) for tunneling communications between APN 101 and service provider network 102 using encryption and/or packet encapsulation.
- cloud conduits e.g., IP addresses of nodes in different networks or site connected via the Internet
- each cloud conduit may use or comprise physical and/or virtual links (e.g., a group of IP addresses connected using commercially available Internet access, cellular networks, leased lines, and/or other connection technologies) and may be associated with various network protocols and/or encryption settings.
- endpoints e.g., CA 116 and SN 104
- packet conversion and/or modification may obfuscate a conduit end user’s identity, thereby making it difficult to discern the identity of a tenant by an outside entity (e.g., an Internet-based node along the cloud conduit path).
- SN 104 may use network management protocols, link aggregation techniques, and/or congestion avoidance mechanisms to reduce latency, increase bandwidth, and otherwise maintain one or more reliable connections between service provider network 102 and APN 101 or sites therein. For example, SN 104 may send network status update messages via a conduit or link thereof every 50 milliseconds and may use information obtained from these messages to select an appropriate link for tunneling communications between service provider network 102 and APN 101 or sites therein.
- SN 104 may be configured to provide at least one of cloud services 108 to one or more tenants.
- APN 101 may represent a network associated with a tenant (e.g., APN 101 or an administrator thereof) that is authorized by a service provider to configure aspects of SN 104 for establishing one or more cloud conduits between service provider network 102 and APN 101 or sites therein.
- SN 104 may be configured to allow other tenants (e.g., administrators of various other APNs) to administer aspects of SN 104 without interfering with APN 101.
- SN 104 prevents such interference by using security measures (e.g., each tenant or related APN has a unique security key for authentication) and access, permission, and usage features (e.g., tenant bandwidth limits set by the service provider, tenant-specific and/or site- specific conduits, etc.) that keep tenants and their workloads independent.
- security measures e.g., each tenant or related APN has a unique security key for authentication
- permission, and usage features e.g., tenant bandwidth limits set by the service provider, tenant-specific and/or site- specific conduits, etc.
- tenant bandwidth limits set by the service provider, tenant-specific and/or site- specific conduits, etc.
- SN 104 may be configured to receive APN configuration information from tenants and to receive other configuration information from a service provider, e.g., using an API, a web-based interface, or user interface associated with SN 104.
- SN 104 may facilitate tenant related configuration of SN 104 by allowing each tenant to configure aspects (e.g., encryption and protocol settings) of one or more cloud conduits and to determine bandwidth allocations from one or more bandwidth capacities configured by the service provider.
- SN 104 may also facilitate service provider related configuration of SN 104 by allowing the service provider to configure other aspects, e.g., a security key for each tenant, how many sites each tenant can connect to service provider network 102 and a total bandwidth for each tenant, APN, and/or site).
- configuration information from various sources may be merged or otherwise shaped into a runtime configuration for SN 104 that satisfies the requirements of the tenants and the service provider.
- configuration parameters that are relevant to SN 104 may be automatically detected and/or provided to SN 104 via an API, e.g., a REST API.
- the API exposed by the service provider’s SD-WAN node may allow the updated configuration information to be received and merged into the configuration of SN 104, e.g., at runtime.
- SN 104 or a related entity may utilize one or more network protocol(s) for detailed measuring and monitoring of the network to ensure conduit traffic is delivered and/or received.
- network protocol(s) for detailed measuring and monitoring of the network to ensure conduit traffic is delivered and/or received.
- managing a cloud conduit may require high-resolution time synchronization and a regular exchange of messages about the status of paths and WAN links to identify and/or mitigate communication issues (e.g., latency or perceived latency).
- Such conduit management can present challenges because the notion of time can differ across APNs (e.g., SD-WANs) because each APN may use a separate time synchronization mechanism (e.g., a master clock).
- SN 104 may be configured to utilize a local time synchronization mechanism (e.g., a master clock in service provider network 102) for determining whether communications that traverse a cloud conduit are old or late.
- a local time synchronization mechanism e.g., a master clock in service provider network 102
- SN 104 or related entity may maintain and/or utilize a separate time synchronization mechanism for multiple cloud conduits.
- SN 104 or related entity may maintain and/or utilize a separate time synchronization mechanism for each conduit.
- Figure 1 is for illustrative purposes and that various nodes and/or modules, locations, and/or functionality described above in relation to Figure 1 may be changed, altered, added, or removed.
- FIG. 2 is a diagram illustrating an example SN 104 for providing cloud services to APNs.
- SN 104 may be in service provider network 102 and may provide APNs 200 reliable access to cloud services 108 via cloud conduits.
- APNs 200 may include APN ‘A’, APN B’ and APN O’. Each of APNs 200 may represent an SD-WAN network administered by a different tenant (e.g., enterprise or administrative domain).
- APN ‘A’ may include CA 212
- APN ‘B’ may include CA 214
- APN ⁇ ’ may include CA 216.
- Each of CAs 212- 216 may have functionality similar to CA 116 as described above.
- each of CAs 212-216 may communicate APN configuration information to SN 104 and may act as an endpoint for one or more cloud conduits between its respective APN (or site thereof) and service provider network 102.
- SN 104 may include a management portal 208 and an SD-WAN gateway (SDWG) 210.
- Management portal 208 may represent any suitable entity for providing GUIs or other means for configuring aspects of SN 104 or a related entity, e.g., SDWG 210.
- management portal 208 be a virtual appliance (e.g., a virtual machine (VM) or a virtual container) running on SN 104 (e.g., a computing platform) and may provide a web-based administration portal.
- management portal 208 may be associated with a private IP address and a port assigned by a service provider or a related administrator.
- SDWG 210 may represent any suitable entity for communicating user traffic via cloud conduits.
- SDWG 210 may be a virtual appliance (e.g., a VM or a virtual container) running on SN 104 (e.g., a computing platform) and may set up and maintain cloud conduits for one or more of APNs 200 using configuration information provided by management portal 208 or related data storage, e.g., data storage 106.
- configuration information is from an administrator of service provider network 102 and at least some other configuration is from a relevant tenant (e.g., an entity that administers or controls the APN and/or site connecting to service provider network 102 via a cloud conduit).
- some configuration information may be provided from a tenant or a related node (e.g., NC 114 or CA 116) to SN 104.
- Example configuration information provided to SN 104 may include network or site configuration information and settings and may be usable for setting up correctly configured conduits for reliably routing traffic between service provider network 102 and an APN or sites thereof.
- the process of providing configuration information to SN 104 may be referred to a ‘mini’ configuration and the information provided in this process may be referred to ‘mini’ configuration information.
- cloud conduits may be dynamic with regard to creation and removal. For example, assuming a ‘mini’ configuration process has been performed, CA 116 may detect when user traffic is destined for a service network provider 102 and may send a trigger message to SDWG 210 for establishing a dynamic cloud conduit. In this example, once the dynamic cloud conduit is established, user traffic may be sent via the cloud conduit. In this example, after determining the cloud conduit is no longer needed or used (e.g., no user traffic traversing the conduit is detected or no status requests are received from CA 116 within a predetermined amount of time), SDWG 210 may automatically clean up or deallocate the resources used for the connection to CA 116 so that the resources are available for (re-)use.
- SDWG 210 may automatically clean up or deallocate the resources used for the connection to CA 116 so that the resources are available for (re-)use.
- management portal 208 and SDWG 210 may each be associated with a unique private IP address and a port assigned by the service provider or a related administrator.
- NAT/router 204 may represent a device or appliance that uses a public IP address to receive SN related traffic and one or more ports to identify whether to route received SN related traffic to management portal 208 and SDWG 210.
- traffic from APN ‘A’ or a related administrator may include either a management port value or a gateway port value as a destination port in an encapsulation packet header.
- NAT/router 204 may route the packet to the private IP address and port associated with management portal 208 and, if NAT/router 204 determines that the encapsulation packet header includes a gateway port value as the destination port, NAT/router 204 may route the packet to the private IP address and port associated with SDWG 210.
- SDWG 210 may be associated with multiple interfaces and/or IP addresses and ports. For example, SDWG 210 may use at least one private IP address and at least one port for a southbound (SB) interface facing APNs 200 and may use at least one public IP address and at least one port for a northbound (NB) interface facing cloud services 108.
- SB southbound
- NB northbound
- Example message flows traversing SN 104 and elements therein are described below with regard to Figures 6A-6B.
- Figure 2 and its related description are for illustrative purposes and that SN 104 and/or various other entities in Figure 2 may include additional and/or different modules, components, or functionality.
- FIG. 3 is a diagram illustrating example APN configuration information 300 usable for configuring aspects of an SD-WAN node, e.g., SN 104 or SDWG 210.
- APN configuration information 300 may include data from a ’mini’ configuration process that involves sending some initial configuration information from a site node (e.g., NC 114 or CA 116) to SN 104.
- APN configuration information 300 may be provided to SN 104 and may be usable for setting up correctly configured conduits for reliably routing traffic between service provider network 102 and an APN or sites thereof.
- APN configuration information 300 may include a subset of network or site configuration information and may be communicated to SN 104 before a conduit is created or use for a given APN or site.
- APN configuration information 300 may be sent to SN 104 in a data format that is readable by SN 104 or modules therein.
- APN configuration information 300 may be provided in an XML data format.
- APN configuration information 300 may include cloud global properties, cloud service default information, IPSec properties, advanced properties, cloud service properties, one or more rules, class information, cloud server (e.g., SN) properties, virtual WAN link information associated with the cloud server, autopath group properties, site appliance (e.g., CA) properties, route information, additional cloud service properties, dynamic cloud conduit routing domain information associated with a cloud service, and/or additional virtual WAN link information associated with the cloud service.
- APN configuration information 300 provided to SN 104 may only include information that is recently changed or added. For example, if an APN has adjusted a network protocol used by its WAN links or has added additional encryption measures, APN configuration information 300 that is provided to SN 104 may include this changed or new information, while not providing unchanged configuration information.
- Line 1 of the example XML configuration file may include a ‘miniconfig’ XML data element indicating a name, revision number, and a timestamp for the XML configuration file.
- Lines 2-9 of the example XML configuration file indicate a ‘cloud_global_properties’ XML data element indicating an encryption mode parameter (line 3), an enhanced message authentication parameter (line 4), an enhanced message authentication type (line 5), an enhanced packet uniqueness parameter (line 6), an enhanced rekey enabled parameter (line 7), and a subscriber APN name parameter (line 8).
- Lines 10-21 of the example XML configuration file indicate a ‘virtual_wan_link_access’ XML data element indicating a WAN link name (line 11 ), a properties define parameter (line 12), an WAN link identifier parameter (line 13), an access type parameter (line 14), a WAN ingress physical rate limit parameter (line 15), a WAN egress physical rate limit parameter (line 16), a WAN ingress permitted rate limit parameter (line 17), a WAN egress permitted rate limit parameter (line 18), a maximum transmission unit in bytes parameter (line 19), a public IP address parameter (line 20), and an enable public IP learning feature parameter (line 21 ).
- APN configuration information 300 may include parameters and settings for adequately configuring SN 104 or SDWG 210 for handling various services via the conduits.
- FIGS 4A-4C are diagrams illustrating GUIs 400-404 for configuring aspects of an SD-WAN node, e.g., SN 104.
- GUIs 400- 404 may represent various pages provided by management portal 208 usable by a service provider or a related administrator to configure SN 104 or aspects thereof, e.g., SDWG 210.
- GUIs 400-404 may also allow a service provider to input various tenant-specific (e.g., subscriber- specific) information, e.g., tenant related bandwidth limits and a maximum number of tenant sites allowed.
- tenant-specific e.g., subscriber- specific
- an administrator of service provider network 102 may need to configure SN 104, add authorized subscribers (e.g., tenants), and add one or more authorized APNs that can utilize SN 104.
- service provider related configuration may be performed prior to a tenant or subscriber providing APN configuration information 300.
- GUI 400 represents a page for receiving input for configuring a service gateway, e.g., SN 104 or SDWG 210.
- GUI 400 may include groups of input fields associated with different configuration aspects and actions buttons, e.g., an ‘Add’ button to add or store inputted information and a ‘Cancel’ button to close or clear page (e.g., ignore inputted information).
- a ‘General’ group may include a service name input field for naming or identifying a particular SD-WAN node in service provider network 102 and a bandwidth capacity input field for indicating a total bandwidth limit associated with the SD-WAN node.
- the total bandwidth limit may be a value that is based on hardware or other considerations of the service provider and this bandwidth limit may be enforced such that the total bandwidth limit associated with the SD-WAN node is not exceed when bandwidths for all tenants of SN 104 are considered, e.g., combined.
- a ‘Management/REST API Interfaces’ group may include a public management IP address input field for inputting a public IP address for connecting to SDWG 210 , a management port input field for inputting a port for connecting to SDWG 210 , and a private management IP address input field for inputting a private IP address for connecting to SDWG 210 (e.g., usable by a NAT/router in service provider network 102).
- a ‘Southbound Conduit Interfaces’ group may include a public conduit IP address input field for inputting a public IP address for connecting to the SD-WAN node in service provider network 102 from an APN, a first conduit port input field for inputting a first port for connecting to the SD-WAN node in service provider network 102, a second conduit port input field for inputting a second port for connecting to the SD-WAN node in service provider network 102, a southbound private virtual IP (VIP) address input field for inputting a private VIP address for connecting to an APN-facing interface of SDWG 210 (e.g., usable by a NAT/router in service provider network 102), a southbound private gateway address input field for inputting a gateway address for reaching a router usable to resolve the southbound private VIP address or a related address range as indicated by an inputted mask value, and a mask input field for inputting a value for indicating an address range that can be assigned for the southbound conduit interfaces.
- VIP virtual IP
- a ‘Northbound Services Interfaces’ group may include a northbound VIP address input field for inputting a private VIP address for connecting to Internet-facing interface of SDWG 210 (e.g., usable by a NAT/router in service provider network 102), a northbound private gateway address input field for inputting a gateway address for reaching a router usable to resolve the northbound private VIP address or a related address range as indicated by an inputted mask value, and a mask input field for inputting a value for indicating an address range that can be assigned for the northbound service interface.
- SDWG 210 e.g., usable by a NAT/router in service provider network 102
- a northbound private gateway address input field for inputting a gateway address for reaching a router usable to resolve the northbound private VIP address or a related address range as indicated by an inputted mask value
- a mask input field for inputting a value for indicating an address range that can be assigned for the northbound service interface.
- GUI 402 represents a page for receiving input for configuring a subscriber, e.g., a tenant.
- GUI 402 may include multiple input fields associated with different configuration aspects and actions buttons, e.g., a ‘Create Subscriber’ button to add or store inputted information and a ‘Close’ button to close or clear page (e.g., ignore inputted information).
- GUI 402 may include a subscriber name input field for inputting a name or identifier for representing a particular subscriber, an email input field for inputting an email address associated with the particular subscriber, a password input field for inputting a password associated with the particular subscriber, a confirm password input field for inputting the password again to that the password inputted in the password input field is correct; and a bandwidth capacity input field for inputting a total bandwidth limit associated with the subscriber (e.g., the subscriber can manage bandwidth allocation among various sites and/or networks in management portal 208 but cannot exceed this subscriber bandwidth limit set by the service provider).
- a subscriber name input field for inputting a name or identifier for representing a particular subscriber
- an email input field for inputting an email address associated with the particular subscriber
- a password input field for inputting a password associated with the particular subscriber
- a confirm password input field for inputting the password again to that the password inputted in the password input field is correct
- a bandwidth capacity input field for inputting a total
- GUI 404 represents a page for receiving input for configuring a network, e.g., APN 101.
- GUI 404 may include multiple input fields associated with different configuration aspects and actions buttons, e.g., a ‘Create Network’ button to add or store inputted information and a ‘Close’ button to close or clear page (e.g., ignore inputted information).
- GUI 404 may include a network name input field for inputting a name or identifier for representing a particular network and a preshared key input field for inputting a preshared key usable for authentication (e.g., so that only an authorized subscriber can interact with and/or modify this network’s configuration settings for SN 104).
- GUI 404 may also include an ‘New PSK’ button for triggering the generation of a unique preshared key, e.g., by a web server, a security device, or another entity.
- GUI 404 may also include a per-site bandwidth limit for inputting a bandwidth limit associated with each site (e.g., a LAN, a sub-network, or a related location) associated with the APN and a per-network bandwidth limit for inputting a bandwidth limit associated with the APN.
- the per-network bandwidth limit cannot exceed the total subscriber capacity (e.g., inputted via GUI 402).
- GUI 404 may also provide feedback of remaining subscriber capacity based on the number of sites used and their related bandwidth limits inputted.
- GUI 404 may also include user interface elements for associating an APN to one or more service gateways, e.g., SD-WAN nodes in service provider network 102).
- GUI 404 may include actions buttons, e.g., a ‘Add Service’ button for selecting a service gateway to associate with the APN and a ‘Remove Service’ button for removing a service gateway from being associated with the APN.
- GUI 404 may provide a table or visual element for indicating maximum bandwidth and maximum connections allowed for each associated service gateway.
- Figures 4A-4C and their related description are for illustrative purposes and that additional and/or different user interface elements may be usable for inputting various information for facilitating configuring aspects of an SD-WAN node by a service provider or a related administrator.
- Figure 5 is a diagram illustrating example actions for providing APN configuration information to SN 104.
- APN configuration information may be stored in data file in one or more formats and may sent to SN 104 via a web GUI, an API, or other communications interface.
- SN 104 may use received APN configuration information for generating or establishing one or more cloud conduits between SN 104 and the related APN.
- NC 114 may send APN configuration information 300 to CA 116.
- NC 114 may have access to most, if not all, relevant APN configuration information 300 and may provide this information to CA 116 periodically (e.g., every 10 minutes), dynamically (e.g., when an SD-WAN change is detected), or on request.
- CA 116 may receive APN configuration information 300 and may, if needed, modify APN configuration information 300 or generated additional configuration information.
- CA 116 may generate or modify site-specific configuration information and add this information to APN configuration information 300 obtained from NC 114.
- CA 116 may send APN configuration information 300 to SN 104.
- CA 116 may utilize a REST API or another mechanism for uploading an APN configuration file to SN 104.
- SN 104 may receive APN configuration information 300 and may parse and store APN configuration information 300 for future use. For example, SN 104 may store various APN configuration settings in a data structure such that relevant APN configuration settings are retrievable when creating a conduit that connects to that APN or a related site.
- Figure 5 is for illustrative purposes and that different and/or additional messages, steps, and/or actions may be used.
- another entity e.g., network controller or an administrator using web-based interface
- various messages, steps, and/or actions described herein may occur in a different order or sequence.
- Figures 6A-6B are diagrams illustrating example messages traversing a cloud conduit that connects an APN and a service provider network.
- Figure 6A depicts an egress message flow from a host 600 in APN ‘A’ to SDWG 210 and Figure 6B depicts an ingress message flow from SDWG 210 to host 600.
- the cloud conduit may utilize one or more WAN links (e.g., connections between public IP addresses) that allow traffic to be tunneled between an APN (or related site) and a service provider network.
- conduits may transmit packets between public IP addresses of CA 212 (or a related device) in APN ‘A’ and NAT/router 204 in service provider network 102.
- the cloud conduit has inherent reliability even if a portion of the WAN links go down or have communications issues.
- the cloud conduit may be configured to send communications using a set of encryption and/or protocol settings that were configured by the APN or a related administrator.
- the cloud conduit may be dynamic in nature, e.g., the cloud conduit may be set up after a message destined for the service provider network is detected, e.g., by CA 212. In such embodiments, this message may need to be queued until the establishment of the cloud conduit is complete.
- SDWG 210 or another entity of SN 104 may learn or derive public IP addresses of WAN links associated with various APNs or sites thereof.
- SDWG 210 may receive encrypted packets via a cloud conduit connecting SDWG 210 and CA 212, but SDWG 210 may be unaware of a public IP address associated with CA 212.
- SDWG 210 may attempt packet decryption using different site keys (e.g., previously generated by the service provider and stored in data storage 106) until the packet decryption is successful and, after successful decryption, the source IP address and source port in the packet header are visible.
- SDWG 210 or another entity of SN 104 may associate the learned source IP address and source port with various tenant related identifiers.
- SDWG 210 may maintain routing table information using a hash based data structure.
- a hash key may be based on a (public or external) source IP address and source port and a corresponding hash entry is based on an APN identifier, a WAN link identifier, and a site identifier.
- SDWG 210 can identify relevant WAN links associated with a site or conduit even when multiple tenants use a same WAN link identifier because the combination of the WAN link identifier, a related APN identifier, and a related site identifier will be unique.
- a request packet (e.g., Internet control management protocol echo request packet) may be sent from host 600 in an APN ‘A’ and may include a header indicating an IP address associated with host 600 as the source address and an IP address associated with an Internet-based service as the destination address.
- the request message may be destined for an Internet service provider, e.g., an Office 365 web-based application.
- CA 212 may receive the request packet and may process the request packet for transmission via the cloud conduit.
- CA 212 may process the request packet by changing a source address in the request packet header to a WAN link IP address and port associated with CA 212 and may encapsulate the request packet by adding an encapsulation header and/or encrypting the request packet, e.g., as a payload in the encapsulated packet.
- the source IP address and/or other parameters in a request packet header may be changed or hidden from service provider and other tenants for privacy purposes.
- obfuscation may be utilize for making the identity of an endpoint sending a packet that traverses a cloud conduit should difficult to discern by the service provider or another entity.
- the encapsulated packet may include an encapsulation header (e.g., a Talari reliable protocol (TRP) header) indicating the WAN link IP address as the source address, an IP address associated with NAT/router 204 as the destination address, and/or a conduit port as the source and destination port .
- TRP Talari reliable protocol
- the encapsulated packet may be sent to NAT/router 204 associated with SN 104.
- NAT/router 204 may receive the encapsulated packet and inspect its header for determining whether to route the packet to SN 104. For example, if the encapsulated packet header indicates a conduit port as the destination port, NAT/router 204 may change the destination address and/or destination port in the encapsulated packet header for directing the encapsulated packet to SDWG 210.
- NAT/router 204 may store state information about the encapsulated packet and/or inner request packet for identifying an appropriate conduit and/or destination in APN ‘A’ when a corresponding response packet is sent back.
- NAT/router 204 may use a data structure (e.g., a hash table or a dictionary) that maps a data tuple (e.g., a source IP address and a source port).
- step 605 the encapsulated packet may be sent to SDWG 210.
- SDWG 210 may receive the encapsulated packet and may decapsulate the encapsulated packet, e.g., by stripping the encapsulation header. SDWG 210 may also modify the source address associated with the request packet from a WAN link IP address associated with CA 212 to a ‘northbound’ IP address associated with NAT/router 202 for routing traffic destined for cloud services 108.
- NAT/router 202 may store state information about the request packet for identifying an appropriate conduit and/or destination in APN ‘A’ when a corresponding response packet is sent back.
- NAT/router 204 may use a data structure that maps a data tuple (e.g., a source IP address and a source port).
- NAT/router 202 may forward or send the request packet toward its destination, e.g., IP address ‘8.8.8.8’.
- a service node in service provider network 102 may process the request packet, and a corresponding response may be generated and sent back to NAT/router 202 associated with SN 104.
- NAT/router 202 may receive the response packet and send or forward the response packet to SDWG 210.
- SDWG 210 may receive a response packet corresponding to the request packet and may process the response packet for transmission via the cloud conduit.
- SDWG 210 may process the response packet by changing a destination address in the response packet header to an IP address associated with NAT/router 204 and may encapsulate the response packet by adding an encapsulation header and/or encrypting the response packet, e.g., as a payload in the encapsulated packet.
- the encapsulated packet may include an encapsulation header (e.g., a TRP header) indicating a private IP address associated with SDWG 210 as the source address, a ‘southbound’ IP address associated with NAT/router 204 as the destination address, a conduit port as the source and destination port.
- the encapsulated packet may be sent to NAT/router 204 associated with SN 104.
- NAT/router 204 may receive the encapsulated packet and inspect its header for determining which conduit to use for routing the encapsulated packet. For example, if the encapsulated packet header indicates the conduit to CA 212 (e.g., based on one or more identifiers or a hash of various values), NAT/router 204 may change the destination address to a WAN link IP address and/or a destination port associated with CA 212 in the encapsulated packet header for directing the encapsulated packet to CA 212. In some embodiments, NAT/router 204 may change the source address in the encapsulated packet header to a public IP address associated with NAT/router 204.
- the encapsulated packet may be sent to CA 212.
- CA 212 may receive the encapsulated packet and may decapsulate the encapsulated packet, e.g., by stripping the encapsulation header. CA 212 may also modify the destination address associated with the response packet from the WAN link IP address associated with CA 212 to the IP address associated with host 600.
- CA 212 may send the response packet to host 600.
- Figure 6 is for illustrative purposes and that different and/or additional messages, steps, and/or actions may be used. It will also be appreciated that various messages, steps, and/or actions described herein may occur in a different order or sequence.
- Figure 7 is a diagram illustrating an example process 700 for providing a multi-tenant SD-WAN node.
- example process 700 described herein, or portions thereof may be performed at or performed by a network node (e.g., SN 104), SDWG 210, and/or another module or node.
- a network node e.g., SN 104
- SDWG 210 e.g., SDWG 210
- another module or node e.g., another module or node.
- user configuration information for a first tenant may be generated using input from an administrator of the service provider network.
- user configuration information may include a security key for allowing the first tenant to configure aspects of the first network node and a total bandwidth limit associated with the first tenant.
- at least some of the user configuration information may be sent to the first tenant.
- user configuration information may include a preshared key, a service bandwidth limit, a per-conduit bandwidth limit, a per-site bandwidth limit, and/or a per-network bandwidth limit.
- the user configuration information may be configured by a service provider or a related administrator and the user configuration information may be provided via an API or a user interface.
- step 706 first configuration information for configuring a first conduit for tunneling communications between the service provider network and a first site associated with the first tenant’s SD-WAN may be received from the first tenant.
- CA 116 or NC 114 may send a message containing a ‘mini’ configuration file to SN 104 via a REST API and/or a web-based management portal 208.
- the message may be directed to an IP address associated with SN 104 and may include a preshared key provided to CA 116 or NC 114 by SN 104.
- the first conduit for tunneling communications between the service provider network and the first site may be configured using the first configuration information (e.g., ‘mini’ configuration information), wherein the first network node is associated with a plurality of conduits, wherein a second conduit of the plurality of conduits is at least in part configured by a second tenant and not the first tenant.
- first configuration information e.g., ‘mini’ configuration information
- the first conduit may be configured after receiving the first configuration information.
- the first conduit may be established in response to a trigger message.
- a trigger message may be a message for requesting at least one service or a message destined for a service provider network.
- a trigger message may include any message directed to service provider network 102 or any message for requesting one or more of cloud services 108 and/or destined for service provider network 102.
- the first conduit may be removed (e.g., related allocated resources at SN 104 freed) when inactivity for a predetermined amount of time is detected.
- communications between the service provider network and the first site may be tunneled via the first conduit.
- communications traversing a first conduit may be encapsulated using a first set of protocol and/or encryption settings and communications traversing a second conduit may be encapsulated using a first set of protocol and/or encryption settings, wherein the first set of protocol and/or encryption settings is different from the second set of protocol and/or encryption settings.
- conduit status information and time synchronization information associated with a first tenant may be communicated via a first conduit using a representational state transfer application programming interface (REST API) and/or an encapsulation protocol.
- a first conduit may use an advanced encryption standard (AES) encrypted tunneling protocol (e.g., a proprietary conduit protocol) with an encapsulating security payload (ESP) tunnel type and 32-bit secure hash algorithm (SHA) based hash value and a second conduit may use a different tunneling protocol with an authentication header (AH) tunnel type and 64-bit SHA based hash value.
- AES advanced encryption standard
- ESP encapsulating security payload
- SHA secure hash algorithm
- AH authentication header
- a tenant may represent or include an entity (e.g., an enterprise or company), a network (e.g., APN 101) or site (e.g., site 110) associated with the entity, or a device, node, or appliance associated with the entity.
- a first tenant may include CA 116, NC 114 or a network administrator associated with APN 101.
- a first network node may include a gateway and a controller implemented using one or more virtual machines or virtual containers.
- SN 104 may include a SDWG 210 and management portal 208, where SDWG 210 and management portal 208 are Linux based virtual machines.
- a first conduit may be associated with a first time manager implemented at a first network node or a node in a service provider network, wherein the first time manager is separate from a second time manager implemented at a network controller or a node in a first tenant’s SD-WAN.
- SN 104 may utilize a master clock or other time synchronization mechanism for communications via one or more cloud conduits associated with service provider network 102 (e.g., service provider site 103).
- NC 114 may provide or utilize another master clock or other time synchronization mechanism for communications within site 110 and/or site 112.
- process 700 may further include receiving, from a first tenant, second configuration information (e.g., ‘mini’ configuration information) for configuring a third conduit for tunneling communications between a service provider network and a second site associated with the first tenant’s SD-WAN; configuring, using the second configuration information, the third conduit for tunneling communications between the service provider network and the second site associated with the first tenant’s SD-WAN, wherein a combined bandwidth of a first conduit and the third conduit does not exceed a total bandwidth limit associated with the first tenant; and tunneling, via the third conduit, communications between the service provider network and the second site associated with the first tenant’s SD-WAN.
- second configuration information e.g., ‘mini’ configuration information
- process 700 may further include learning a public IP address associated with a first site (e.g., site 112), wherein learning the public IP address includes receiving an encrypted packet via a first conduit, decrypting the encrypted packet using stored preshared keys associated with different sites until the encrypted packet is decrypted successfully, and identifying the public IP address from a decrypted packet header.
- a first site e.g., site 112
- learning the public IP address includes receiving an encrypted packet via a first conduit, decrypting the encrypted packet using stored preshared keys associated with different sites until the encrypted packet is decrypted successfully, and identifying the public IP address from a decrypted packet header.
- SN 104 or SDGW 210 may learn an IP address associated with CA 116 by attempting to decrypt packets using different preshared keys associated with various APNs or tenants.
- SN 104 or SDGW 210 may associate the IP source address and the source port located in a packet header of the packet with a related APN identifier, a WAN link identifier, and a site identifier.
- process 700 may further include maintaining statistics and/or traffic monitoring information associated with each conduit, site, and/or tenant.
- Figure 7 is for illustrative purposes and that different and/or additional steps and/or actions may be used. It will also be appreciated that various steps and/or actions described herein may occur in a different order or sequence.
- a network node, SN 104, SDWG 210, and/or functionality described herein may constitute a special purpose computing device. Further, a network node, SN 104, SDWG 210, and/or functionality described herein can improve the technological field of communications networks and SD-WAN connectivity. For example, by providing a SN 104 in a service provider network capable of facilitating conduits configured by multiple tenants, SD-WAN users can receive cloud services in a reliable manner and/or at a reliable service level. Further, by using a multi-tenant SD-WAN node, a service providers can provide cloud services to SD-WAN users associated with multiple tenants more reliably and in an effective and cost- efficient manner.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP20786379.6A EP4035339B1 (en) | 2019-09-27 | 2020-09-21 | Methods, systems, and computer readable media for providing a multi-tenant software-defined wide area network (sd-wan) node |
| CN202080063378.XA CN114402574B (zh) | 2019-09-27 | 2020-09-21 | 用于提供多租户软件定义的广域网(sd-wan)节点的方法、系统和计算机可读介质 |
| JP2022519341A JP7641276B2 (ja) | 2019-09-27 | 2020-09-21 | マルチテナントソフトウェア定義ワイドエリアネットワーク(sd-wan)ノードを提供するための方法、システム、およびコンピュータ読取可能媒体 |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/586,300 US11082304B2 (en) | 2019-09-27 | 2019-09-27 | Methods, systems, and computer readable media for providing a multi-tenant software-defined wide area network (SD-WAN) node |
| US16/586,300 | 2019-09-27 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021061581A1 true WO2021061581A1 (en) | 2021-04-01 |
Family
ID=72752518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2020/051882 Ceased WO2021061581A1 (en) | 2019-09-27 | 2020-09-21 | Methods, systems, and computer readable media for providing a multi-tenant software-defined wide area network (sd-wan) node |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US11082304B2 (https=) |
| EP (1) | EP4035339B1 (https=) |
| JP (1) | JP7641276B2 (https=) |
| CN (1) | CN114402574B (https=) |
| WO (1) | WO2021061581A1 (https=) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11483228B2 (en) | 2021-01-29 | 2022-10-25 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for network testing using an emulated data center environment |
| US11575605B2 (en) | 2016-01-19 | 2023-02-07 | Talari Networks Incorporated | Adaptive private network (APN) bandwidth enhancements |
| US11716283B2 (en) | 2021-03-05 | 2023-08-01 | Oracle International Corporation | Methods, systems, and computer readable media for selecting a software defined wide area network (SD-WAN) link using network slice information |
| US11799793B2 (en) | 2012-12-19 | 2023-10-24 | Talari Networks Incorporated | Adaptive private network with dynamic conduit process |
Families Citing this family (63)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10749711B2 (en) | 2013-07-10 | 2020-08-18 | Nicira, Inc. | Network-link method useful for a last-mile connectivity in an edge-gateway multipath system |
| US10498652B2 (en) | 2015-04-13 | 2019-12-03 | Nicira, Inc. | Method and system of application-aware routing with crowdsourcing |
| US10135789B2 (en) | 2015-04-13 | 2018-11-20 | Nicira, Inc. | Method and system of establishing a virtual private network in a cloud service for branch networking |
| US20180219765A1 (en) | 2017-01-31 | 2018-08-02 | Waltz Networks | Method and Apparatus for Network Traffic Control Optimization |
| US11706127B2 (en) | 2017-01-31 | 2023-07-18 | Vmware, Inc. | High performance software-defined core network |
| US10778528B2 (en) | 2017-02-11 | 2020-09-15 | Nicira, Inc. | Method and system of connecting to a multipath hub in a cluster |
| US10523539B2 (en) | 2017-06-22 | 2019-12-31 | Nicira, Inc. | Method and system of resiliency in cloud-delivered SD-WAN |
| US11115480B2 (en) | 2017-10-02 | 2021-09-07 | Vmware, Inc. | Layer four optimization for a virtual network defined over public cloud |
| US10594516B2 (en) | 2017-10-02 | 2020-03-17 | Vmware, Inc. | Virtual network provider |
| US10999100B2 (en) | 2017-10-02 | 2021-05-04 | Vmware, Inc. | Identifying multiple nodes in a virtual network defined over a set of public clouds to connect to an external SAAS provider |
| US11223514B2 (en) | 2017-11-09 | 2022-01-11 | Nicira, Inc. | Method and system of a dynamic high-availability mode based on current wide area network connectivity |
| US12495019B1 (en) * | 2019-01-30 | 2025-12-09 | R&D Industries, Inc. | Apparatus, systems and methods for multi-carrier and multi-tenant end-to-end private wide area network |
| US11310170B2 (en) | 2019-08-27 | 2022-04-19 | Vmware, Inc. | Configuring edge nodes outside of public clouds to use routes defined through the public clouds |
| US11489783B2 (en) | 2019-12-12 | 2022-11-01 | Vmware, Inc. | Performing deep packet inspection in a software defined wide area network |
| US11722925B2 (en) | 2020-01-24 | 2023-08-08 | Vmware, Inc. | Performing service class aware load balancing to distribute packets of a flow among multiple network links |
| US11323918B2 (en) * | 2020-01-24 | 2022-05-03 | Cisco Technology, Inc. | Switch and backhaul capacity-based radio resource management |
| US11329883B2 (en) * | 2020-03-12 | 2022-05-10 | Fortinet, Inc. | Dynamic establishment of application-specific network tunnels between network devices by an SDWAN controller |
| WO2022005912A1 (en) | 2020-06-29 | 2022-01-06 | Illumina, Inc. | Policy-based genomic data sharing for software-as-a-service tenants |
| CA3177396A1 (en) * | 2020-06-29 | 2022-01-06 | Prabhu PALANISAMY | Temporary cloud provider credentials via secure discovery framework |
| US11245641B2 (en) | 2020-07-02 | 2022-02-08 | Vmware, Inc. | Methods and apparatus for application aware hub clustering techniques for a hyper scale SD-WAN |
| US12615239B1 (en) * | 2020-08-24 | 2026-04-28 | Graphiant, Inc. | Multi-tenant network service architecture |
| US11575591B2 (en) | 2020-11-17 | 2023-02-07 | Vmware, Inc. | Autonomous distributed forwarding plane traceability based anomaly detection in application traffic for hyper-scale SD-WAN |
| US11575600B2 (en) | 2020-11-24 | 2023-02-07 | Vmware, Inc. | Tunnel-less SD-WAN |
| US11929903B2 (en) | 2020-12-29 | 2024-03-12 | VMware LLC | Emulating packet flows to assess network links for SD-WAN |
| CN116783874A (zh) * | 2021-01-18 | 2023-09-19 | Vm维尔股份有限公司 | 网络感知的负载平衡 |
| US12218845B2 (en) | 2021-01-18 | 2025-02-04 | VMware LLC | Network-aware load balancing |
| US11979325B2 (en) | 2021-01-28 | 2024-05-07 | VMware LLC | Dynamic SD-WAN hub cluster scaling with machine learning |
| US12368676B2 (en) | 2021-04-29 | 2025-07-22 | VMware LLC | Methods for micro-segmentation in SD-WAN for virtual networks |
| US12009987B2 (en) | 2021-05-03 | 2024-06-11 | VMware LLC | Methods to support dynamic transit paths through hub clustering across branches in SD-WAN |
| US11729065B2 (en) | 2021-05-06 | 2023-08-15 | Vmware, Inc. | Methods for application defined virtual network service among multiple transport in SD-WAN |
| US12250114B2 (en) | 2021-06-18 | 2025-03-11 | VMware LLC | Method and apparatus for deploying tenant deployable elements across public clouds based on harvested performance metrics of sub-types of resource elements in the public clouds |
| US12015536B2 (en) | 2021-06-18 | 2024-06-18 | VMware LLC | Method and apparatus for deploying tenant deployable elements across public clouds based on harvested performance metrics of types of resource elements in the public clouds |
| US12047282B2 (en) | 2021-07-22 | 2024-07-23 | VMware LLC | Methods for smart bandwidth aggregation based dynamic overlay selection among preferred exits in SD-WAN |
| US12267364B2 (en) | 2021-07-24 | 2025-04-01 | VMware LLC | Network management services in a virtual network |
| US11943146B2 (en) | 2021-10-01 | 2024-03-26 | VMware LLC | Traffic prioritization in SD-WAN |
| US11695690B1 (en) * | 2021-11-08 | 2023-07-04 | Graphiant, Inc. | Network address translation with in-band return path resolution |
| US12603848B2 (en) | 2022-01-04 | 2026-04-14 | VMware LLC | Efficient mechanism for the transmission of multipath duplicate packets |
| US12184557B2 (en) | 2022-01-04 | 2024-12-31 | VMware LLC | Explicit congestion notification in a virtual environment |
| US12507120B2 (en) | 2022-01-12 | 2025-12-23 | Velocloud Networks, Llc | Heterogeneous hub clustering and application policy based automatic node selection for network of clouds |
| US12425395B2 (en) | 2022-01-15 | 2025-09-23 | VMware LLC | Method and system of securely adding an edge device operating in a public network to an SD-WAN |
| US12506678B2 (en) | 2022-01-25 | 2025-12-23 | VMware LLC | Providing DNS service in an SD-WAN |
| US11909815B2 (en) | 2022-06-06 | 2024-02-20 | VMware LLC | Routing based on geolocation costs |
| CN115664969A (zh) * | 2022-06-13 | 2023-01-31 | 深圳市高德信通信股份有限公司 | 一种sd-wan系统、sd-wan系统的使用方法及装置 |
| US20240022626A1 (en) | 2022-07-18 | 2024-01-18 | Vmware, Inc. | Dns-based gslb-aware sd-wan for low latency saas applications |
| US20240028378A1 (en) | 2022-07-20 | 2024-01-25 | Vmware, Inc. | Method for modifying an sd-wan using metric-based heat maps |
| US20240073743A1 (en) | 2022-08-28 | 2024-02-29 | Vmware, Inc. | Dynamic use of multiple wireless network links to connect a vehicle to an sd-wan |
| US20240073126A1 (en) * | 2022-08-29 | 2024-02-29 | Vmware, Inc. | Seamless failover for private mobile networks |
| WO2024104168A1 (zh) * | 2022-11-16 | 2024-05-23 | 华为云计算技术有限公司 | 跨区域的虚拟私有云之间通信的配置方法及相关装置 |
| US20240323076A1 (en) * | 2022-12-30 | 2024-09-26 | Solutions Humanitas Inc. | Stem-like network node |
| US12034587B1 (en) | 2023-03-27 | 2024-07-09 | VMware LLC | Identifying and remediating anomalies in a self-healing network |
| US12425332B2 (en) | 2023-03-27 | 2025-09-23 | VMware LLC | Remediating anomalies in a self-healing network |
| US12057993B1 (en) | 2023-03-27 | 2024-08-06 | VMware LLC | Identifying and remediating anomalies in a self-healing network |
| US12438698B2 (en) | 2023-07-27 | 2025-10-07 | Cisco Technology, Inc. | Managing encryption keys of secure tunnels in multi-tenant edge devices |
| US12355655B2 (en) | 2023-08-16 | 2025-07-08 | VMware LLC | Forwarding packets in multi-regional large scale deployments with distributed gateways |
| US12261777B2 (en) | 2023-08-16 | 2025-03-25 | VMware LLC | Forwarding packets in multi-regional large scale deployments with distributed gateways |
| US12483968B2 (en) | 2023-08-16 | 2025-11-25 | Velocloud Networks, Llc | Distributed gateways for multi-regional large scale deployments |
| US12587468B2 (en) | 2023-08-16 | 2026-03-24 | Velocloud Networks, Llc | Route filtering for clusters in multi-regional large scale deployments with distributed gateways |
| US12507148B2 (en) | 2023-08-16 | 2025-12-23 | Velocloud Networks, Llc | Interconnecting clusters in multi-regional large scale deployments with distributed gateways |
| US12603827B2 (en) | 2023-08-16 | 2026-04-14 | Velocloud Networks, Llc | Asymmetric routing resolutions in multi-regional large scale deployments with distributed gateways |
| US12563438B2 (en) | 2023-08-16 | 2026-02-24 | Velocloud Networks, Llc | Distributed gateways for multi-regional large scale deployments |
| US12507153B2 (en) | 2023-08-16 | 2025-12-23 | Velocloud Networks, Llc | Dynamic edge-to-edge across multiple hops in multi-regional large scale deployments with distributed gateways |
| US12519746B2 (en) * | 2023-12-27 | 2026-01-06 | Cisco Technology, Inc. | SDWAN data-plane resiliency for extended survivability |
| CN118282866B (zh) * | 2024-06-03 | 2024-07-26 | 中宇联云计算服务(上海)有限公司 | 基于容器集群的多租户隔离部署方法、系统、设备及介质 |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160315808A1 (en) * | 2008-11-12 | 2016-10-27 | Teloip Inc. | System and method for providing a control plane for quality of service |
| US20180013556A1 (en) * | 2016-07-06 | 2018-01-11 | Teloip Inc. | System, apparatus and method for encrypting overlay networks using quantum key distribution |
| US20190280962A1 (en) * | 2017-01-31 | 2019-09-12 | The Mode Group | High performance software-defined core network |
Family Cites Families (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1998041040A2 (en) | 1997-03-13 | 1998-09-17 | Urizen Ltd. | Apparatus and method for expanding communication networks |
| US7715312B2 (en) | 2005-04-25 | 2010-05-11 | Verizon Services Corp. | Methods and systems for maintaining quality of service (QOS) levels for data transmissions |
| US20070248077A1 (en) | 2006-04-20 | 2007-10-25 | Fusion Telecommunications International, Inc. | Distributed voice over internet protocol apparatus and systems |
| US8125907B2 (en) | 2008-06-12 | 2012-02-28 | Talari Networks Incorporated | Flow-based adaptive private network with multiple WAN-paths |
| US10476765B2 (en) | 2009-06-11 | 2019-11-12 | Talari Networks Incorporated | Methods and apparatus for providing adaptive private network centralized management system discovery processes |
| US10785117B2 (en) | 2009-06-11 | 2020-09-22 | Talari Networks Incorporated | Methods and apparatus for configuring a standby WAN link in an adaptive private network |
| US8452846B2 (en) | 2010-08-12 | 2013-05-28 | Talari Networks Incorporated | Adaptive private network asynchronous distributed shared memory services |
| US10698923B2 (en) | 2009-06-11 | 2020-06-30 | Talari Networks, Inc. | Methods and apparatus for providing adaptive private network database schema migration and management processes |
| US9069727B2 (en) | 2011-08-12 | 2015-06-30 | Talari Networks Incorporated | Adaptive private network with geographically redundant network control nodes |
| US20130077701A1 (en) | 2011-09-23 | 2013-03-28 | Advanced Micro Devices, Inc. | Method and integrated circuit for adjusting the width of an input/output link |
| US8943000B2 (en) * | 2012-01-20 | 2015-01-27 | Cisco Technology, Inc. | Connectivity system for multi-tenant access networks |
| US10129096B2 (en) | 2012-06-20 | 2018-11-13 | Fusionlayer Oy | Commissioning/decommissioning networks in orchestrated or software-defined computing environments |
| US9407557B2 (en) | 2012-12-22 | 2016-08-02 | Edgewater Networks, Inc. | Methods and systems to split equipment control between local and remote processing units |
| CN103957155B (zh) | 2014-05-06 | 2018-01-23 | 华为技术有限公司 | 报文传输方法、装置及互联接口 |
| US9756135B2 (en) * | 2014-07-31 | 2017-09-05 | Ca, Inc. | Accessing network services from external networks |
| TW201728124A (zh) * | 2014-09-16 | 2017-08-01 | 科勞簡尼克斯股份有限公司 | 以彈性地定義之通信網路控制器為基礎之網路控制、操作及管理 |
| US10015287B2 (en) * | 2015-03-04 | 2018-07-03 | Oracle International Corporation | Efficient tunneled streams for real-time communications |
| US9894491B2 (en) | 2015-05-22 | 2018-02-13 | Ford Global Technologies, Llc | Context-based wireless network link access prioritization system |
| US9634893B2 (en) * | 2015-07-21 | 2017-04-25 | Cisco Technology, Inc. | Auto-provisioning edge devices in a communication network using control plane communications |
| US10608985B2 (en) * | 2015-08-14 | 2020-03-31 | Oracle International Corporation | Multihoming for tunneled encapsulated media |
| US20170055133A1 (en) | 2015-08-17 | 2017-02-23 | Adtran, Inc. | Multicast connection admission control |
| EP3417570B1 (en) * | 2016-02-18 | 2021-04-28 | FusionLayer Oy | Commissioning/decommissioning networks in orchestrated or software-defined computing environments |
| US10015097B2 (en) * | 2016-08-19 | 2018-07-03 | Oracle International Corporation | Fast access telecommunication tunnel cloning |
| WO2018042459A1 (en) | 2016-09-02 | 2018-03-08 | Muthukumarasamy Murugavel | Adaptive and seamless traffic steering among multiple paths based on application qoe needs |
| US10616224B2 (en) * | 2016-09-16 | 2020-04-07 | Oracle International Corporation | Tenant and service management for a multi-tenant identity and data security management cloud service |
| US10523556B2 (en) | 2017-08-08 | 2019-12-31 | Versa Networks, Inc. | Method and system for routing connections in a software-defined wide area network |
| US10673747B2 (en) | 2017-08-15 | 2020-06-02 | Level 3 Communications, Llc | Device deployment and network management using a self-service portal |
| WO2019043827A1 (ja) | 2017-08-30 | 2019-03-07 | エヌ・ティ・ティ・コミュニケーションズ株式会社 | ネットワーク制御装置、通信システム、ネットワーク制御方法、プログラム、及び記録媒体 |
-
2019
- 2019-09-27 US US16/586,300 patent/US11082304B2/en active Active
-
2020
- 2020-09-21 JP JP2022519341A patent/JP7641276B2/ja active Active
- 2020-09-21 CN CN202080063378.XA patent/CN114402574B/zh active Active
- 2020-09-21 EP EP20786379.6A patent/EP4035339B1/en active Active
- 2020-09-21 WO PCT/US2020/051882 patent/WO2021061581A1/en not_active Ceased
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160315808A1 (en) * | 2008-11-12 | 2016-10-27 | Teloip Inc. | System and method for providing a control plane for quality of service |
| US20180013556A1 (en) * | 2016-07-06 | 2018-01-11 | Teloip Inc. | System, apparatus and method for encrypting overlay networks using quantum key distribution |
| US20190280962A1 (en) * | 2017-01-31 | 2019-09-12 | The Mode Group | High performance software-defined core network |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11799793B2 (en) | 2012-12-19 | 2023-10-24 | Talari Networks Incorporated | Adaptive private network with dynamic conduit process |
| US11575605B2 (en) | 2016-01-19 | 2023-02-07 | Talari Networks Incorporated | Adaptive private network (APN) bandwidth enhancements |
| US11483228B2 (en) | 2021-01-29 | 2022-10-25 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for network testing using an emulated data center environment |
| US11716283B2 (en) | 2021-03-05 | 2023-08-01 | Oracle International Corporation | Methods, systems, and computer readable media for selecting a software defined wide area network (SD-WAN) link using network slice information |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2022550356A (ja) | 2022-12-01 |
| CN114402574A (zh) | 2022-04-26 |
| US11082304B2 (en) | 2021-08-03 |
| EP4035339B1 (en) | 2025-11-12 |
| EP4035339A1 (en) | 2022-08-03 |
| US20210099360A1 (en) | 2021-04-01 |
| JP7641276B2 (ja) | 2025-03-06 |
| CN114402574B (zh) | 2024-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4035339B1 (en) | Methods, systems, and computer readable media for providing a multi-tenant software-defined wide area network (sd-wan) node | |
| CN111066301B (zh) | 用于强制执行统一全局策略的方法、系统及存储介质 | |
| CN108551464B (zh) | 一种混合云的连接建立、数据传输方法、装置和系统 | |
| US9596077B2 (en) | Community of interest-based secured communications over IPsec | |
| US8743890B2 (en) | System and method for supporting sub-subnet in an infiniband (IB) network | |
| US8418244B2 (en) | Instant communication with TLS VPN tunnel management | |
| CN112422397B (zh) | 业务转发方法及通信装置 | |
| EP4323898B1 (en) | Computer-implemented methods and systems for establishing and/or controlling network connectivity | |
| JP5679343B2 (ja) | クラウドシステム、ゲートウェイ装置、通信制御方法、及び通信制御プログラム | |
| WO2017162030A1 (zh) | 一种虚拟网络的生成方法和装置 | |
| CN108462752B (zh) | 一种访问共享网络的方法、系统及vpc管理设备以及可读存储介质 | |
| CN110830351B (zh) | 基于SaaS服务模式的租户管理及服务提供方法、装置 | |
| US10491400B2 (en) | System and apparatus for providing network security | |
| CN109743316B (zh) | 数据传输方法、出口路由器、防火墙及双台防火墙系统 | |
| US20220210192A1 (en) | Network configuration security using encrypted transport | |
| US11569997B1 (en) | Security mechanisms for data plane extensions of provider network services | |
| EP4595406A1 (en) | System and method for creating a private service access network | |
| CN115378578B (zh) | 一种基于国密sm4的sd-wan实现方法及系统 | |
| CN119835058B (zh) | 网络设备管理方法、系统、计算机设备及可读存储介质 | |
| US12363073B1 (en) | System and method for establishing cryptographically secure tunnels | |
| EP4569744A1 (en) | Scalable creation of connections | |
| WO2024035634A1 (en) | Scalable creation of connections | |
| Cossu et al. | D5. 2: XIFI Core Backbone |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20786379 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 2022519341 Country of ref document: JP Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2020786379 Country of ref document: EP |
|
| ENP | Entry into the national phase |
Ref document number: 2020786379 Country of ref document: EP Effective date: 20220428 |
|
| WWG | Wipo information: grant in national office |
Ref document number: 202247007060 Country of ref document: IN |
|
| WWG | Wipo information: grant in national office |
Ref document number: 2020786379 Country of ref document: EP |