US20190036779A1 - Virtualized software-defined network - Google Patents
Virtualized software-defined network Download PDFInfo
- Publication number
- US20190036779A1 US20190036779A1 US16/019,478 US201816019478A US2019036779A1 US 20190036779 A1 US20190036779 A1 US 20190036779A1 US 201816019478 A US201816019478 A US 201816019478A US 2019036779 A1 US2019036779 A1 US 2019036779A1
- Authority
- US
- United States
- Prior art keywords
- sdn
- devices
- monitor
- data
- instructions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0895—Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
- H04L41/0816—Configuration setting characterised by the conditions triggering a change of settings the condition being an adaptation, e.g. in response to network events
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0823—Configuration setting characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/142—Network analysis or design using statistical or mathematical methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/34—Signalling channels for network management communication
- H04L41/342—Signalling channels for network management communication between virtual entities, e.g. orchestrators, SDN or NFV entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/40—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using virtualisation of network functions or resources, e.g. SDN or NFV entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/12—Network monitoring probes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/20—Arrangements for monitoring or testing data switching networks the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
Definitions
- the embodiments discussed in the present disclosure are related to a virtualized software-defined network.
- One or more embodiments of the present disclosure may include a method that may include identifying a set of devices in a software-defined network (SDN).
- the method may include determining a set of characteristics that the set of devices in the SDN are to monitor within the SDN.
- the method may include generating a set of instructions for the set of devices in the SDN to monitor the set of characteristics.
- the method may further include sending the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path.
- the method may also include receiving monitor data via the control plane from at least one device of the set of devices in the SDN.
- the monitor data may be based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time.
- the method may include determining a change to a set of parameters of the SDN.
- the method may further include generating a policy, based on the change to the set of parameters of the SDN.
- the method may also include sending the policy to the set of devices in the SDN. An execution of the policy at one or more of the set of devices in the SDN adjusts the monitor data to more closely match a template.
- One or more embodiments of the present disclosure may additionally include systems and/or non-transitory computer readable media for facilitating the performance of such methods.
- FIG. 1 illustrates an example system of network components implementing a software-defined network (SDN);
- SDN software-defined network
- FIG. 2 illustrates another example system implementing a SDN
- FIG. 3 illustrates an additional example system as part of a SDN
- FIG. 4 illustrates another example system implementing a SDN
- FIG. 5 illustrates a flowchart of an example method to monitor a set of characteristics in a SDN
- FIG. 6 illustrates a flowchart of an example method to generate a data model based on monitor data of a SDN
- FIG. 7 illustrates a flowchart of an example method to generate a policy based on a data model and monitor data in a SDN
- FIG. 8 illustrates an example computing system.
- a network control plane may be functionally separated from physical topology and a data plane, unlike conventional networking.
- the SDN may include one or more virtualized components, such as a router that is run on a virtual machine, a hypervisor, or a cloud-based server.
- a control device may monitor the SDN and path characteristics of the data plane tunnels between devices (e.g., routers). The control device may use this information to generate data models and generate policies. The policies may be sent to devices in the network to steer data traffic onto various links. Because the SDN extends a single virtual fabric over all available transports, paths or tunnels that satisfy the performance service level agreements (SLAs) for an application can be chosen for forwarding. Choice of transport can be influenced by cost, quality of service, service availability in specific geographies, circuit delivery times, operational processes, organizational policies, and regulations, such as in-country regulations and policies. Application-aware routing results in improved user experience and better bandwidth utilization.
- SLAs performance service level agreements
- One or more embodiments of the present disclosure may include solutions to technology-based problems associated with software-defined networking.
- Embodiments of the present disclosure may provide improvements to computer networks and to the operation of computers themselves. For example, using one or more embodiments of the present disclosure, network traffic may flow with increased performance preserving valuable network resources such as bandwidth and providing increased response times. Additionally, the amount of traffic flowing through the internal network domain may be reduced, providing superior performance for the internal network domain. As another example, path availability may be guaranteed for a rerouted path, which may improve reliability for important applications. As an additional example, the performance of applications utilizing third party resources may be improved because a path with an optimal or improved performance may be used for the application, allowing for increased response times, increased data throughput per unit time, among others.
- FIG. 1 illustrates an example system 100 of network components implementing a software-defined network (SDN), in accordance with one or more embodiments of the present disclosure.
- the SDN may include any type of network or network topology.
- the SDN may include a software-defined wide area network (SD-WAN), software-defined local area network (LAN), software-defined metropolitan area network (MAN), or any other type of network.
- SD-WAN software-defined wide area network
- LAN software-defined local area network
- MAN software-defined metropolitan area network
- the system 100 may include an internal network domain 105 and one or more external network domains.
- the system 100 may include one or more edge network devices 110 (such as the edge network devices 110 a - 110 d ), a control device 120 , a communication network 130 , and external network devices 140 and 141 (such as the external network devices 140 a - 140 d and 141 a - 141 d ).
- edge network devices 110 such as the edge network devices 110 a - 110 d
- control device 120 such as the control device 120
- communication network 130 such as the external network devices 140 a - 140 d and 141 a - 141 d
- external network devices 140 and 141 such as the external network devices 140 a - 140 d and 141 a - 141 d .
- the SDN may support multiple types of connections or communication links, such as the Internet, MultiProtocol Label Switching (MPLS) connections, and/or cellular connections (such as Long Term Evolution (LTE), LTE Advanced, Worldwide Interoperability for Microwave Access (WiMAX), Evolved High Speed Packet Access (HSPA+), and/or others). Additionally, the SDN may support load balancing or load sharing between the various connections. Further, because of the distributed nature of some networks, the SDN may support virtual private networks (VPNs), firewalls, and other security services.
- MPLS MultiProtocol Label Switching
- cellular connections such as Long Term Evolution (LTE), LTE Advanced, Worldwide Interoperability for Microwave Access (WiMAX), Evolved High Speed Packet Access (HSPA+), and/or others.
- LTE Long Term Evolution
- WiMAX Worldwide Interoperability for Microwave Access
- HSPA+ Evolved High Speed Packet Access
- a control plane may be functionally separated from the physical topology.
- the SDN may separate the control plane of the network (to be managed via software) from a data plane of the network (operating on the hardware of the network).
- control plane may refer to communications and connections used in the control and administration of a network itself, rather than the transmission of data through the network, which may occur at the data plane.
- the control plane may be used in conjunction with the collection of data from various components in the SDN.
- the term data plane may refer to communications and connections used in the transmission and reception of data through the network.
- control plane may include administrative traffic directed to a network device within a network
- data plane may include traffic that passes through network devices within the network.
- the control plane may be used to exchange data between the control device 120 and various components in the SDN separately than traffic on the data plane.
- data on the control plane may be isolated from traffic on the data plane.
- the isolation may be physical isolation (e.g., in different physical hardware and wires), or logical (e.g., isolated in the internal network domain 105 via software). Isolation of the data on the control plane from traffic on the data plane may provide increased security for one or both of the data on the control plane and the traffic on the data plane.
- Data on the control plane may be exchanged between the control device 120 and various components in the SDN at or near real-time.
- the control device 120 may be configured to manage the control plane of an internal network domain 105 by directing one or more aspects of the operation of the edge network devices 110 .
- the control device 120 may direct data collection activities at various components of the SDN.
- the control device 120 may direct one or more of the edge network devices 110 to monitor various characteristics of the SDN, such as performance, SLA, security, etc.
- Monitored data include data associated with the monitored characteristics.
- the monitored data may include bandwidth data, usage data, carrier data, geography data, user data, application data, site data, loss, latency, jitter, cost data, network events, syslogs, among other data.
- the control device 120 may use one or more control messages to direct the data collection activities at the various components of the SDN.
- the control device 120 may generate a set of instructions for a set of devices in the SDN to monitor a set of characteristics.
- the control device 120 may send the set of instructions as a control message to the set of devices in the SDN via the control plane.
- the set of devices in the SDN may collect the monitor data.
- the devices in the system may send the monitor data securely over a data-bus in real-time to the control device 120 .
- the edge network devices 110 may provide for one or more QoS metrics that may be monitored for any communication link, such as jitter, bandwidth, error rate, bit rate, throughput, and/or others.
- the various components of the SDN may send monitored data to the control device 120 via the control plane.
- the control device 120 may generate data models and/or policies based at least in part on the monitor data collected from the various components of the SDN.
- the control device 120 may receive an input signal (e.g., input from a system administrator, a software-based trigger) to generate a data model in view of a set of input parameters.
- the input signal may include the set of input parameters.
- the set of input parameters may include various QoS characteristics and/or the QoS metrics (e.g., jitter, bandwidth, error rate, bit rate, throughput, and/or others).
- the control device 120 may generate the data model based on the set of input parameters and the monitor data.
- control device 120 may apply machine learning algorithms to the monitored data in order to satisfy use-cases of network planning (e.g., forecasting), network operations (e.g., SLA policy recommendation, carrier selection), what-if analysis, and network security (anomaly detection).
- network planning e.g., forecasting
- network operations e.g., SLA policy recommendation, carrier selection
- what-if analysis e.g., SLA policy recommendation, carrier selection
- network security e.g., anomaly detection
- the control device 120 may, for example, use the monitor data and input parameters (e.g., bandwidth, apps, sites, alarm level) to provide a forecast of future bandwidth for apps and/or sites based on the monitor data (e.g., past usage) and the alarm level.
- the control device 120 may use one or more regression algorithms, such as regression algorithms directed to a generalized linear model with extreme gradient boost.
- the control device 120 may, for example, use the monitor data and various input parameters (e.g., carrier details, time) for carrier selection.
- the control device 120 may provide a data model around one or more carriers (e.g., Comcast, AT&T, Verizon) to illustrate carrier performance to customers.
- the customers may use the data model in determining whether to provision new sites in a particular geographical area.
- the data model may provide guidance on which carrier(s) are the top performing carriers in that particular geographical area.
- the control device 120 may, for example, use the monitor data for correlation and root cause analysis.
- the system 100 may include many (e.g., hundreds, thousands, hundreds of thousands, millions, etc.) of events coming from a SDN overlay network.
- the control device 120 may define categories of correlated events and then use network domain knowledge (e.g., the monitor data) to establish a cause and effect relationship between these correlated events.
- the control device 120 may then narrow the list to a set of potential root causes.
- the control device 120 may use an algorithm based on correlation with pearson correlation coefficient.
- the control device 120 may, for example, use the monitor data and input parameters to generate SLA policy recommendations.
- the control device 120 may estimate a cost function of making a change to the system 100 .
- the control device 120 may generate an approximation of a sensitivity score that may represent how sensitive a process may be to changes in loss, latency, jitter relative to other programs.
- the control device 120 may group classes of devices where a relative importance of loss, latency, and jitter within the devices in a particular class may be fixed or relatively constant.
- the sensitivity score may be generated using various functions. For example, the control device 120 may classify applications such as video applications into a first class and web application into a second class.
- the video applications may have a first sensitivity score that may be represented by the function of x+y+ ⁇ 2.
- the web applications may have a second sensitivity score that may be represented by a second function, 2x+y+log z.
- the control device 120 may then use an algorithm (e.g., a random forest of regression trees, a neural net) and the monitor data (e.g., historical SLA data) to generate an estimate of how triggering an SLA policy may affect the SLA characteristics of the system 100 .
- the control device 120 may use the output of the random forest algorithm and the estimated cost function as input to a linear regression function to compute a data model—an expected cost of different SLAs.
- the function applied may include a convex function.
- the control device 120 may use the expected cost of different SLAs against the monitor data to generate a policy that can be executed on an edge network device 110 .
- the control device 120 may also be used to generate various data models for “what-if” analyses.
- the “what if” analysis may be used to predict effects on the system 100 in response to various changes that may be made to the system 100 .
- a “what-if” data model may be used to predict how bandwidth needs may change in various scenarios, such as when new users are added to existing sites, new applications are added to existing sites, new sites are up with users and/or applications, etc.
- the “what-if” data model may also be used to provide an analysis on how SLA characteristics may change when some or all traffic is moved from a first tunnel to a second tunnel.
- Another aspect to moving applications from one tunnel to another includes network cost analysis, which may include a cost versus performance assessment. For example, a performance level of an applications on a broadband tunnel may be lesser than a performance of the application on an MPLS tunnel, but the cost savings of using the broadband tunnel may outweigh the increased performance that may be realized on the MPLS tunnel.
- the control device 120 may also be used to for network security, such as anomaly detection.
- the control device 120 may periodically analyze the monitor data for irregularities and/or anomalies.
- the control device 120 may use anomaly detection of applications at each site in the network for enforcing security related use-cases or to enforce improved planning.
- the control device 120 may use standard deviations and weighted moving averages. For example, a detection of the monitor data that is, for example, 2-3 standard deviations away from a moving average(baseline) may indicate an anomaly for normal distributions.
- the control device 120 may also use K-means for clustering. Any other suitable algorithm may be used.
- anomalies that may be detected include: anomaly detection for bandwidth usage on a per application/site basis, anomaly detection for network events, and anomaly detection for network performance metrics of loss/latency/jitter on a per application/site basis. These different uses for anomaly detection may assist the control device 120 and/or a network administrator to focus on relevant data points. These anomalous data points may also form a basis for various actions that may be taken within the system 100 . In addition to being useful for security purposes, anomaly detection may also be used for network operations, such as optimizing network bandwidth, application performance, and network troubleshooting, among others.
- the control device 120 may cause an action pertaining to the SDN in view of the data model. For example, based on the monitor data and/or analysis of the monitor data and data model, the control device 120 may generate a policy to affect various changes in the SDN. An execution of the policy at one or more of the set of devices in the SDN may adjust the monitor data to more closely match a template.
- the template may include a predetermined set of characteristics and/or outputs of the SDN.
- the control device 120 may distribute the policies to one or more of the edge network devices 110 .
- a policy may include a rule or set of rules bearing on the handling of network traffic, such as routing, priority, media, etc.
- the policy may include edge a threshold level for one or more QoS metrics, such as bandwidth, availability, jitter, and/or others.
- a policy on a given edge network device 110 may be configured to adjust or otherwise modify one or more properties of how the given edge network device 110 handles or routes traffic to better comply with one or more SLAs. For example, the traffic flow for one application may be sent via a particular communication link so that the traffic flow may comply with a corresponding SLA.
- the internal network domain 105 may operate as a secured and controlled domain with specific functionality and/or protocols.
- the edge network devices 110 may operate based on one or more policies created and/or propagated by the control device 120 .
- the edge network devices 110 may route data traffic within the internal network domain 105 based on the policies created and/or propagated by the control device 120 .
- control device 120 may form a control plane connection with each of the edge network devices 110 .
- the control plane connection may facilitate the exchange of data (e.g., various analytics data obtained by the edge network devices 110 ) between the edge network devices 110 and the control device 120 for management and control of the internal network domain 105 .
- the control plane connection may be separate from a data transfer plane.
- the control plane connection may operate via a tunnel through the communication network 130 , such as a Datagram Transport Layer Security (DTLS) tunnel.
- DTLS Datagram Transport Layer Security
- data transmitted over the control plane connection may facilitate the control device 120 determining topology and other characteristics of the communication network 130 .
- control device 120 may communicate with the edge network devices 110 to determine what physical connections exist between and among the edge network devices 110 in the communication network 130 . Additionally or alternatively, data transmitted over the control plane connection may facilitate the control device 120 determining available, optimal or desired paths across the communication network 130 between and among the edge network devices 110 . Additionally or alternatively, data transmitted over the control plane connection may facilitate the edge network devices 110 determining available, optimal or desired paths across the communication network 130 between and among the edge network devices 110 . Additionally or alternatively, the control device 120 may communicate information (e.g., control messages, sets of instructions, monitor data, policies) to and from the edge network devices 110 over the control plane connection.
- information e.g., control messages, sets of instructions, monitor data, policies
- control plane connection may include a permanent connection between the control device 120 and the edge network devices 110 such that if the connection between the control device 120 and a given edge network device 110 is broken, the edge network device 110 may be unable or otherwise disallowed from communicating over the internal network domain 105 .
- the control device 120 may maintain a central route table that stores route information within the internal network domain 105 .
- the control device 120 may communicate with various edge network devices 110 to determine the physical and/or logical connections available to the edge network devices 110 through the communication network 130 .
- the edge network devices 110 may include one or more physical and/or logical connections to each other.
- the control device 120 may generate and/or update one or more policies in conjunction with the monitor data and/or the central route table to help the edge network devices 110 to determine data traffic routes through the internal network domain 105 .
- the control device 120 may provide policies and other configuration preferences related to traffic flows to the edge network devices 110 rather than being involved with every individual flow through the internal network domain 105 .
- the edge network devices 110 may not have stored the topology and/or route paths of the entire system 100 . Each of the edge network devices 110 may not need to query each other individually to determine reachability. Instead, the control device 120 may provide such information to the edge network devices 110 . In these and other embodiments, the edge network devices 110 may route traffic through a most direct route, a most cost effective route, a most reliable route, or through some other route based on one or more other policies received from of the control device 120 , characteristics of the traffic, characteristics of the route path, a source edge network device 110 , and a destination (e.g., edge network device 110 ).
- the one or more policies may include guidance regarding determining next-hop and route path instructions.
- a particular policy may instruct a particular edge network device 110 where to route the traffic next for a particular category, class, or group of traffic flows, rather than providing a complete end-to-end route for the traffic.
- the edge network device 110 a may receive data from an external network device 140 a directed to an address of the external network device 141 c .
- the edge network device 110 a may have stored a first policy that the network device 110 a may use to determine the route path for the data, including that a “next-hop” for network traffic destined for the address of the external network device 141 c is to be routed to the edge network device 110 d.
- control device 120 may generate policies based at least in part on monitor data collected within the system and/or analytics performed on the monitor data to cause certain network traffic flows within the internal network domain 105 to be routed over certain types of connections or communication links (e.g., LTE, Internet, MPLS) and/or through certain edge network devices 110 .
- a link classification may indicate a type of communication link.
- the edge network devices 110 may use a policy to determine an action. For example, for a given set of data, the policy may indicate that a first routing path may be selected based on a configuration preference and based on a type or classification of a communication link.
- the edge network devices 110 may also determine an action (e.g., determine a route path) for all packets from a particular source, all packets that originated from a particular computer laptop, a type of traffic (e.g., voice), etc.
- a policy have a configuration preference that may indicate that all flows with IP addresses in the range 100.1/16 may be categorized as voice flows.
- the edge network devices 110 is connected to three different communication links (e.g., an Internet link an MPLS link, a cellular link).
- the configuration preference may indicate that voice traffic is to be routed over the Internet link.
- the edge network devices 110 may route all flows with IP addresses in the range 100.1/16 over the Internet link.
- configuration preferences may include costs (e.g., monetary, time, route, hops, transport health (such as loss, latency, and/or jitter), bandwidth, path geography, source (e.g., device, geography, user), destination (e.g., device, geography, user), applications (e.g., business, social), user groups (e.g., finance on a first subnet IP, HR on a second subnet IP, engineering on a third subnet IP, among others. Any type of data or information may be used as a configuration preference.
- the edge network device 110 may identify metadata associated with the traffic, such as a header.
- the header may include a DSCP value in the header to indicate a predefined routing path.
- the edge network device 110 may also make routing decisions based on a specific application (which may sometimes be referred to as a layer7 header or http header). For example, the edge network device 110 may route OFFICE365 traffic on a first path and SALESFORCE traffic on a second path.
- a specific application which may sometimes be referred to as a layer7 header or http header.
- the control device 120 may receive one or more keys from the edge network devices 110 used in communication of data over the data plane. For example, one or more data packets may use one or more keys for security purposes in transmitting data from one edge network device 110 to another edge network device 110 . In these and other embodiments, the control device 120 may reflect the received keys to one or more other edge network devices 110 that may be in the traffic flow based on the central route table and/or the policies implemented by the control device 120 . In these and other embodiments, a given edge network device 110 may generate symmetrical keys to facilitate secure communication between edge network devices.
- a pair of symmetrical keys may be generated by the given edge network device 110 , with one remaining with the given edge network device 110 and the other provided to the control device 120 such that the control device 120 may distribute the other symmetrical key to other edge network devices that communicate with the given edge network device 110 .
- each edge network device that is to communicate with the given edge network device 110 based on the policies of the control device 120 may receive the symmetrical key.
- traffic within the internal network domain 105 may be encrypted, such as with two-way authentication using Advanced Encryption Standard (AES) with a 256-bit length key over one or more Datagram Transport Layer Security (DTLS) and/or Transport Layer Security (TLS) connections between edge network devices 110 .
- AES Advanced Encryption Standard
- DTLS Datagram Transport Layer Security
- TLS Transport Layer Security
- the control device 120 may store authentication information for one or more (or all) of the edge network devices 110 within the internal network domain 105 .
- a device may be prevented from communicating within the internal network domain 105 unless the device has authentication information that matches or otherwise corresponds to the stored authentication information of the control device 120 .
- the authentication information may be used when the edge network devices 110 first come on line to establish the control plane connection, and any device without a control plane connection with the control device 120 may be prevented from communicating within the internal network domain 105 .
- the edge network devices 110 may operate at a boundary of the internal network domain 105 .
- the edge network devices 110 may include one or more physical and/or logical connections that may operate within the internal network domain 105 . Such connections may be illustrated as part of the communication network 130 . Additionally or alternatively, the edge network devices 110 may include one or more physical and/or logical connections operating outside of the internal network domain 105 . For example, the edge network devices 110 may be connected to the external network device(s) 140 and/or 141 .
- the edge network devices 110 may operate to route traffic from associated external network devices 140 and 141 into the internal network domain 105 . Additionally or alternatively, the edge network devices 110 may operate to route traffic from the internal network domain 105 to the associated external network devices 140 and 141 . In some embodiments, the edge network devices 110 may communicate with associated external network devices 140 and 141 using typical communication protocols, such as Open Shortest Path First (OSPF), Border Gateway Protocol (BGP), Virtual Router Redundancy Protocol (VRRP), Bi-directional Forwarding Detection (BFD), among others.
- OSPF Open Shortest Path First
- Border Gateway Protocol BGP
- VRRP Virtual Router Redundancy Protocol
- BFD Bi-directional Forwarding Detection
- the edge network devices 110 may support other network functionalities such as differentiated services code point (DSCP) tagging or type of service (TOS) tagging, Quality of Service (QoS) monitoring, Service Level Agreements (SLA), Internet Protocol (IP) forwarding, Internet Protocol Security (IPsec), Access Control Lists (ACL), among others.
- DSCP differentiated services code point
- TOS type of service
- QoS Quality of Service
- SLA Service Level Agreements
- IP Internet Protocol
- IPsec Internet Protocol Security
- ACL Access Control Lists
- the edge network devices 110 may be configured to insert a DSCP or TOS tag into a packet header.
- a DSCP or TOS tag may identify one (or a preferences for one) communication link of multiple communication links on which to send certain types of network traffic. Based on the DSCP or TOS tag, the edge network devices 110 may route the network traffic via one or more types of communication link.
- the edge network devices 110 may include one or more policies to route packets via a particular type of communication link in an 1 P network. For example, such a policy may take into account factors such as packet size, services specified by a header, characteristics of potential links to other routers in the network, and/or others. Using such factors, the edge network devices 110 may forward packets based on a selected algorithm, such as a shortest path.
- the edge network devices 110 may use IPsec to authenticate and/or encrypt network traffic.
- a given edge network device 110 may authenticate one or more computing devices to communicate with the given edge network device 110 and/or encrypt one or more packets communicated between the computing device and the given edge network device 110 .
- the edge network devices 110 may include a set of rules indicative of one or more addresses, hosts, and/or networks that may be permitted to use a given port or a particular communication link.
- the edge network devices 110 may include ACLs that are applicable to inbound traffic, outbound traffic, or both.
- the edge network devices 110 may locally maintain one or more route tables. In some embodiments, the edge network devices 110 may adjust or modify the route tables based on one or more policies sent from the control device 120 . For example, one or more entries may be removed, discarded, or otherwise not added to the route tables by the edge network devices 110 based on the one or more policies. In some embodiments, the edge network devices 110 may include logic to update, modify, and/or generate the route tables based on policies from the control device 120 and/or from traffic handled by the edge network devices 110 . The one or more route tables may be automatically populated by the edge network devices 110 based on direct interface routes, static routes, and/or dynamic routes learned using one or more network protocols such as BGP and/or OSPF.
- BGP BGP and/or OSPF
- routing decisions for data outside of the internal network domain 105 may be performed by a particular edge network device 110 without specific direction, input, or control from the control device 120 .
- the particular edge network device 110 may compute a routing decision based on the one or more policies that the particular edge network device 110 has received from the control device 120 .
- one or more of the edge network devices 110 and/or the control device 120 may be implemented as one or more virtual machines operating on one or more physical computing devices. Additionally or alternatively, the edge network devices 110 and/or the control device 120 may each include an individual stand-alone computing device.
- the system 100 may include any number of edge network devices 110 and control devices 120 , such as thousands or tens of thousands of edge network devices 110 and more than five control devices 120 .
- the communication network 130 may include multiple types of communication connections.
- FIG. 2 illustrates another example system 200 of network components implementing a SDN, in accordance with one or more embodiments of the present disclosure.
- the system 200 may include one or more edge network devices 210 (such as edge network devices 210 a - 210 o ), one or more control devices 220 (such as control devices 220 a , 220 b , and 220 c ), and one or more communication networks 230 (such as communication networks 230 a , 230 b , and 230 c ).
- the edge network devices 210 may be similar or comparable to the edge network devices 110 of FIG. 1
- the control devices 220 may be similar or comparable to the control device 120 of FIG. 1
- the communication networks 230 may be similar or comparable to the communication network 130 of FIG. 1 .
- the system 200 may be a similar or comparable system to the system 100 of FIG. 1 , although expanded to include additional network components and additional external network domains.
- the system 200 may include an internal network domain 205 in and between the edge network devices 210 , in a similar or comparable manner to that described with respect to the system 100 of FIG. 1 .
- the system 200 additionally may include multiple external network domains.
- a data center 240 may represent a first external network domain
- a campus 250 may represent a second external network domain
- a branch 260 may represent a third external network domain
- a remote site 270 may represent a fourth external network domain.
- each external network domain may include one or more edge network devices 210 acting as a bridge between the internal network domain 205 and the given external network domain.
- one or more of the external network domains may functionally operate as being accessible from the other external network domains as though in a single network by being communicatively coupled through the internal network domain 205 .
- the system 200 may include one or more external resources 280 (such as the external resources 280 a - 280 c ).
- the external resources 280 may be operated by the same entity or organization that operates the internal network domain 205 , or may be operated by a different entity.
- the system 200 may include an edge network device 210 that may be associated with a particular external resource 280 .
- the system 200 may include an edge network device 210 located within a regional co-location facility.
- a regional co-location facility may include a location with directed or guaranteed access to the Internet or other communication protocols at a given physical location.
- a regional co-location facility may include a prioritized or improved connection to one or more of the external resources 280 .
- the regional co-location facility may be at a designated geographical location that may be physically proximate one or more of the external network domains.
- the data center 240 may be located in New York
- the branch 260 may be located in Dallas Tex.
- the edge network device 210 n may be in a regional co-location facility in Houston, Tex.
- the external resources 280 may include any computing service available for consumption by the system 200 .
- the external resources 280 may include a cloud-based service such as a software subscription or software as a service (SaaS) (such as Microsoft Office 365®, Azure®, Google Apps®, Workforce®, Amazon Web Services®, WorkDay®, DocuSign®, GoToMeeting®, WebEx®, QuickBooks®, and/or others), media services (such as YouTube®, NetFlix®, Pandora®, Spotify®, and/or others), and/or others.
- the external resources 280 may include a third party network to facilitate access to the external resource 280 with one or more access points at various geographical locations.
- a SaaS may include an access server in Austin, Tex.; Palo Alto, Calif.; and New York, N.Y. for accessing the third party network.
- the system 200 may be geographically distributed.
- the data center 240 may be located in St. Paul, Minn.; the campus 250 may be located in Des Moines, Iowa; there may be branches 260 in Seattle, Wash.; Los Angeles, Calif.; Atlanta, Ga.; and Orlando, Fla.; and there may be remote sites 270 in London, England; Berlin, Germany; and Seoul, Korea.
- the system 200 may use the communication networks 230 and the internal network domain 205 to facilitate communication between all of these distributed physical locations as a single network.
- one or more of the external network domains may use one or more applications with resources in the data center 240 , such as Microsoft Exchange®, SharePoint®, Oracle e-Business Suite®, and/or others.
- applications such as Microsoft Exchange®, SharePoint®, Oracle e-Business Suite®, and/or others.
- a workstation operating at the campus 250 may operate Microsoft Exchange®.
- the operation of the application may include a data flow that goes from the workstation to the edge network device 210 e in the external network domain of the campus 250 .
- the data flow may go from the edge network device 210 e to one of the edge network devices 210 b , 210 c , and/or 210 d associated with the data center 240 through the internal network domain 205 .
- the one of the edge network devices 210 b , 210 c , and/or 210 d may route the traffic to the Microsoft Exchange® server in the external network domain of the data center 240 . Additionally or alternatively, the operation of the application may include a data flow in the reverse order of data flowing from the Microsoft Exchange® server to the workstation.
- the system 200 may include a network management device 290 that may communicate with the control devices 220 over a management network 232 .
- the network management device 290 may provide management and control of one or more devices associated with the internal network domain 205 , including the edge network devices 210 , the control devices 220 , and/or others.
- the network management device 290 may provide a graphical user interface (GUI) that provides a network administrator with access to control or observe operation of the internal network domain 205 .
- GUI graphical user interface
- the network administrator may input policies via the network management device 290 that may be communicated to the control devices 220 for implementation via the edge network devices 210 .
- the network management device 290 may provide a GUI dashboard with a visual and/or textual description of one or more properties of the internal network domain 205 , such as a number and/or status and/or health of edge network devices 210 , a number and/or status of control devices 220 , a number of and/or last time of reboot, transport health (such as loss, latency, and/or jitter), a number of sites that are operating or not operating, application consumption of network resources, application routing, and/or others.
- a visual and/or textual description of one or more properties of the internal network domain 205 such as a number and/or status and/or health of edge network devices 210 , a number and/or status of control devices 220 , a number of and/or last time of reboot, transport health (such as loss, latency, and/or jitter), a number of sites that are operating or not operating, application consumption of network resources, application routing, and/or others.
- the network management device 290 may be configured to recognize approved edge network devices 210 and/or control device 220 .
- the network management device 290 may maintain a list of serial numbers, MAC addresses, or other uniquely identifying information for the edge network devices 210 and/or the control devices 220 .
- communication in the internal network domain 205 may be restricted to edge network devices 210 and/or control devices 220 with identifying information on the list maintained by the network management device 290 .
- the network management device 290 may be configured to generate and/or store configurations of one or more edge network devices 210 and/or control devices 220 .
- a network administrator may use the network management device 290 to configure a particular edge network device 210 and may store that configuration as a template that may be applied to future edge network devices.
- a template for the edge network devices 210 may be provided by a third party and applied to a new edge network device 210 .
- a template for the control devices 220 may be generated, stored, and/or applied to a new control device 220 . Additionally or alternatively, such a template may be used to automatically configure a newly deployed edge network device 210 .
- the newly deployed edge network device 210 may be brought online and connected to a corresponding control device 220 .
- the corresponding control device 220 may verify the serial number of the edge network device 210 with the network management device 290 , and may obtain a template from the network management device 290 for the edge network device 210 .
- the control device 220 may send the template to the edge network device 210 to be automatically installed to configure the edge network device 210 according to the template.
- the network management device 290 may be configured to manage data collection activities at various components of the SDN. For example, the network management device 290 may direct one or more of the edge network devices 110 to monitor various characteristics of the SDN, such as performance, SLA, security, etc. in a system with multiple control devices 220 .
- the network management device 290 may use one or more control messages to direct the data collection activities at the various components of the SDN. For example, the network management device 290 may generate a set of instructions for a set of devices in the SDN to monitor a set of characteristics. The network management device 290 may send the set of instructions as a control message to the set of devices in the SDN via the control plane. In at least some embodiments, the network management device 290 sends the control message to the control devices 220 and the control devices 220 may distribute the control message to devices that are connect to the respective control device 220 .
- the set of devices in the SDN may collect the monitor data.
- the devices in the system may send the monitor data securely over a data-bus (e.g., the control plane) in real-time to the network management device 290 , such as directly to the network management device 290 or indirectly, such as via the control devices 220 .
- the edge network devices 110 may provide for one or more QoS metrics that may be monitored for any communication link, such as jitter, bandwidth, error rate, bit rate, throughput, and/or others.
- the various components of the SDN may send monitored data to the network management device 290 via the control plane.
- the network management device 290 may generate data models and/or policies based at least in part on the monitor data collected from the various components of the SDN.
- the network management device 290 may receive an input signal (e.g., input from a system administrator, a software-based trigger) to generate a data model in view of a set of input parameters.
- the network management device 290 may generate the data model based on the set of input parameters and the monitor data.
- the network management device 290 may apply machine learning algorithms to the monitored data in order to satisfy use-cases of network planning (e.g., forecasting), network operations (e.g., SLA policy recommendation, carrier selection), what-if analysis, and network security (anomaly detection).
- the network management device 290 may generate a policy to affect various changes in the SDN.
- the network management device 290 may distribute the policies to one or more of the edge network devices 110 .
- control devices 220 may generate and communicate control messages and/or policies independently of the network management device 290 and the other control devices 220 , such as is described in conjunction with FIG. 1 .
- the network management device 290 may be implemented as a physical device or a virtualized machine. In these and other embodiments, the network management device 290 may be physically located proximate a centralized location, such as within the data center 240 or at the campus 250 .
- FIG. 2 While illustrated as including a certain number of edge network devices 210 and external network domains, the system 200 may include any number of edge network devices 210 and external network domains.
- FIG. 3 illustrates an additional example system 300 , in accordance with one or more embodiments of the present disclosure.
- FIG. 3 illustrates an edge network device 310 a that may include multiple potential communication links for communicating across an internal network domain 305 to another edge network device 310 b .
- the edge network device 310 a may communicate across the internal network domain 305 using the Internet 360 , an MPLS network 370 , and/or an LTE network 380 .
- the edge network devices 310 a and 310 b may be similar or comparable to the edge network device 110 of FIG. 1 and/or the edge network devices 210 a - 210 o of FIG. 2 .
- the system 300 may additionally include an external local device 350 that may be communicatively coupled to the edge network device 310 a across an external network domain.
- the edge network device 310 a may include an Internet connection 320 , an MPLS connection 330 , and an LTE connection 340 . As illustrated by the ellipses below the LTE connection 340 , any number of additional or other potential connections may also be included. In these and other embodiments, the edge network device 310 a may include multiple circuits for connecting to the one or more potential communication links. For example, the edge network device 310 a may include a circuit A 322 and a circuit B 324 for the Internet connection 320 , a circuit A 332 and a circuit B 334 for the MPLS connection 330 , and a circuit A 342 and a circuit B 344 for the LTE connection 340 . In these and other embodiments, the edge network device 310 a may be configured to route traffic along one or more of the circuits, based on one or more policies stored by the edge network device 310 a.
- control plane may be implemented on one or more of the circuits 322 , 324 , 332 , 334 , 342 , 344 .
- control plane may be implemented on MPLS circuit A 332 and, to achieve isolation of the control plane from the data plane, the data plane maybe implemented on any of the other circuits 324 , 334 , 342 , 344 .
- the edge network device 310 a may be configured to monitor one or more properties of the various connections. For example, the edge network device 310 a may monitor the jitter, latency, loss, and/or bandwidth of the various communication links from the edge network device 310 a to the edge network device 310 b . In these and other embodiments, the edge network device 310 a may also monitor and/or store security properties of the various communication links. For example, links 362 and 364 over the Internet 360 may be considered at a first level of security, and links 372 and 374 over the MPLS network 370 may be considered at a second level of security higher than the first level of security.
- the edge network device 310 a may receive a data flow for a security-sensitive application (such as an accounting application) and may have a policy that data for that application is to be routed along one of the MPLS links 372 and/or 374 , even if other traffic may be routed along the Internet link 362 .
- the edge network device 310 a may include an SLA that a given application have a bandwidth of 10 MB/s available to the application. The edge network device 310 a may make the link 362 over the Internet 360 available to the application, but the link 362 may provide 5 MB/s of bandwidth.
- the edge network device 310 a may also provide the links 382 and 384 to the application such that the overall combined bandwidth of the links 362 , 382 , and 384 exceed the bandwidth agreement of the SLA.
- the edge network device 310 a may route traffic for a given application via a particular type of communication link.
- the edge network device 310 a may prohibit traffic for a given application along a particular type of communication link. For example, traffic associated with a social network or video application may be routed over the Internet connections 362 , 364 and prohibited from being routing over MPLS connections 372 , 374 .
- the edge network device 310 a may be configured to perform such routing based on initially receiving a data flow, during an on-going data flow (e.g., in fast path), based on a triggering event of the data flow, and/or others or combinations thereof. Additionally or alternatively, such routing may combine multiple links of multiple types of connections for a single flow in routing traffic flows.
- the edge network device 310 a may be configured to route traffic to the various links based on the source of the traffic. For example, one or more policies may indicate that traffic from one corporate department of a business is routed along the MPLS connection 330 , while traffic for another corporate department may be routed along any link.
- the edge network device 310 a may be implemented as a computing system, such as the computing system 600 illustrated in FIG. 6 .
- the system 300 may include any number of edge network devices 310 .
- the system 300 may include any number of edge network devices 310 .
- any number of communication networks may be used.
- FIG. 4 illustrates another example system 400 implementing a SDN, in accordance with one or more embodiments of the present disclosure.
- the system 400 may include one or more edge network devices 410 (such as the edge network devices 410 a - 410 f ), which may be similar or comparable to the edge network devices 110 of FIG. 1, 210 of FIG. 2 , and/or 310 of FIG. 3 .
- the system 400 may also include one or more control devices 420 , which may be similar or comparable to the control device 120 of FIG. 1 , and/or 220 of FIG. 2 .
- the system 400 may additionally include one or more communication networks 430 and/or 432 (such as the communication networks 432 a - 432 c ), which may be similar or comparable to the communication network 130 of FIG.
- the system may additionally include a data center 440 , which may be similar or comparable to the data center 240 of FIG. 2 .
- the system may also include one or more third party resources 480 (such as the third party resources 480 a - 480 c ), which may be similar or comparable to the third party resources 280 a - c of FIG. 2 .
- the third party resources 480 a - 480 c may serve the same third party resource and may represent distinct servers for accessing the third party resource.
- the third party resource 480 a may include a server for accessing a cloud based service in Seattle, Wash.
- the third party resource 480 b may include a server for accessing the cloud based service in Los Angeles, Calif.
- the third party resource 480 c may include a server for accessing the cloud based service in New York, N.Y.
- the edge network device 410 a may determine which path a traffic flow may take based on a policy. For example, the edge network device 410 a may identify metadata associated with the traffic flow and determine, based on the policy and metadata, a configuration preference in the policy for the data to use routing paths with a first link classification. The edge network device 410 a may select a first routing path based on the configuration preference in the policy and based on the first routing path including the first communication link associated with the first link classification. The edge network device 410 a may transmit the data along the first routing path via the first communication link. In at least one embodiment, there may be multiple paths to a particular destination via different edge network device connections to the same underlying transport (e.g., Internet or MPLS). Alternatively or additionally, there may be multiple paths via the same two sets of edge network devices connected via different underlying transport. For example a first underlying transport may include Internet and a second underlying transport may include MPLS.
- underlying transport e.g., Internet or MPLS
- each of the edge network devices 410 may assess the performance of paths between a given edge network device 410 and the other edge network devices 410 .
- the edge network device 410 a may monitor the performance of the paths 461 , 462 , 465 , and 466 ; and the edge network device 410 b may monitor the performance of the paths 463 , 464 , 467 , and 468 .
- the edge network devices 410 may monitor one or more of jitter, latency, loss, and/or bandwidth of the various paths.
- one or more test packets may be communicated among or between the edge network devices 410 and characteristics of the travel time and/or integrity of the test packets may be used to determine the performance metrics of the paths. Additionally or alternatively, one or more of the performance metrics may be combined into a single score reflecting the performance of the paths within the internal network domain 405 .
- one or more of the edge network devices 410 may communicate the determined performance metrics with one or more components of the system 400 .
- the edge network devices 410 may communicate the performance metrics to the control device 420 , and the control device 420 may distribute the performance metrics to one or more of the edge network devices 410 .
- the edge network devices 410 may communicate the performance metrics to one or more other edge network devices 410 (e.g., the edge network device 410 b may communicate the performance metrics for the paths 463 , 464 , 467 , and 468 to the edge network device 410 a ).
- the control device 420 may determine and/or updated a policy based at least in part on the performance metrics.
- one or more of the edge network devices 410 may assess the performance of paths between a given edge network device 410 and one or more connections to the third party resource 480 .
- the edge network devices 410 e and/or 410 f may monitor the performance of the path 491
- the edge network device 410 c may monitor the performance of the path 492
- the edge network device 410 d may monitor the performance of the path 493 .
- the edge network devices 410 may monitor one or more of jitter, latency, loss, and/or bandwidth of the various paths.
- one or more requests may be communicated from the edge network devices 410 to the third party resource 480 and characteristics of the travel time and/or integrity of the response to the request may be used to determine the performance metrics of the paths.
- the edge network devices 410 may use httping, or some other similar tool.
- one or more of the performance metrics may be combined into a single score reflecting the performance of the path outside of the internal network domain 405 .
- the edge network devices 410 may maintain a table, database, or other storage structure of the scores of the performance metrics of the various paths in the system 400 . In these and other embodiments, the edge network devices 410 may use the stored scores when determining routing paths for data. For example, the edge network device 410 a may store a table with a single score for each of the paths in the system 400 .
- the edge network device 410 a may compare scores of the potential paths to the third party resource 480 to determine which path the rerouted traffic may flow along. For example, the edge network device 410 a may compare the combined scores of the paths 461 + 491 , 462 + 491 , 465 + 493 , 466 + 492 , 467 + 493 , and 468 + 492 . In these and other embodiments, the edge network device 410 may determine which score represents the best performance for the traffic.
- the internal network domain 405 may include multiple possible paths between two edge network devices 410 .
- the path 465 between the edge network device 410 a and the edge network device 410 d may represent an MPLS connection
- a second connection (not illustrated) between the edge network device 410 a and the edge network device 410 d may include an Internet or cellular connection.
- each path, including multiple paths between the same two edge network devices 410 may each include a unique score generated by the control device 420 . Using such unique scores, the control device 420 may determine which path to be used and generate and distribute a policy accordingly.
- the edge network device 410 a may route the traffic along the multiple paths with the best score. For example, a first set of the data may be routed along the first path and a second set of the data may be routed along a second path with the same score as the first path. In determining whether to route along the first path or the second path, the edge network device 410 a may hash the header of a packet within the flow and, depending on the output of the hash, may route the flow to one of the first path or the second path. While described as the path or paths with the best score, the path with a score relative to a threshold may also be selected.
- the policy may designate a primary path and a backup path for the rerouted path.
- the edge network device 410 a may monitor the performance of the primary path of the rerouted path and, based on changes in the score for the primary path and the policy, the edge network device 410 a may reroute the traffic to the backup path or a different path.
- the score may be monitored and/or rerouted relative to an SLA.
- the primary path may be determined based on the configuration preference.
- FIG. 4 Modifications, additions, or omissions may be made to FIG. 4 without departing from the scope of the present disclosure.
- the system 400 may include any number of edge network devices 410 .
- any number of paths over any number of mediums may be included between edge network devices 410 .
- FIGS. 5-7 illustrates a flowchart of an example method 500 of monitoring, analyzing and taking action with respect to a SDN, in accordance with one or more embodiments of the present disclosure.
- the method may be performed by processing logic that may include hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both, which processing logic may be included in the any of the network devices (e.g., the control devices 120 , 220 , 420 of FIGS. 1, 2 and 4 , and/or the network management device 290 of FIG. 2 ), or another computer system or device. However, another system, or combination of systems, may be used to perform the methods.
- processing logic may include hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both, which processing logic may be included in the any of the network devices (e.g., the control devices 120 , 220 , 420
- FIG. 5 illustrates a flowchart of an example method 500 to monitor a set of characteristics in a SDN.
- the method 500 may begin at block 505 , where the processing logic may identify a set of devices in a software-defined network (SDN). At least one device of the set of devices in the SDN may include a virtual device.
- the set of devices include any type or location of devices, such as one or more edge network devices 110 (such as the edge network devices 110 a - 110 d ), a control device 120 , a communication network 130 , and external network devices 140 and 141 (such as the external network devices 140 a - 140 d and 141 a - 141 d ), described in more detail in conjunction with FIG. 1 or any other device or connection described in conjunction with FIGS. 2-4 .
- the processing logic may determine a set of characteristics that the set of devices in the SDN are to monitor within the SDN.
- the processing logic may determine the set of characteristics based on input received from a system administrator, and/or in response to machine-based input (e.g., in response to machine learned characteristics to monitor).
- determining the set of characteristics that the set of devices in the SDN are to monitor within the SDN includes determining a first set of characteristics for a first device type and determining a second set of characteristics for a second device type.
- the first set of characteristics may be related to an edge device and the second set of characteristics may be related to a circuit.
- determining the set of characteristics that the set of devices in the SDN are to monitor within the SDN includes determining to monitor at least one of: an anomaly, performance data, location data, payload data, carrier data, data related to an application, and network characteristics. In at least some embodiments, determining the set of characteristics that the set of devices in the SDN are to monitor within the SDN includes selecting a subset of devices of the set of devices in the SDN. For example, the processing logic may select devices of a certain type, or a subset of devices within a given type.
- the processing logic may generate a set of instructions for the set of devices in the SDN to monitor the set of characteristics.
- generating the set of instructions for the set of devices in the SDN to monitor the set of characteristics may include generating the set of instructions for a subset of devices.
- the processing logic may send the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path.
- sending the set of instructions as the control message to the set of devices in the SDN via the control plane that is isolated from the packet forwarding path may include sending a first set of characteristics to one or more devices of a first device type and sending a second set of characteristics to one or more devices of a second device type.
- sending the set of instructions as the control message to the set of devices in the SDN via the control plane that is isolated from the packet forwarding path may include sending the set of instructions as the control message to a subset of devices via the control plane that is isolated from the packet forwarding path.
- the processing logic may receive monitor data via the control plane from at least one device of the set of devices in the SDN.
- the monitor data may be based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time.
- the monitor data received via the control plane is encrypted.
- the processing logic may decrypt such encrypted monitor data.
- the processing logic may determine a change to a set of parameters of the SDN.
- determining the change to the set of parameters of the SDN may include comparing the monitor data with a template and determining at least one difference between the monitor data and the template.
- the change to the set of parameters of the SDN is determined to adjust the monitor data to more closely match the template.
- the template may include a predetermined set of characteristics that set of devices in the SDN are to have.
- the template may be defined by a system administrator, an SLA, etc.
- the processing logic may generate a policy, based on the change to the set of parameters of the SDN.
- An execution of the policy at one or more of the set of devices in the SDN may adjust the monitor data to more closely match the template
- the processing logic may send the policy to the set of devices in the SDN.
- the processing logic may send the policy to the set of devices in the SDN via the control plane.
- FIG. 6 illustrates a flowchart of an example method 600 to generate a data model based on monitor data of a SDN.
- the method 600 may begin at block 605 , where the processing logic may generate a set of instructions for a set of devices in a software-defined network (SDN) to monitor a set of characteristics, as further described in conjunction with FIG. 5 .
- SDN software-defined network
- the processing logic may send the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path, as further described in conjunction with FIG. 5 .
- the processing logic may receive monitor data via the control plane from at least one device of the set of devices in the SDN, the monitor data being based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time.
- the processing logic may receive an input signal to generate a data model in view of a set of input parameters.
- the input signal may be received from a system administrator via a user interface, or may be generated based on machine-learned activities.
- the processing logic may generate the data model based on the set of input parameters and the monitor data.
- the data model may pertain to a network planning for the SDN based on the set of input parameters and the monitor data.
- the data model may pertain to one or more carriers of the packet forwarding path in a particular geographical region.
- the data model may pertain network security in the SDN.
- generating the data model based on the set of input parameters and the monitor data may include determining, based on the set of input parameters, at least one network security characteristic in view of the monitor data.
- the data model may pertain to a what-if analysis for at least one of: adding a new user to an existing site in the SDN, adding a new application to an existing site in the SDN, adding a new site to the SDN.
- the processing logic may cause an action pertaining to the SDN in view of the data model.
- causing the action pertaining to the SDN in view of the data model may include generating a policy based on the data model and sending the policy to the set of devices in the SDN via the control plane.
- the policy may include a dynamic service level agreement (SLA) policy.
- the data model may include a model for how SLA characteristics may be affected in response to a change to the SDN.
- the data model may include a traffic model for when traffic may be moved from one tunnel to another tunnel.
- An execution of the policy at one or more of the set of devices in the SDN adjusts the monitor data to more closely match a template, such as the template further described in conjunction with FIG. 5 .
- generating the policy based on the data model may include determining a change to a set of operation parameters of the SDN by comparing the monitor data with a baseline set of activity data and determining at least one difference between the monitor data and the baseline set of activity data. The change to the set of parameters of the SDN is determined by the processing logic to adjust the monitor data to more closely match the template.
- causing the action pertaining to the SDN in view of the data model may include selecting one carrier of one or more carriers to handle the packet forwarding path.
- causing the action pertaining to the SDN in view of the data model may include generating a parameter recommendation for the policy based on the data model.
- the parameter recommendation for the policy is generated based on at least one network characteristic.
- the at least one network characteristic may include at least one of a loss, a latency, and a jitter.
- FIG. 7 illustrates a flowchart of an example method 700 to generate a policy based on a data model and monitor data in a SDN.
- the method 700 may begin at block 705 , where the processing logic may generate a set of instructions for a set of devices in a software-defined network (SDN) to monitor a set of characteristics.
- SDN software-defined network
- the processing logic may send the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path.
- the processing logic may receive monitor data via the control plane from at least one device of the set of devices in the SDN, the monitor data being based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time.
- the processing logic may generate a data model based on a set of SDN parameters and the monitor data.
- generating the data model based on the set of SDN parameters and the monitor data includes generating an approximation of a sensitivity score for a process.
- the sensitivity score for the process is based on how sensitive the process may be to changes in loss, latency and jitter.
- the processing logic may also generate an estimate on how the change may affect the set of devices in the SDN.
- generating the policy, based on the change for at least one device of the set of devices in the SDN may include generating the policy to reduce the effect on the set of devices in the SDN below a threshold level.
- generating the estimate on how the change may affect the set of devices in the SDN may include using a random forest algorithm and historical data to estimate on how the change may affect the set of devices in the SDN.
- generating the data model based on the set of SDN parameters and the monitor data further may include generating the data model based on an output of the random forest algorithm and an estimated cost function to determine expected cost of different SLA.
- the processing logic may determine a change for at least one device of the set of devices in the SDN based on the data model.
- the change for at least one device of the set of devices in the SDN includes a change to at least one data route path.
- the processing logic may generate a policy, based on the change for at least one device of the set of devices in the SDN.
- the processing logic may send the policy via the control plane to the set of devices in the SDN, wherein an execution of the policy at one or more of the set of devices in the SDN adjusts the monitor data to more closely match a template.
- the processing logic may update the policy based on additional monitor data.
- the processing logic may receive additional monitor data after sending the policy via the control plane to the set of devices in the SDN.
- the processing logic may update based on the additional monitor data.
- FIG. 8 illustrates an example computing system 800 , according to at least one embodiment described in the present disclosure.
- the system 800 may include any suitable system, apparatus, or device configured to test software.
- the computing system 800 may include a processor 810 , a memory 820 , a data storage 830 , and a communication unit 840 , which all may be communicatively coupled.
- any of the network devices e.g., the edge network devices 110 , 210 , 310 , or 410 of FIGS. 1-4
- control devices e.g., the control devices 120 , 220 , 320 , or 420 of FIGS. 1-4
- local computing devices e.g., the local computing device 450 of FIG.
- network management devices e.g., the network management device 290 of FIG. 2
- other computing devices of the present disclosure may be implemented as the computing system 800 .
- one or more of the network devices, control devices, local computing devices or other computing devices may be implemented as virtualized machines operating on a physical computing system such as the computing system 800 .
- the processor 810 may include any suitable special-purpose or general-purpose computer, computing entity, or processing device including various computer hardware or software modules and may be configured to execute instructions stored on any applicable computer-readable storage media.
- the processor 810 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a Field-Programmable Gate Array (FPGA), or any other digital or analog circuitry configured to interpret and/or to execute program instructions and/or to process data.
- DSP digital signal processor
- ASIC application-specific integrated circuit
- FPGA Field-Programmable Gate Array
- the processor 810 may include any number of processors distributed across any number of network or physical locations that are configured to perform individually or collectively any number of operations described in the present disclosure.
- the processor 810 may interpret and/or execute program instructions and/or process data stored in the memory 820 , the data storage 830 , or the memory 820 and the data storage 830 .
- the processor 810 may fetch program instructions from the data storage 830 and load the program instructions into the memory 820 .
- the processor 810 may execute the program instructions, such as instructions to perform the methods 500 , 600 , and/or 700 FIGS. 5-7 , respectively. For example, the processor 810 may determine that a traffic flow is associated with a rerouting application and reroute the traffic flow along the path with the best performance score. As another example, the processor 810 may rewrite DNS queries and/or DNS replies. As an additional example, the processor 810 may route flows such that an NAT exit point associated with a rerouted path may be used. As an additional example, the processor 810 may determine which path from multiple paths is the best path and reroute traffic accordingly.
- the processor 810 may determine that a traffic flow is associated with a rerouting application and reroute the traffic flow along the path with the best performance score.
- the processor 810 may rewrite DNS queries and/or DNS replies.
- the processor 810 may route flows such that an NAT exit point associated with a rerouted path may be used.
- the processor 810 may determine which path from multiple paths
- the memory 820 and the data storage 830 may include computer-readable storage media or one or more computer-readable storage mediums for carrying or having computer-executable instructions or data structures stored thereon.
- Such computer-readable storage media may be any available media that may be accessed by a general-purpose or special-purpose computer, such as the processor 810 .
- the computing system 800 may or may not include either of the memory 820 and the data storage 830 .
- such computer-readable storage media may include non-transitory computer-readable storage media including Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid state memory devices), or any other storage medium which may be used to carry or store desired program code in the form of computer-executable instructions or data structures and which may be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer-readable storage media.
- Computer-executable instructions may include, for example, instructions and data configured to cause the processor 810 to perform a certain operation or group of operations.
- the communication unit 840 may include any component, device, system, or combination thereof that is configured to transmit or receive information over a network, such as an MPLS connection, the Internet, a cellular network (e.g., an LTE network), etc.
- a network such as an MPLS connection, the Internet, a cellular network (e.g., an LTE network), etc.
- the communication unit 840 may communicate with other devices at other locations, the same location, or even other components within the same system.
- the communication unit 840 may include a modem, a network card (wireless or wired), an optical communication device, an infrared communication device, a wireless communication device (such as an antenna), a chipset (such as a Bluetooth device, an 802.6 device (e.g., Metropolitan Area Network (MAN)), a WiFi device, a WiMax device, cellular communication facilities, or others), and/or the like, or any combinations thereof.
- the communication unit 840 may permit data to be exchanged with a network and/or any other devices or systems described in the present disclosure.
- the communication unit 840 may allow the system 800 to communicate with other systems, such as network devices, control devices, and/or other networks.
- the data storage 830 may be multiple different storage mediums located in multiple locations and accessed by the processor 810 through a network.
- embodiments described in the present disclosure may include the use of a special purpose or general purpose computer (e.g., the processor 810 of FIG. 8 ) including various computer hardware or software modules, as discussed in greater detail below. Further, as indicated above, embodiments described in the present disclosure may be implemented using computer-readable media (e.g., the memory 820 or data storage 830 of FIG. 8 ) for carrying or having computer-executable instructions or data structures stored thereon.
- a special purpose or general purpose computer e.g., the processor 810 of FIG. 8
- embodiments described in the present disclosure may be implemented using computer-readable media (e.g., the memory 820 or data storage 830 of FIG. 8 ) for carrying or having computer-executable instructions or data structures stored thereon.
- module or “component” may refer to specific hardware implementations configured to perform the actions of the module or component and/or software objects or software routines that may be stored on and/or executed by general purpose hardware (e.g., computer-readable media, processing devices, or some other hardware) of the computing system.
- general purpose hardware e.g., computer-readable media, processing devices, or some other hardware
- the different components, modules, engines, and services described in the present disclosure may be implemented as objects or processes that execute on the computing system (e.g., as separate threads). While some of the systems and methods described in the present disclosure are generally described as being implemented in software (stored on and/or executed by general purpose hardware), specific hardware implementations or a combination of software and specific hardware implementations are also possible and contemplated.
- a “computing entity” may be any computing system as previously defined in the present disclosure, or any module or combination of modulates running on a computing system.
- any disjunctive word or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms.
- the phrase “A or B” should be understood to include the possibilities of “A” or “B” or “A and B.”
- first,” “second,” “third,” etc. are not necessarily used herein to connote a specific order or number of elements.
- the terms “first,” “second,” “third,” etc. are used to distinguish between different elements as generic identifiers. Absence a showing that the terms “first,” “second,” “third,” etc., connote a specific order, these terms should not be understood to connote a specific order. Furthermore, absence a showing that the terms “first,” “second,” “third,” etc., connote a specific number of elements, these terms should not be understood to connote a specific number of elements.
- a first widget may be described as having a first side and a second widget may be described as having a second side.
- the use of the term “second side” with respect to the second widget may be to distinguish such side of the second widget from the “first side” of the first widget and not to connote that the second widget has two sides.
Abstract
Description
- This application claims priority to U.S. Patent App. No. 62/539,491, filed Jul. 31, 2017, which is hereby incorporated by reference in its entirety.
- The embodiments discussed in the present disclosure are related to a virtualized software-defined network.
- The use of networks is a useful tool in allowing communication between distinct computing devices. Despite the proliferation of computers and networks over which computers communicate, there still remains various limitations to current network technologies.
- The subject matter claimed in the present disclosure is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one example technology area where some embodiments described in the present disclosure may be practiced.
- One or more embodiments of the present disclosure may include a method that may include identifying a set of devices in a software-defined network (SDN). The method may include determining a set of characteristics that the set of devices in the SDN are to monitor within the SDN. The method may include generating a set of instructions for the set of devices in the SDN to monitor the set of characteristics. The method may further include sending the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path. The method may also include receiving monitor data via the control plane from at least one device of the set of devices in the SDN. The monitor data may be based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time. The method may include determining a change to a set of parameters of the SDN. The method may further include generating a policy, based on the change to the set of parameters of the SDN. The method may also include sending the policy to the set of devices in the SDN. An execution of the policy at one or more of the set of devices in the SDN adjusts the monitor data to more closely match a template.
- One or more embodiments of the present disclosure may additionally include systems and/or non-transitory computer readable media for facilitating the performance of such methods.
- The object and advantages of the embodiments will be realized and achieved at least by the elements, features, and combinations particularly pointed out in the claims.
- It is to be understood that both the foregoing general description and the following detailed description are merely examples and explanatory and are not restrictive of the invention, as claimed.
- Example embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
-
FIG. 1 illustrates an example system of network components implementing a software-defined network (SDN); -
FIG. 2 illustrates another example system implementing a SDN; -
FIG. 3 illustrates an additional example system as part of a SDN; -
FIG. 4 illustrates another example system implementing a SDN; -
FIG. 5 illustrates a flowchart of an example method to monitor a set of characteristics in a SDN; -
FIG. 6 illustrates a flowchart of an example method to generate a data model based on monitor data of a SDN; -
FIG. 7 illustrates a flowchart of an example method to generate a policy based on a data model and monitor data in a SDN; and -
FIG. 8 illustrates an example computing system. - Some embodiments of the present disclosure relate to improvements to the operation of software-defined networks (SDN) and routing of network traffic. In a SDN, a network control plane may be functionally separated from physical topology and a data plane, unlike conventional networking. For example, the SDN may include one or more virtualized components, such as a router that is run on a virtual machine, a hypervisor, or a cloud-based server.
- Over the control plane, a control device may monitor the SDN and path characteristics of the data plane tunnels between devices (e.g., routers). The control device may use this information to generate data models and generate policies. The policies may be sent to devices in the network to steer data traffic onto various links. Because the SDN extends a single virtual fabric over all available transports, paths or tunnels that satisfy the performance service level agreements (SLAs) for an application can be chosen for forwarding. Choice of transport can be influenced by cost, quality of service, service availability in specific geographies, circuit delivery times, operational processes, organizational policies, and regulations, such as in-country regulations and policies. Application-aware routing results in improved user experience and better bandwidth utilization.
- One or more embodiments of the present disclosure may include solutions to technology-based problems associated with software-defined networking. Embodiments of the present disclosure may provide improvements to computer networks and to the operation of computers themselves. For example, using one or more embodiments of the present disclosure, network traffic may flow with increased performance preserving valuable network resources such as bandwidth and providing increased response times. Additionally, the amount of traffic flowing through the internal network domain may be reduced, providing superior performance for the internal network domain. As another example, path availability may be guaranteed for a rerouted path, which may improve reliability for important applications. As an additional example, the performance of applications utilizing third party resources may be improved because a path with an optimal or improved performance may be used for the application, allowing for increased response times, increased data throughput per unit time, among others.
- Embodiments of the present disclosure are explained with reference to the accompanying drawings.
-
FIG. 1 illustrates anexample system 100 of network components implementing a software-defined network (SDN), in accordance with one or more embodiments of the present disclosure. The SDN may include any type of network or network topology. For example, the SDN may include a software-defined wide area network (SD-WAN), software-defined local area network (LAN), software-defined metropolitan area network (MAN), or any other type of network. Thesystem 100 may include aninternal network domain 105 and one or more external network domains. Thesystem 100 may include one or more edge network devices 110 (such as the edge network devices 110 a-110 d), acontrol device 120, acommunication network 130, and external network devices 140 and 141 (such as the external network devices 140 a-140 d and 141 a-141 d). - For ease and clarity in explanation, some examples of the present disclosure are described with respect to a WAN where the network is managed at least partially by software rather than controlled by hardware. As such, the SDN may support multiple types of connections or communication links, such as the Internet, MultiProtocol Label Switching (MPLS) connections, and/or cellular connections (such as Long Term Evolution (LTE), LTE Advanced, Worldwide Interoperability for Microwave Access (WiMAX), Evolved High Speed Packet Access (HSPA+), and/or others). Additionally, the SDN may support load balancing or load sharing between the various connections. Further, because of the distributed nature of some networks, the SDN may support virtual private networks (VPNs), firewalls, and other security services. In an SD-WAN, for example, a control plane may be functionally separated from the physical topology. In some embodiments, the SDN may separate the control plane of the network (to be managed via software) from a data plane of the network (operating on the hardware of the network). As used herein, the term control plane may refer to communications and connections used in the control and administration of a network itself, rather than the transmission of data through the network, which may occur at the data plane. The control plane may be used in conjunction with the collection of data from various components in the SDN. As used herein, the term data plane may refer to communications and connections used in the transmission and reception of data through the network. For example, the control plane may include administrative traffic directed to a network device within a network, while the data plane may include traffic that passes through network devices within the network. The control plane may be used to exchange data between the
control device 120 and various components in the SDN separately than traffic on the data plane. In at least one embodiment, data on the control plane may be isolated from traffic on the data plane. The isolation may be physical isolation (e.g., in different physical hardware and wires), or logical (e.g., isolated in theinternal network domain 105 via software). Isolation of the data on the control plane from traffic on the data plane may provide increased security for one or both of the data on the control plane and the traffic on the data plane. Data on the control plane may be exchanged between thecontrol device 120 and various components in the SDN at or near real-time. - In some embodiments, the
control device 120 may be configured to manage the control plane of aninternal network domain 105 by directing one or more aspects of the operation of the edge network devices 110. Thecontrol device 120 may direct data collection activities at various components of the SDN. For example, thecontrol device 120 may direct one or more of the edge network devices 110 to monitor various characteristics of the SDN, such as performance, SLA, security, etc. Monitored data include data associated with the monitored characteristics. The monitored data may include bandwidth data, usage data, carrier data, geography data, user data, application data, site data, loss, latency, jitter, cost data, network events, syslogs, among other data. Thecontrol device 120 may use one or more control messages to direct the data collection activities at the various components of the SDN. For example, thecontrol device 120 may generate a set of instructions for a set of devices in the SDN to monitor a set of characteristics. Thecontrol device 120 may send the set of instructions as a control message to the set of devices in the SDN via the control plane. - According to the set of instructions in the control message, the set of devices in the SDN may collect the monitor data. The devices in the system may send the monitor data securely over a data-bus in real-time to the
control device 120. As an example, with QoS monitoring of communication links, the edge network devices 110 may provide for one or more QoS metrics that may be monitored for any communication link, such as jitter, bandwidth, error rate, bit rate, throughput, and/or others. The various components of the SDN may send monitored data to thecontrol device 120 via the control plane. - The
control device 120 may generate data models and/or policies based at least in part on the monitor data collected from the various components of the SDN. Thecontrol device 120 may receive an input signal (e.g., input from a system administrator, a software-based trigger) to generate a data model in view of a set of input parameters. The input signal may include the set of input parameters. For example, for QoS monitoring of communication links, the set of input parameters may include various QoS characteristics and/or the QoS metrics (e.g., jitter, bandwidth, error rate, bit rate, throughput, and/or others). Thecontrol device 120 may generate the data model based on the set of input parameters and the monitor data. For example, thecontrol device 120 may apply machine learning algorithms to the monitored data in order to satisfy use-cases of network planning (e.g., forecasting), network operations (e.g., SLA policy recommendation, carrier selection), what-if analysis, and network security (anomaly detection). - For network planning and forecasting, the
control device 120 may, for example, use the monitor data and input parameters (e.g., bandwidth, apps, sites, alarm level) to provide a forecast of future bandwidth for apps and/or sites based on the monitor data (e.g., past usage) and the alarm level. For forecasting, for example, thecontrol device 120 may use one or more regression algorithms, such as regression algorithms directed to a generalized linear model with extreme gradient boost. - For network planning, the
control device 120 may, for example, use the monitor data and various input parameters (e.g., carrier details, time) for carrier selection. For example, thecontrol device 120 may provide a data model around one or more carriers (e.g., Comcast, AT&T, Verizon) to illustrate carrier performance to customers. The customers may use the data model in determining whether to provision new sites in a particular geographical area. The data model may provide guidance on which carrier(s) are the top performing carriers in that particular geographical area. - For network operations, the
control device 120 may, for example, use the monitor data for correlation and root cause analysis. As an example, thesystem 100 may include many (e.g., hundreds, thousands, hundreds of thousands, millions, etc.) of events coming from a SDN overlay network. In order to determine a real root cause, thecontrol device 120 may define categories of correlated events and then use network domain knowledge (e.g., the monitor data) to establish a cause and effect relationship between these correlated events. Thecontrol device 120 may then narrow the list to a set of potential root causes. In some embodiments, thecontrol device 120 may use an algorithm based on correlation with pearson correlation coefficient. - Additionally, for network operations, the
control device 120 may, for example, use the monitor data and input parameters to generate SLA policy recommendations. In at least some embodiments, thecontrol device 120 may estimate a cost function of making a change to thesystem 100. In at least some embodiments, thecontrol device 120 may generate an approximation of a sensitivity score that may represent how sensitive a process may be to changes in loss, latency, jitter relative to other programs. Thecontrol device 120 may group classes of devices where a relative importance of loss, latency, and jitter within the devices in a particular class may be fixed or relatively constant. The sensitivity score may be generated using various functions. For example, thecontrol device 120 may classify applications such as video applications into a first class and web application into a second class. The video applications may have a first sensitivity score that may be represented by the function of x+y+ẑ2. The web applications may have a second sensitivity score that may be represented by a second function, 2x+y+log z. Thecontrol device 120 may then use an algorithm (e.g., a random forest of regression trees, a neural net) and the monitor data (e.g., historical SLA data) to generate an estimate of how triggering an SLA policy may affect the SLA characteristics of thesystem 100. Thecontrol device 120 may use the output of the random forest algorithm and the estimated cost function as input to a linear regression function to compute a data model—an expected cost of different SLAs. In at least some embodiments, the function applied may include a convex function. Thecontrol device 120 may use the expected cost of different SLAs against the monitor data to generate a policy that can be executed on an edge network device 110. - The
control device 120 may also be used to generate various data models for “what-if” analyses. The “what if” analysis may be used to predict effects on thesystem 100 in response to various changes that may be made to thesystem 100. For example, a “what-if” data model may be used to predict how bandwidth needs may change in various scenarios, such as when new users are added to existing sites, new applications are added to existing sites, new sites are up with users and/or applications, etc. The “what-if” data model may also be used to provide an analysis on how SLA characteristics may change when some or all traffic is moved from a first tunnel to a second tunnel. Another aspect to moving applications from one tunnel to another includes network cost analysis, which may include a cost versus performance assessment. For example, a performance level of an applications on a broadband tunnel may be lesser than a performance of the application on an MPLS tunnel, but the cost savings of using the broadband tunnel may outweigh the increased performance that may be realized on the MPLS tunnel. - The
control device 120 may also be used to for network security, such as anomaly detection. Thecontrol device 120 may periodically analyze the monitor data for irregularities and/or anomalies. For example, thecontrol device 120 may use anomaly detection of applications at each site in the network for enforcing security related use-cases or to enforce improved planning. In at least some embodiments, thecontrol device 120 may use standard deviations and weighted moving averages. For example, a detection of the monitor data that is, for example, 2-3 standard deviations away from a moving average(baseline) may indicate an anomaly for normal distributions. Thecontrol device 120 may also use K-means for clustering. Any other suitable algorithm may be used. Other example anomalies that may be detected include: anomaly detection for bandwidth usage on a per application/site basis, anomaly detection for network events, and anomaly detection for network performance metrics of loss/latency/jitter on a per application/site basis. These different uses for anomaly detection may assist thecontrol device 120 and/or a network administrator to focus on relevant data points. These anomalous data points may also form a basis for various actions that may be taken within thesystem 100. In addition to being useful for security purposes, anomaly detection may also be used for network operations, such as optimizing network bandwidth, application performance, and network troubleshooting, among others. - The
control device 120 may cause an action pertaining to the SDN in view of the data model. For example, based on the monitor data and/or analysis of the monitor data and data model, thecontrol device 120 may generate a policy to affect various changes in the SDN. An execution of the policy at one or more of the set of devices in the SDN may adjust the monitor data to more closely match a template. The template may include a predetermined set of characteristics and/or outputs of the SDN. - The
control device 120 may distribute the policies to one or more of the edge network devices 110. A policy may include a rule or set of rules bearing on the handling of network traffic, such as routing, priority, media, etc. As an example, with SLAs, the policy may include edge a threshold level for one or more QoS metrics, such as bandwidth, availability, jitter, and/or others. In these and other embodiments, a policy on a given edge network device 110 may be configured to adjust or otherwise modify one or more properties of how the given edge network device 110 handles or routes traffic to better comply with one or more SLAs. For example, the traffic flow for one application may be sent via a particular communication link so that the traffic flow may comply with a corresponding SLA. - The
internal network domain 105 may operate as a secured and controlled domain with specific functionality and/or protocols. In some embodiments, the edge network devices 110 may operate based on one or more policies created and/or propagated by thecontrol device 120. In these and other embodiments, the edge network devices 110 may route data traffic within theinternal network domain 105 based on the policies created and/or propagated by thecontrol device 120. - In some embodiments, the
control device 120 may form a control plane connection with each of the edge network devices 110. The control plane connection may facilitate the exchange of data (e.g., various analytics data obtained by the edge network devices 110) between the edge network devices 110 and thecontrol device 120 for management and control of theinternal network domain 105. The control plane connection may be separate from a data transfer plane. For example, the control plane connection may operate via a tunnel through thecommunication network 130, such as a Datagram Transport Layer Security (DTLS) tunnel. In some embodiments, data transmitted over the control plane connection may facilitate thecontrol device 120 determining topology and other characteristics of thecommunication network 130. For example, thecontrol device 120 may communicate with the edge network devices 110 to determine what physical connections exist between and among the edge network devices 110 in thecommunication network 130. Additionally or alternatively, data transmitted over the control plane connection may facilitate thecontrol device 120 determining available, optimal or desired paths across thecommunication network 130 between and among the edge network devices 110. Additionally or alternatively, data transmitted over the control plane connection may facilitate the edge network devices 110 determining available, optimal or desired paths across thecommunication network 130 between and among the edge network devices 110. Additionally or alternatively, thecontrol device 120 may communicate information (e.g., control messages, sets of instructions, monitor data, policies) to and from the edge network devices 110 over the control plane connection. In these and other embodiments, the control plane connection may include a permanent connection between thecontrol device 120 and the edge network devices 110 such that if the connection between thecontrol device 120 and a given edge network device 110 is broken, the edge network device 110 may be unable or otherwise disallowed from communicating over theinternal network domain 105. - In some embodiments, the
control device 120 may maintain a central route table that stores route information within theinternal network domain 105. For example, thecontrol device 120 may communicate with various edge network devices 110 to determine the physical and/or logical connections available to the edge network devices 110 through thecommunication network 130. In some embodiments, the edge network devices 110 may include one or more physical and/or logical connections to each other. In these and other embodiments, thecontrol device 120 may generate and/or update one or more policies in conjunction with the monitor data and/or the central route table to help the edge network devices 110 to determine data traffic routes through theinternal network domain 105. For example, thecontrol device 120 may provide policies and other configuration preferences related to traffic flows to the edge network devices 110 rather than being involved with every individual flow through theinternal network domain 105. - In these and other embodiments, the edge network devices 110 may not have stored the topology and/or route paths of the
entire system 100. Each of the edge network devices 110 may not need to query each other individually to determine reachability. Instead, thecontrol device 120 may provide such information to the edge network devices 110. In these and other embodiments, the edge network devices 110 may route traffic through a most direct route, a most cost effective route, a most reliable route, or through some other route based on one or more other policies received from of thecontrol device 120, characteristics of the traffic, characteristics of the route path, a source edge network device 110, and a destination (e.g., edge network device 110). - In some embodiments, the one or more policies may include guidance regarding determining next-hop and route path instructions. For example, a particular policy may instruct a particular edge network device 110 where to route the traffic next for a particular category, class, or group of traffic flows, rather than providing a complete end-to-end route for the traffic. For example, the
edge network device 110 a may receive data from anexternal network device 140 a directed to an address of theexternal network device 141 c. Theedge network device 110 a may have stored a first policy that thenetwork device 110 a may use to determine the route path for the data, including that a “next-hop” for network traffic destined for the address of theexternal network device 141 c is to be routed to theedge network device 110 d. - In some embodiments, the
control device 120 may generate policies based at least in part on monitor data collected within the system and/or analytics performed on the monitor data to cause certain network traffic flows within theinternal network domain 105 to be routed over certain types of connections or communication links (e.g., LTE, Internet, MPLS) and/or through certain edge network devices 110. In some embodiments, a link classification may indicate a type of communication link. The edge network devices 110 may use a policy to determine an action. For example, for a given set of data, the policy may indicate that a first routing path may be selected based on a configuration preference and based on a type or classification of a communication link. Based on a policy, the edge network devices 110 may also determine an action (e.g., determine a route path) for all packets from a particular source, all packets that originated from a particular computer laptop, a type of traffic (e.g., voice), etc. In another example, a policy have a configuration preference that may indicate that all flows with IP addresses in the range 100.1/16 may be categorized as voice flows. When the edge network devices 110 is connected to three different communication links (e.g., an Internet link an MPLS link, a cellular link). The configuration preference may indicate that voice traffic is to be routed over the Internet link. Thus, the edge network devices 110 may route all flows with IP addresses in the range 100.1/16 over the Internet link. Other examples of configuration preferences may include costs (e.g., monetary, time, route, hops, transport health (such as loss, latency, and/or jitter), bandwidth, path geography, source (e.g., device, geography, user), destination (e.g., device, geography, user), applications (e.g., business, social), user groups (e.g., finance on a first subnet IP, HR on a second subnet IP, engineering on a third subnet IP, among others. Any type of data or information may be used as a configuration preference. In an example, the edge network device 110 may identify metadata associated with the traffic, such as a header. The header may include a DSCP value in the header to indicate a predefined routing path. The edge network device 110 may also make routing decisions based on a specific application (which may sometimes be referred to as a layer7 header or http header). For example, the edge network device 110 may route OFFICE365 traffic on a first path and SALESFORCE traffic on a second path. - In some embodiments, the
control device 120 may receive one or more keys from the edge network devices 110 used in communication of data over the data plane. For example, one or more data packets may use one or more keys for security purposes in transmitting data from one edge network device 110 to another edge network device 110. In these and other embodiments, thecontrol device 120 may reflect the received keys to one or more other edge network devices 110 that may be in the traffic flow based on the central route table and/or the policies implemented by thecontrol device 120. In these and other embodiments, a given edge network device 110 may generate symmetrical keys to facilitate secure communication between edge network devices. In these and other embodiments, a pair of symmetrical keys may be generated by the given edge network device 110, with one remaining with the given edge network device 110 and the other provided to thecontrol device 120 such that thecontrol device 120 may distribute the other symmetrical key to other edge network devices that communicate with the given edge network device 110. In such a way, each edge network device that is to communicate with the given edge network device 110 based on the policies of thecontrol device 120 may receive the symmetrical key. - In some embodiments, traffic within the
internal network domain 105 may be encrypted, such as with two-way authentication using Advanced Encryption Standard (AES) with a 256-bit length key over one or more Datagram Transport Layer Security (DTLS) and/or Transport Layer Security (TLS) connections between edge network devices 110. - In some embodiments, the
control device 120 may store authentication information for one or more (or all) of the edge network devices 110 within theinternal network domain 105. In these and other embodiments, a device may be prevented from communicating within theinternal network domain 105 unless the device has authentication information that matches or otherwise corresponds to the stored authentication information of thecontrol device 120. In some embodiments, the authentication information may be used when the edge network devices 110 first come on line to establish the control plane connection, and any device without a control plane connection with thecontrol device 120 may be prevented from communicating within theinternal network domain 105. - The edge network devices 110 may operate at a boundary of the
internal network domain 105. The edge network devices 110 may include one or more physical and/or logical connections that may operate within theinternal network domain 105. Such connections may be illustrated as part of thecommunication network 130. Additionally or alternatively, the edge network devices 110 may include one or more physical and/or logical connections operating outside of theinternal network domain 105. For example, the edge network devices 110 may be connected to the external network device(s) 140 and/or 141. - In some embodiments, the edge network devices 110 may operate to route traffic from associated external network devices 140 and 141 into the
internal network domain 105. Additionally or alternatively, the edge network devices 110 may operate to route traffic from theinternal network domain 105 to the associated external network devices 140 and 141. In some embodiments, the edge network devices 110 may communicate with associated external network devices 140 and 141 using typical communication protocols, such as Open Shortest Path First (OSPF), Border Gateway Protocol (BGP), Virtual Router Redundancy Protocol (VRRP), Bi-directional Forwarding Detection (BFD), among others. Additionally or alternatively, the edge network devices 110 may support other network functionalities such as differentiated services code point (DSCP) tagging or type of service (TOS) tagging, Quality of Service (QoS) monitoring, Service Level Agreements (SLA), Internet Protocol (IP) forwarding, Internet Protocol Security (IPsec), Access Control Lists (ACL), among others. - For example, with DSCP or TOS tagging, the edge network devices 110 may be configured to insert a DSCP or TOS tag into a packet header. Such a DSCP or TOS tag may identify one (or a preferences for one) communication link of multiple communication links on which to send certain types of network traffic. Based on the DSCP or TOS tag, the edge network devices 110 may route the network traffic via one or more types of communication link.
- As another example, with IP forwarding, the edge network devices 110 may include one or more policies to route packets via a particular type of communication link in an 1P network. For example, such a policy may take into account factors such as packet size, services specified by a header, characteristics of potential links to other routers in the network, and/or others. Using such factors, the edge network devices 110 may forward packets based on a selected algorithm, such as a shortest path.
- As an additional example, with IPsec, the edge network devices 110 may use IPsec to authenticate and/or encrypt network traffic. For example, a given edge network device 110 may authenticate one or more computing devices to communicate with the given edge network device 110 and/or encrypt one or more packets communicated between the computing device and the given edge network device 110.
- As another example, with ACLs, the edge network devices 110 may include a set of rules indicative of one or more addresses, hosts, and/or networks that may be permitted to use a given port or a particular communication link. In these and other embodiments, the edge network devices 110 may include ACLs that are applicable to inbound traffic, outbound traffic, or both.
- In some embodiments, the edge network devices 110 may locally maintain one or more route tables. In some embodiments, the edge network devices 110 may adjust or modify the route tables based on one or more policies sent from the
control device 120. For example, one or more entries may be removed, discarded, or otherwise not added to the route tables by the edge network devices 110 based on the one or more policies. In some embodiments, the edge network devices 110 may include logic to update, modify, and/or generate the route tables based on policies from thecontrol device 120 and/or from traffic handled by the edge network devices 110. The one or more route tables may be automatically populated by the edge network devices 110 based on direct interface routes, static routes, and/or dynamic routes learned using one or more network protocols such as BGP and/or OSPF. In some embodiments, routing decisions for data outside of theinternal network domain 105 may be performed by a particular edge network device 110 without specific direction, input, or control from thecontrol device 120. For example, the particular edge network device 110 may compute a routing decision based on the one or more policies that the particular edge network device 110 has received from thecontrol device 120. - In some embodiments, one or more of the edge network devices 110 and/or the
control device 120 may be implemented as one or more virtual machines operating on one or more physical computing devices. Additionally or alternatively, the edge network devices 110 and/or thecontrol device 120 may each include an individual stand-alone computing device. - Modifications, additions, or omissions may be made to
FIG. 1 without departing from the scope of the present disclosure. For example, while illustrated as including four edge network devices 110 and onecontrol device 120, thesystem 100 may include any number of edge network devices 110 andcontrol devices 120, such as thousands or tens of thousands of edge network devices 110 and more than fivecontrol devices 120. As another example, as illustrated as asingle communication network 130, thecommunication network 130 may include multiple types of communication connections. -
FIG. 2 illustrates anotherexample system 200 of network components implementing a SDN, in accordance with one or more embodiments of the present disclosure. Thesystem 200 may include one or more edge network devices 210 (such as edge network devices 210 a-210 o), one or more control devices 220 (such ascontrol devices communication networks FIG. 1 , the control devices 220 may be similar or comparable to thecontrol device 120 ofFIG. 1 , and the communication networks 230 may be similar or comparable to thecommunication network 130 ofFIG. 1 . Thesystem 200 may be a similar or comparable system to thesystem 100 ofFIG. 1 , although expanded to include additional network components and additional external network domains. - The
system 200 may include aninternal network domain 205 in and between the edge network devices 210, in a similar or comparable manner to that described with respect to thesystem 100 ofFIG. 1 . Thesystem 200 additionally may include multiple external network domains. For example, adata center 240 may represent a first external network domain, acampus 250 may represent a second external network domain, abranch 260 may represent a third external network domain, and aremote site 270 may represent a fourth external network domain. In these and other embodiments, each external network domain may include one or more edge network devices 210 acting as a bridge between theinternal network domain 205 and the given external network domain. Additionally or alternatively, one or more of the external network domains may functionally operate as being accessible from the other external network domains as though in a single network by being communicatively coupled through theinternal network domain 205. - In some embodiments, the
system 200 may include one or more external resources 280 (such as the external resources 280 a-280 c). The external resources 280 may be operated by the same entity or organization that operates theinternal network domain 205, or may be operated by a different entity. In these and other embodiments, thesystem 200 may include an edge network device 210 that may be associated with a particular external resource 280. For example, thesystem 200 may include an edge network device 210 located within a regional co-location facility. A regional co-location facility may include a location with directed or guaranteed access to the Internet or other communication protocols at a given physical location. In some embodiments, a regional co-location facility may include a prioritized or improved connection to one or more of the external resources 280. In some embodiments, the regional co-location facility may be at a designated geographical location that may be physically proximate one or more of the external network domains. For example, thedata center 240 may be located in New York, and thebranch 260 may be located in Dallas Tex., and theedge network device 210 n may be in a regional co-location facility in Houston, Tex. - The external resources 280 may include any computing service available for consumption by the
system 200. For example, the external resources 280 may include a cloud-based service such as a software subscription or software as a service (SaaS) (such as Microsoft Office 365®, Azure®, Google Apps®, Workforce®, Amazon Web Services®, WorkDay®, DocuSign®, GoToMeeting®, WebEx®, QuickBooks®, and/or others), media services (such as YouTube®, NetFlix®, Pandora®, Spotify®, and/or others), and/or others. In these and other embodiments, the external resources 280 may include a third party network to facilitate access to the external resource 280 with one or more access points at various geographical locations. For example, a SaaS may include an access server in Austin, Tex.; Palo Alto, Calif.; and New York, N.Y. for accessing the third party network. - In some embodiments, the
system 200 may be geographically distributed. For example, thedata center 240 may be located in St. Paul, Minn.; thecampus 250 may be located in Des Moines, Iowa; there may bebranches 260 in Seattle, Wash.; Los Angeles, Calif.; Atlanta, Ga.; and Orlando, Fla.; and there may beremote sites 270 in London, England; Berlin, Germany; and Seoul, Korea. In these and other embodiments, thesystem 200 may use the communication networks 230 and theinternal network domain 205 to facilitate communication between all of these distributed physical locations as a single network. - In some embodiments, one or more of the external network domains may use one or more applications with resources in the
data center 240, such as Microsoft Exchange®, SharePoint®, Oracle e-Business Suite®, and/or others. For example, a workstation operating at thecampus 250 may operate Microsoft Exchange®. The operation of the application may include a data flow that goes from the workstation to theedge network device 210 e in the external network domain of thecampus 250. The data flow may go from theedge network device 210 e to one of theedge network devices data center 240 through theinternal network domain 205. The one of theedge network devices data center 240. Additionally or alternatively, the operation of the application may include a data flow in the reverse order of data flowing from the Microsoft Exchange® server to the workstation. - In some embodiments, the
system 200 may include anetwork management device 290 that may communicate with the control devices 220 over amanagement network 232. Thenetwork management device 290 may provide management and control of one or more devices associated with theinternal network domain 205, including the edge network devices 210, the control devices 220, and/or others. For example, thenetwork management device 290 may provide a graphical user interface (GUI) that provides a network administrator with access to control or observe operation of theinternal network domain 205. In some embodiments, the network administrator may input policies via thenetwork management device 290 that may be communicated to the control devices 220 for implementation via the edge network devices 210. In some embodiments, thenetwork management device 290 may provide a GUI dashboard with a visual and/or textual description of one or more properties of theinternal network domain 205, such as a number and/or status and/or health of edge network devices 210, a number and/or status of control devices 220, a number of and/or last time of reboot, transport health (such as loss, latency, and/or jitter), a number of sites that are operating or not operating, application consumption of network resources, application routing, and/or others. - In some embodiments, the
network management device 290 may be configured to recognize approved edge network devices 210 and/or control device 220. For example, thenetwork management device 290 may maintain a list of serial numbers, MAC addresses, or other uniquely identifying information for the edge network devices 210 and/or the control devices 220. In these and other embodiments, communication in theinternal network domain 205 may be restricted to edge network devices 210 and/or control devices 220 with identifying information on the list maintained by thenetwork management device 290. - In some embodiments, the
network management device 290 may be configured to generate and/or store configurations of one or more edge network devices 210 and/or control devices 220. For example, a network administrator may use thenetwork management device 290 to configure a particular edge network device 210 and may store that configuration as a template that may be applied to future edge network devices. Additionally or alternatively, a template for the edge network devices 210 may be provided by a third party and applied to a new edge network device 210. In these and other embodiments, a template for the control devices 220 may be generated, stored, and/or applied to a new control device 220. Additionally or alternatively, such a template may be used to automatically configure a newly deployed edge network device 210. For example, the newly deployed edge network device 210 may be brought online and connected to a corresponding control device 220. The corresponding control device 220 may verify the serial number of the edge network device 210 with thenetwork management device 290, and may obtain a template from thenetwork management device 290 for the edge network device 210. The control device 220 may send the template to the edge network device 210 to be automatically installed to configure the edge network device 210 according to the template. - In at least some embodiments, the
network management device 290 may be configured to manage data collection activities at various components of the SDN. For example, thenetwork management device 290 may direct one or more of the edge network devices 110 to monitor various characteristics of the SDN, such as performance, SLA, security, etc. in a system with multiple control devices 220. - The
network management device 290 may use one or more control messages to direct the data collection activities at the various components of the SDN. For example, thenetwork management device 290 may generate a set of instructions for a set of devices in the SDN to monitor a set of characteristics. Thenetwork management device 290 may send the set of instructions as a control message to the set of devices in the SDN via the control plane. In at least some embodiments, thenetwork management device 290 sends the control message to the control devices 220 and the control devices 220 may distribute the control message to devices that are connect to the respective control device 220. - Following the set of instructions in the control message, the set of devices in the SDN may collect the monitor data. The devices in the system may send the monitor data securely over a data-bus (e.g., the control plane) in real-time to the
network management device 290, such as directly to thenetwork management device 290 or indirectly, such as via the control devices 220. As an example, with QoS monitoring of communication links, the edge network devices 110 may provide for one or more QoS metrics that may be monitored for any communication link, such as jitter, bandwidth, error rate, bit rate, throughput, and/or others. The various components of the SDN may send monitored data to thenetwork management device 290 via the control plane. - The
network management device 290 may generate data models and/or policies based at least in part on the monitor data collected from the various components of the SDN. Thenetwork management device 290 may receive an input signal (e.g., input from a system administrator, a software-based trigger) to generate a data model in view of a set of input parameters. Thenetwork management device 290 may generate the data model based on the set of input parameters and the monitor data. For example, thenetwork management device 290 may apply machine learning algorithms to the monitored data in order to satisfy use-cases of network planning (e.g., forecasting), network operations (e.g., SLA policy recommendation, carrier selection), what-if analysis, and network security (anomaly detection). Based on the monitor data and/or analysis of the monitor data and data model, thenetwork management device 290 may generate a policy to affect various changes in the SDN. Thenetwork management device 290 may distribute the policies to one or more of the edge network devices 110. - In at least some embodiments, the control devices 220 may generate and communicate control messages and/or policies independently of the
network management device 290 and the other control devices 220, such as is described in conjunction withFIG. 1 . - In some embodiments, the
network management device 290 may be implemented as a physical device or a virtualized machine. In these and other embodiments, thenetwork management device 290 may be physically located proximate a centralized location, such as within thedata center 240 or at thecampus 250. - Modifications, additions, or omissions may be made to
FIG. 2 without departing from the scope of the present disclosure. For example, while illustrated as including a certain number of edge network devices 210 and external network domains, thesystem 200 may include any number of edge network devices 210 and external network domains. -
FIG. 3 illustrates anadditional example system 300, in accordance with one or more embodiments of the present disclosure.FIG. 3 illustrates anedge network device 310 a that may include multiple potential communication links for communicating across aninternal network domain 305 to anotheredge network device 310 b. For example, theedge network device 310 a may communicate across theinternal network domain 305 using theInternet 360, anMPLS network 370, and/or anLTE network 380. Theedge network devices FIG. 1 and/or the edge network devices 210 a-210 o ofFIG. 2 . Thesystem 300 may additionally include an externallocal device 350 that may be communicatively coupled to theedge network device 310 a across an external network domain. - In some embodiments, the
edge network device 310 a may include anInternet connection 320, anMPLS connection 330, and anLTE connection 340. As illustrated by the ellipses below theLTE connection 340, any number of additional or other potential connections may also be included. In these and other embodiments, theedge network device 310 a may include multiple circuits for connecting to the one or more potential communication links. For example, theedge network device 310 a may include acircuit A 322 and acircuit B 324 for theInternet connection 320, acircuit A 332 and acircuit B 334 for theMPLS connection 330, and acircuit A 342 and acircuit B 344 for theLTE connection 340. In these and other embodiments, theedge network device 310 a may be configured to route traffic along one or more of the circuits, based on one or more policies stored by theedge network device 310 a. - In at least some embodiments, the control plane may be implemented on one or more of the
circuits MPLS circuit A 332 and, to achieve isolation of the control plane from the data plane, the data plane maybe implemented on any of theother circuits - In some embodiments, the
edge network device 310 a may be configured to monitor one or more properties of the various connections. For example, theedge network device 310 a may monitor the jitter, latency, loss, and/or bandwidth of the various communication links from theedge network device 310 a to theedge network device 310 b. In these and other embodiments, theedge network device 310 a may also monitor and/or store security properties of the various communication links. For example,links Internet 360 may be considered at a first level of security, andlinks MPLS network 370 may be considered at a second level of security higher than the first level of security. - In some embodiments, the
edge network device 310 a may route traffic for one or more applications to specific circuits based on one or more policies and/or based on one or more properties of the various connections. For example, a video application may be particularly susceptible to jitter. Theedge network device 310 a may determine that the video traffic may be travelling across thelink 382 with a jitter of 10 ms, and that thelink 362 may have a jitter of 4 ms. Theedge network device 310 a may shift the traffic for the video application to thelink 362 rather than thelink 382 because of the lower jitter. In some embodiments, shifting from thelink 382 to thelink 362 may be based on a jitter-based SLA. As another example, theedge network device 310 a may receive a data flow for a security-sensitive application (such as an accounting application) and may have a policy that data for that application is to be routed along one of the MPLS links 372 and/or 374, even if other traffic may be routed along theInternet link 362. As an additional example, theedge network device 310 a may include an SLA that a given application have a bandwidth of 10 MB/s available to the application. Theedge network device 310 a may make thelink 362 over theInternet 360 available to the application, but thelink 362 may provide 5 MB/s of bandwidth. Theedge network device 310 a may also provide thelinks links edge network device 310 a may route traffic for a given application via a particular type of communication link. In at least one embodiment, theedge network device 310 a may prohibit traffic for a given application along a particular type of communication link. For example, traffic associated with a social network or video application may be routed over theInternet connections MPLS connections edge network device 310 a may be configured to perform such routing based on initially receiving a data flow, during an on-going data flow (e.g., in fast path), based on a triggering event of the data flow, and/or others or combinations thereof. Additionally or alternatively, such routing may combine multiple links of multiple types of connections for a single flow in routing traffic flows. - In some embodiments, the
edge network device 310 a may be configured to route traffic to the various links based on the source of the traffic. For example, one or more policies may indicate that traffic from one corporate department of a business is routed along theMPLS connection 330, while traffic for another corporate department may be routed along any link. - In some embodiments, the
edge network device 310 a may be implemented as a computing system, such as thecomputing system 600 illustrated inFIG. 6 . - Modifications, additions, or omissions may be made to
FIG. 3 without departing from the scope of the present disclosure. For example, while illustrated as including a certain number of edge network devices 310, thesystem 300 may include any number of edge network devices 310. As another example, while illustrated as including three communication networks (theInternet 360, the MPLS-basednetwork 370, and the LTE network 380) any number of communication networks may be used. -
FIG. 4 illustrates anotherexample system 400 implementing a SDN, in accordance with one or more embodiments of the present disclosure. Thesystem 400 may include one or more edge network devices 410 (such as the edge network devices 410 a-410 f), which may be similar or comparable to the edge network devices 110 ofFIG. 1, 210 ofFIG. 2 , and/or 310 ofFIG. 3 . Thesystem 400 may also include one ormore control devices 420, which may be similar or comparable to thecontrol device 120 ofFIG. 1 , and/or 220 ofFIG. 2 . Thesystem 400 may additionally include one ormore communication networks 430 and/or 432 (such as the communication networks 432 a-432 c), which may be similar or comparable to thecommunication network 130 ofFIG. 1, 230 ofFIG. 2 , and/or the combination of any of theInternet 360, theMPLS network 370, and theLTE network 380 ofFIG. 3 . The system may additionally include adata center 440, which may be similar or comparable to thedata center 240 ofFIG. 2 . The system may also include one or more third party resources 480 (such as the third party resources 480 a-480 c), which may be similar or comparable to the third party resources 280 a-c ofFIG. 2 . For the purposes of discussingFIG. 4 , the third party resources 480 a-480 c may serve the same third party resource and may represent distinct servers for accessing the third party resource. For example, thethird party resource 480 a may include a server for accessing a cloud based service in Seattle, Wash., thethird party resource 480 b may include a server for accessing the cloud based service in Los Angeles, Calif., and thethird party resource 480 c may include a server for accessing the cloud based service in New York, N.Y. - In some embodiments, the
edge network device 410 a may determine which path a traffic flow may take based on a policy. For example, theedge network device 410 a may identify metadata associated with the traffic flow and determine, based on the policy and metadata, a configuration preference in the policy for the data to use routing paths with a first link classification. Theedge network device 410 a may select a first routing path based on the configuration preference in the policy and based on the first routing path including the first communication link associated with the first link classification. Theedge network device 410 a may transmit the data along the first routing path via the first communication link. In at least one embodiment, there may be multiple paths to a particular destination via different edge network device connections to the same underlying transport (e.g., Internet or MPLS). Alternatively or additionally, there may be multiple paths via the same two sets of edge network devices connected via different underlying transport. For example a first underlying transport may include Internet and a second underlying transport may include MPLS. - In some embodiments, each of the edge network devices 410 may assess the performance of paths between a given edge network device 410 and the other edge network devices 410. For example, the
edge network device 410 a may monitor the performance of thepaths edge network device 410 b may monitor the performance of thepaths internal network domain 405. - In some embodiments, one or more of the edge network devices 410 may communicate the determined performance metrics with one or more components of the
system 400. For example, the edge network devices 410 may communicate the performance metrics to thecontrol device 420, and thecontrol device 420 may distribute the performance metrics to one or more of the edge network devices 410. As another example, the edge network devices 410 may communicate the performance metrics to one or more other edge network devices 410 (e.g., theedge network device 410 b may communicate the performance metrics for thepaths edge network device 410 a). Thecontrol device 420 may determine and/or updated a policy based at least in part on the performance metrics. - In some embodiments, one or more of the edge network devices 410 may assess the performance of paths between a given edge network device 410 and one or more connections to the third party resource 480. For example, the
edge network devices 410 e and/or 410 f may monitor the performance of thepath 491, theedge network device 410 c may monitor the performance of thepath 492, and theedge network device 410 d may monitor the performance of thepath 493. In these and other embodiments, the edge network devices 410 may monitor one or more of jitter, latency, loss, and/or bandwidth of the various paths. For example, one or more requests may be communicated from the edge network devices 410 to the third party resource 480 and characteristics of the travel time and/or integrity of the response to the request may be used to determine the performance metrics of the paths. For example, the edge network devices 410 may use httping, or some other similar tool. In some embodiments, one or more of the performance metrics may be combined into a single score reflecting the performance of the path outside of theinternal network domain 405. - In some embodiments, the edge network devices 410 may maintain a table, database, or other storage structure of the scores of the performance metrics of the various paths in the
system 400. In these and other embodiments, the edge network devices 410 may use the stored scores when determining routing paths for data. For example, theedge network device 410 a may store a table with a single score for each of the paths in thesystem 400. - In some embodiments, the
edge network device 410 a may compare scores of the potential paths to the third party resource 480 to determine which path the rerouted traffic may flow along. For example, theedge network device 410 a may compare the combined scores of the paths 461+491, 462+491, 465+493, 466+492, 467+493, and 468+492. In these and other embodiments, the edge network device 410 may determine which score represents the best performance for the traffic. - In some embodiments, the
internal network domain 405 may include multiple possible paths between two edge network devices 410. For example, thepath 465 between theedge network device 410 a and theedge network device 410 d may represent an MPLS connection, and a second connection (not illustrated) between theedge network device 410 a and theedge network device 410 d may include an Internet or cellular connection. In these and other embodiments, each path, including multiple paths between the same two edge network devices 410, may each include a unique score generated by thecontrol device 420. Using such unique scores, thecontrol device 420 may determine which path to be used and generate and distribute a policy accordingly. - In some embodiments, if multiple paths have the same score representing the best score for routing data, based on the policy the
edge network device 410 a may route the traffic along the multiple paths with the best score. For example, a first set of the data may be routed along the first path and a second set of the data may be routed along a second path with the same score as the first path. In determining whether to route along the first path or the second path, theedge network device 410 a may hash the header of a packet within the flow and, depending on the output of the hash, may route the flow to one of the first path or the second path. While described as the path or paths with the best score, the path with a score relative to a threshold may also be selected. - In some embodiments, the policy may designate a primary path and a backup path for the rerouted path. The
edge network device 410 a may monitor the performance of the primary path of the rerouted path and, based on changes in the score for the primary path and the policy, theedge network device 410 a may reroute the traffic to the backup path or a different path. In some embodiments, the score may be monitored and/or rerouted relative to an SLA. In at least one embodiment, the primary path may be determined based on the configuration preference. - Modifications, additions, or omissions may be made to
FIG. 4 without departing from the scope of the present disclosure. For example, while illustrated as including a certain number of edge network devices 410, thesystem 400 may include any number of edge network devices 410. As another example, while illustrated as including a single path between any two edge network devices 410, any number of paths over any number of mediums may be included between edge network devices 410. -
FIGS. 5-7 illustrates a flowchart of anexample method 500 of monitoring, analyzing and taking action with respect to a SDN, in accordance with one or more embodiments of the present disclosure. The method may be performed by processing logic that may include hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both, which processing logic may be included in the any of the network devices (e.g., thecontrol devices FIGS. 1, 2 and 4 , and/or thenetwork management device 290 ofFIG. 2 ), or another computer system or device. However, another system, or combination of systems, may be used to perform the methods. For simplicity of explanation, methods described herein are depicted and described as a series of acts. However, acts in accordance with this disclosure may occur in various orders and/or concurrently, and with other acts not presented and described herein. Further, not all illustrated acts may be used to implement the methods in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the methods may alternatively be represented as a series of interrelated states via a state diagram or events. Additionally, the methods disclosed in this specification are capable of being stored on an article of manufacture, such as a non-transitory computer-readable medium, to facilitate transporting and transferring such methods to computing devices. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from any computer-readable device or storage media. Although illustrated as discrete blocks, various blocks may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. -
FIG. 5 illustrates a flowchart of anexample method 500 to monitor a set of characteristics in a SDN. Themethod 500 may begin atblock 505, where the processing logic may identify a set of devices in a software-defined network (SDN). At least one device of the set of devices in the SDN may include a virtual device. The set of devices include any type or location of devices, such as one or more edge network devices 110 (such as the edge network devices 110 a-110 d), acontrol device 120, acommunication network 130, and external network devices 140 and 141 (such as the external network devices 140 a-140 d and 141 a-141 d), described in more detail in conjunction withFIG. 1 or any other device or connection described in conjunction withFIGS. 2-4 . - At
block 510, the processing logic may determine a set of characteristics that the set of devices in the SDN are to monitor within the SDN. The processing logic may determine the set of characteristics based on input received from a system administrator, and/or in response to machine-based input (e.g., in response to machine learned characteristics to monitor). In at least some embodiments, determining the set of characteristics that the set of devices in the SDN are to monitor within the SDN includes determining a first set of characteristics for a first device type and determining a second set of characteristics for a second device type. For example, the first set of characteristics may be related to an edge device and the second set of characteristics may be related to a circuit. In at least some embodiments, determining the set of characteristics that the set of devices in the SDN are to monitor within the SDN includes determining to monitor at least one of: an anomaly, performance data, location data, payload data, carrier data, data related to an application, and network characteristics. In at least some embodiments, determining the set of characteristics that the set of devices in the SDN are to monitor within the SDN includes selecting a subset of devices of the set of devices in the SDN. For example, the processing logic may select devices of a certain type, or a subset of devices within a given type. - At
block 515, the processing logic may generate a set of instructions for the set of devices in the SDN to monitor the set of characteristics. In at least some embodiments, generating the set of instructions for the set of devices in the SDN to monitor the set of characteristics may include generating the set of instructions for a subset of devices. - At
block 520, the processing logic may send the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path. In at least some embodiments, sending the set of instructions as the control message to the set of devices in the SDN via the control plane that is isolated from the packet forwarding path may include sending a first set of characteristics to one or more devices of a first device type and sending a second set of characteristics to one or more devices of a second device type. In at least some embodiments, sending the set of instructions as the control message to the set of devices in the SDN via the control plane that is isolated from the packet forwarding path may include sending the set of instructions as the control message to a subset of devices via the control plane that is isolated from the packet forwarding path. - At
block 525, the processing logic may receive monitor data via the control plane from at least one device of the set of devices in the SDN. The monitor data may be based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time. In at least some embodiments, the monitor data received via the control plane is encrypted. The processing logic may decrypt such encrypted monitor data. - At
block 530, the processing logic may determine a change to a set of parameters of the SDN. In at least some embodiments, determining the change to the set of parameters of the SDN may include comparing the monitor data with a template and determining at least one difference between the monitor data and the template. The change to the set of parameters of the SDN is determined to adjust the monitor data to more closely match the template. The template may include a predetermined set of characteristics that set of devices in the SDN are to have. The template may be defined by a system administrator, an SLA, etc. - At
block 535, the processing logic may generate a policy, based on the change to the set of parameters of the SDN. An execution of the policy at one or more of the set of devices in the SDN may adjust the monitor data to more closely match the template - At
block 540, the processing logic may send the policy to the set of devices in the SDN. In at least some embodiments, the processing logic may send the policy to the set of devices in the SDN via the control plane. - One skilled in the art will appreciate that, for these processes, operations, and methods, the functions and/or operations performed may be implemented in differing order. Further, the outlined functions and operations are only provided as examples, and some of the functions and operations may be optional, combined into fewer functions and operations, or expanded into additional functions and operations without detracting from the essence of the disclosed embodiments.
-
FIG. 6 illustrates a flowchart of anexample method 600 to generate a data model based on monitor data of a SDN. Themethod 600 may begin atblock 605, where the processing logic may generate a set of instructions for a set of devices in a software-defined network (SDN) to monitor a set of characteristics, as further described in conjunction withFIG. 5 . - At
block 610, the processing logic may send the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path, as further described in conjunction withFIG. 5 . - At
block 615, the processing logic may receive monitor data via the control plane from at least one device of the set of devices in the SDN, the monitor data being based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time. - At
block 620, the processing logic may receive an input signal to generate a data model in view of a set of input parameters. The input signal may be received from a system administrator via a user interface, or may be generated based on machine-learned activities. - At
block 625, the processing logic may generate the data model based on the set of input parameters and the monitor data. In at least some embodiments, the data model may pertain to a network planning for the SDN based on the set of input parameters and the monitor data. In at least some embodiments, the data model may pertain to one or more carriers of the packet forwarding path in a particular geographical region. In at least some embodiments, the data model may pertain network security in the SDN. In at least some embodiments, generating the data model based on the set of input parameters and the monitor data may include determining, based on the set of input parameters, at least one network security characteristic in view of the monitor data. In at least some embodiments, the data model may pertain to a what-if analysis for at least one of: adding a new user to an existing site in the SDN, adding a new application to an existing site in the SDN, adding a new site to the SDN. - At
block 630, the processing logic may cause an action pertaining to the SDN in view of the data model. In at least some embodiments, causing the action pertaining to the SDN in view of the data model may include generating a policy based on the data model and sending the policy to the set of devices in the SDN via the control plane. In at least some embodiments, the policy may include a dynamic service level agreement (SLA) policy. The data model may include a model for how SLA characteristics may be affected in response to a change to the SDN. For example, the data model may include a traffic model for when traffic may be moved from one tunnel to another tunnel. An execution of the policy at one or more of the set of devices in the SDN adjusts the monitor data to more closely match a template, such as the template further described in conjunction withFIG. 5 . In at least some embodiments, generating the policy based on the data model may include determining a change to a set of operation parameters of the SDN by comparing the monitor data with a baseline set of activity data and determining at least one difference between the monitor data and the baseline set of activity data. The change to the set of parameters of the SDN is determined by the processing logic to adjust the monitor data to more closely match the template. In at least some embodiments, causing the action pertaining to the SDN in view of the data model may include selecting one carrier of one or more carriers to handle the packet forwarding path. In at least some embodiments, causing the action pertaining to the SDN in view of the data model may include generating a parameter recommendation for the policy based on the data model. In at least some embodiments, the parameter recommendation for the policy is generated based on at least one network characteristic. The at least one network characteristic may include at least one of a loss, a latency, and a jitter. - One skilled in the art will appreciate that, for these processes, operations, and methods, the functions and/or operations performed may be implemented in differing order. Further, the outlined functions and operations are only provided as examples, and some of the functions and operations may be optional, combined into fewer functions and operations, or expanded into additional functions and operations without detracting from the essence of the disclosed embodiments.
-
FIG. 7 illustrates a flowchart of an example method 700 to generate a policy based on a data model and monitor data in a SDN. The method 700 may begin atblock 705, where the processing logic may generate a set of instructions for a set of devices in a software-defined network (SDN) to monitor a set of characteristics. - At
block 710, the processing logic may send the set of instructions as a control message to the set of devices in the SDN via a control plane that is isolated from a packet forwarding path. - At
block 715, the processing logic may receive monitor data via the control plane from at least one device of the set of devices in the SDN, the monitor data being based on the at least one device of the set of devices in the SDN monitoring of the set of characteristics substantially in real-time. - At
block 720, the processing logic may generate a data model based on a set of SDN parameters and the monitor data. In at least some embodiments, generating the data model based on the set of SDN parameters and the monitor data includes generating an approximation of a sensitivity score for a process. The sensitivity score for the process is based on how sensitive the process may be to changes in loss, latency and jitter. The processing logic may also generate an estimate on how the change may affect the set of devices in the SDN. In at least some embodiments, generating the policy, based on the change for at least one device of the set of devices in the SDN may include generating the policy to reduce the effect on the set of devices in the SDN below a threshold level. In at least some embodiments, generating the estimate on how the change may affect the set of devices in the SDN may include using a random forest algorithm and historical data to estimate on how the change may affect the set of devices in the SDN. In at least some embodiments, generating the data model based on the set of SDN parameters and the monitor data further may include generating the data model based on an output of the random forest algorithm and an estimated cost function to determine expected cost of different SLA. - At
block 725, the processing logic may determine a change for at least one device of the set of devices in the SDN based on the data model. In at least some embodiments, the change for at least one device of the set of devices in the SDN includes a change to at least one data route path. - At
block 730, the processing logic may generate a policy, based on the change for at least one device of the set of devices in the SDN. - At
block 735, the processing logic may send the policy via the control plane to the set of devices in the SDN, wherein an execution of the policy at one or more of the set of devices in the SDN adjusts the monitor data to more closely match a template. - At
block 740, the processing logic may update the policy based on additional monitor data. The processing logic may receive additional monitor data after sending the policy via the control plane to the set of devices in the SDN. The processing logic may update based on the additional monitor data. In at least some embodiments, updating the policy based on the additional monitor data may include generating a second data model based on the set of SDN parameters, the monitor data, and the additional monitor data. Updating the policy based on the additional monitor data may further include determining a second change for at least one device of the set of devices in the SDN based on the second data model. Updating the policy based on the additional monitor data may also include updating the policy based on the second change for at least one device of the set of devices in the SDN. - One skilled in the art will appreciate that, for these processes, operations, and methods, the functions and/or operations performed may be implemented in differing order. Further, the outlined functions and operations are only provided as examples, and some of the functions and operations may be optional, combined into fewer functions and operations, or expanded into additional functions and operations without detracting from the essence of the disclosed embodiments.
-
FIG. 8 illustrates anexample computing system 800, according to at least one embodiment described in the present disclosure. Thesystem 800 may include any suitable system, apparatus, or device configured to test software. Thecomputing system 800 may include aprocessor 810, amemory 820, adata storage 830, and acommunication unit 840, which all may be communicatively coupled. In some embodiments, any of the network devices (e.g., the edge network devices 110, 210, 310, or 410 ofFIGS. 1-4 ), control devices (e.g., thecontrol devices FIGS. 1-4 ), local computing devices (e.g., thelocal computing device 450 ofFIG. 4 , network management devices (e.g., thenetwork management device 290 ofFIG. 2 ) or other computing devices of the present disclosure may be implemented as thecomputing system 800. Additionally or alternatively, one or more of the network devices, control devices, local computing devices or other computing devices may be implemented as virtualized machines operating on a physical computing system such as thecomputing system 800. - Generally, the
processor 810 may include any suitable special-purpose or general-purpose computer, computing entity, or processing device including various computer hardware or software modules and may be configured to execute instructions stored on any applicable computer-readable storage media. For example, theprocessor 810 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a Field-Programmable Gate Array (FPGA), or any other digital or analog circuitry configured to interpret and/or to execute program instructions and/or to process data. - Although illustrated as a single processor in
FIG. 8 , it is understood that theprocessor 810 may include any number of processors distributed across any number of network or physical locations that are configured to perform individually or collectively any number of operations described in the present disclosure. In some embodiments, theprocessor 810 may interpret and/or execute program instructions and/or process data stored in thememory 820, thedata storage 830, or thememory 820 and thedata storage 830. In some embodiments, theprocessor 810 may fetch program instructions from thedata storage 830 and load the program instructions into thememory 820. - After the program instructions are loaded into the
memory 820, theprocessor 810 may execute the program instructions, such as instructions to perform themethods FIGS. 5-7 , respectively. For example, theprocessor 810 may determine that a traffic flow is associated with a rerouting application and reroute the traffic flow along the path with the best performance score. As another example, theprocessor 810 may rewrite DNS queries and/or DNS replies. As an additional example, theprocessor 810 may route flows such that an NAT exit point associated with a rerouted path may be used. As an additional example, theprocessor 810 may determine which path from multiple paths is the best path and reroute traffic accordingly. - The
memory 820 and thedata storage 830 may include computer-readable storage media or one or more computer-readable storage mediums for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable storage media may be any available media that may be accessed by a general-purpose or special-purpose computer, such as theprocessor 810. In some embodiments, thecomputing system 800 may or may not include either of thememory 820 and thedata storage 830. - By way of example, and not limitation, such computer-readable storage media may include non-transitory computer-readable storage media including Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid state memory devices), or any other storage medium which may be used to carry or store desired program code in the form of computer-executable instructions or data structures and which may be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer-readable storage media. Computer-executable instructions may include, for example, instructions and data configured to cause the
processor 810 to perform a certain operation or group of operations. - The
communication unit 840 may include any component, device, system, or combination thereof that is configured to transmit or receive information over a network, such as an MPLS connection, the Internet, a cellular network (e.g., an LTE network), etc. In some embodiments, thecommunication unit 840 may communicate with other devices at other locations, the same location, or even other components within the same system. For example, thecommunication unit 840 may include a modem, a network card (wireless or wired), an optical communication device, an infrared communication device, a wireless communication device (such as an antenna), a chipset (such as a Bluetooth device, an 802.6 device (e.g., Metropolitan Area Network (MAN)), a WiFi device, a WiMax device, cellular communication facilities, or others), and/or the like, or any combinations thereof. Thecommunication unit 840 may permit data to be exchanged with a network and/or any other devices or systems described in the present disclosure. For example, thecommunication unit 840 may allow thesystem 800 to communicate with other systems, such as network devices, control devices, and/or other networks. - Modifications, additions, or omissions may be made to the
system 800 without departing from the scope of the present disclosure. For example, thedata storage 830 may be multiple different storage mediums located in multiple locations and accessed by theprocessor 810 through a network. - As indicated above, the embodiments described in the present disclosure may include the use of a special purpose or general purpose computer (e.g., the
processor 810 ofFIG. 8 ) including various computer hardware or software modules, as discussed in greater detail below. Further, as indicated above, embodiments described in the present disclosure may be implemented using computer-readable media (e.g., thememory 820 ordata storage 830 ofFIG. 8 ) for carrying or having computer-executable instructions or data structures stored thereon. - As used in the present disclosure, the terms “module” or “component” may refer to specific hardware implementations configured to perform the actions of the module or component and/or software objects or software routines that may be stored on and/or executed by general purpose hardware (e.g., computer-readable media, processing devices, or some other hardware) of the computing system. In some embodiments, the different components, modules, engines, and services described in the present disclosure may be implemented as objects or processes that execute on the computing system (e.g., as separate threads). While some of the systems and methods described in the present disclosure are generally described as being implemented in software (stored on and/or executed by general purpose hardware), specific hardware implementations or a combination of software and specific hardware implementations are also possible and contemplated. In this description, a “computing entity” may be any computing system as previously defined in the present disclosure, or any module or combination of modulates running on a computing system.
- In accordance with common practice, the various features illustrated in the drawings may not be drawn to scale. The illustrations presented in the present disclosure are not meant to be actual views of any particular apparatus (e.g., device, system, etc.) or method, but are merely idealized representations that are employed to describe various embodiments of the disclosure. Accordingly, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may be simplified for clarity. Thus, the drawings may not depict all of the components of a given apparatus (e.g., device) or all operations of a particular method.
- Terms used in the present disclosure and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including, but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes, but is not limited to,” among others).
- Additionally, if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations.
- In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” or “one or more of A, B, and C, etc.” is used, in general such a construction is intended to include A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, etc.
- Further, any disjunctive word or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” should be understood to include the possibilities of “A” or “B” or “A and B.”
- However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations.
- Additionally, the use of the terms “first,” “second,” “third,” etc., are not necessarily used herein to connote a specific order or number of elements. Generally, the terms “first,” “second,” “third,” etc., are used to distinguish between different elements as generic identifiers. Absence a showing that the terms “first,” “second,” “third,” etc., connote a specific order, these terms should not be understood to connote a specific order. Furthermore, absence a showing that the terms “first,” “second,” “third,” etc., connote a specific number of elements, these terms should not be understood to connote a specific number of elements. For example, a first widget may be described as having a first side and a second widget may be described as having a second side. The use of the term “second side” with respect to the second widget may be to distinguish such side of the second widget from the “first side” of the first widget and not to connote that the second widget has two sides.
- All examples and conditional language recited in the present disclosure are intended for pedagogical objects to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Although embodiments of the present disclosure have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the present disclosure.
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/019,478 US20190036779A1 (en) | 2017-07-31 | 2018-06-26 | Virtualized software-defined network |
US17/189,996 US11405279B2 (en) | 2017-07-31 | 2021-03-02 | Virtualized software-defined network |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762539491P | 2017-07-31 | 2017-07-31 | |
US16/019,478 US20190036779A1 (en) | 2017-07-31 | 2018-06-26 | Virtualized software-defined network |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/189,996 Continuation US11405279B2 (en) | 2017-07-31 | 2021-03-02 | Virtualized software-defined network |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190036779A1 true US20190036779A1 (en) | 2019-01-31 |
Family
ID=65038338
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/019,478 Abandoned US20190036779A1 (en) | 2017-07-31 | 2018-06-26 | Virtualized software-defined network |
US17/189,996 Active US11405279B2 (en) | 2017-07-31 | 2021-03-02 | Virtualized software-defined network |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/189,996 Active US11405279B2 (en) | 2017-07-31 | 2021-03-02 | Virtualized software-defined network |
Country Status (1)
Country | Link |
---|---|
US (2) | US20190036779A1 (en) |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190036814A1 (en) * | 2017-07-31 | 2019-01-31 | Cisco Technology, Inc. | Traffic steering with path ordering |
US20190036842A1 (en) * | 2017-07-31 | 2019-01-31 | Cisco Technology, Inc. | Traffic steering in fastpath |
US10721146B2 (en) * | 2012-07-31 | 2020-07-21 | Micro Focus Llc | Monitoring for managed services |
CN111917645A (en) * | 2020-08-20 | 2020-11-10 | 深圳多拉多通信技术有限公司 | SDN-based path optimization method and system for mobile network |
CN112019371A (en) * | 2019-05-31 | 2020-12-01 | 瞻博网络公司 | Dynamic application SLA metric generation, distribution and intent-based SD-WAN link selection |
US10951490B2 (en) | 2019-05-09 | 2021-03-16 | Cisco Technology, Inc. | Intelligent tunnel assignment and dynamic SLA threshold configuration to increase availability and utilization of SD-WAN tunnels |
US20210203594A1 (en) * | 2018-08-23 | 2021-07-01 | Agora Lab, Inc. | Large-Scale Real-Time Multimedia Communications |
US20210250237A1 (en) * | 2017-09-22 | 2021-08-12 | Webroot, Inc. | State-based entity behavior analysis |
US20210311780A1 (en) * | 2020-11-30 | 2021-10-07 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and system for arranging business process, computing device, and non-transitory computer readable storage medium |
US11146463B2 (en) | 2019-06-05 | 2021-10-12 | Cisco Technology, Inc. | Predicting network states for answering what-if scenario outcomes |
US11216742B2 (en) | 2019-03-04 | 2022-01-04 | Iocurrents, Inc. | Data compression and communication using machine learning |
US20220078122A1 (en) * | 2019-04-24 | 2022-03-10 | Huawei Technologies Co., Ltd. | Method and apparatus for accessing gateway |
US20220345399A1 (en) * | 2019-09-30 | 2022-10-27 | Orange | Method for monitoring a data stream associated with a process within a shared network |
CN115516829A (en) * | 2020-06-02 | 2022-12-23 | 思科技术公司 | Provisioning configuration changes with deployment freeze options |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102477931B1 (en) * | 2021-11-17 | 2022-12-15 | 주식회사 와이엘아이티 | SDN Platform and SDC Platform Automation system and the method using it |
Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8234660B2 (en) * | 2006-04-12 | 2012-07-31 | Sap Portals Israel, Ltd. | Method and apparatus for a support platform |
US20140280834A1 (en) * | 2013-03-15 | 2014-09-18 | Cisco Technology, Inc. | Programmable management engine for networks |
US20140325649A1 (en) * | 2013-04-29 | 2014-10-30 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system to dynamically detect traffic anomalies in a network |
US20150112933A1 (en) * | 2013-10-18 | 2015-04-23 | Cisco Technology, Inc. | System and method for software defined network aware data replication |
US20150317169A1 (en) * | 2014-05-04 | 2015-11-05 | Midfin Systems Inc. | Constructing and operating high-performance unified compute infrastructure across geo-distributed datacenters |
US20160050132A1 (en) * | 2014-08-18 | 2016-02-18 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system to dynamically collect statistics of traffic flows in a software-defined networking (sdn) system |
US9276877B1 (en) * | 2012-09-20 | 2016-03-01 | Wiretap Ventures, LLC | Data model for software defined networks |
US20160234121A1 (en) * | 2012-03-22 | 2016-08-11 | Futurewei Technologies, Inc. | Supporting Software Defined Networking with Application Layer Traffic Optimization |
US20160277525A1 (en) * | 2013-11-27 | 2016-09-22 | Huawei Technologies Co., Ltd. | Method and controller for clustering applications in a software-defined network |
US20170048126A1 (en) * | 2015-08-11 | 2017-02-16 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system for debugging in a software-defined networking (sdn) system |
US9584477B2 (en) * | 2015-02-26 | 2017-02-28 | International Business Machines Corporation | Packet processing in a multi-tenant software defined network (SDN) |
US9674057B2 (en) * | 2014-10-07 | 2017-06-06 | At&T Intellectual Property I, L.P. | Method and system to monitor a network |
US20170230267A1 (en) * | 2016-02-08 | 2017-08-10 | Ciena Corporation | Traffic-adaptive network control systems and methods |
US20190036780A1 (en) * | 2017-07-31 | 2019-01-31 | Cisco Technology, Inc. | Generating a data model for a virtualized software-defined network |
US20190166013A1 (en) * | 2016-04-29 | 2019-05-30 | Sanctum Networks Limited | A data driven intent based networking approach using a light weight distributed SDN controller for delivering intelligent consumer experience |
US10320644B1 (en) * | 2015-09-14 | 2019-06-11 | Amazon Technologies, Inc. | Traffic analyzer for isolated virtual networks |
US10440082B1 (en) * | 2016-06-21 | 2019-10-08 | Amazon Technologies, Inc. | Adjusting parameter settings for bitrate selection algorithms |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130110757A1 (en) * | 2011-10-26 | 2013-05-02 | Joël R. Calippe | System and method for analyzing attribute change impact within a managed network |
-
2018
- 2018-06-26 US US16/019,478 patent/US20190036779A1/en not_active Abandoned
-
2021
- 2021-03-02 US US17/189,996 patent/US11405279B2/en active Active
Patent Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8234660B2 (en) * | 2006-04-12 | 2012-07-31 | Sap Portals Israel, Ltd. | Method and apparatus for a support platform |
US20160234121A1 (en) * | 2012-03-22 | 2016-08-11 | Futurewei Technologies, Inc. | Supporting Software Defined Networking with Application Layer Traffic Optimization |
US9276877B1 (en) * | 2012-09-20 | 2016-03-01 | Wiretap Ventures, LLC | Data model for software defined networks |
US20140280834A1 (en) * | 2013-03-15 | 2014-09-18 | Cisco Technology, Inc. | Programmable management engine for networks |
US20140325649A1 (en) * | 2013-04-29 | 2014-10-30 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system to dynamically detect traffic anomalies in a network |
US20150112933A1 (en) * | 2013-10-18 | 2015-04-23 | Cisco Technology, Inc. | System and method for software defined network aware data replication |
US20160277525A1 (en) * | 2013-11-27 | 2016-09-22 | Huawei Technologies Co., Ltd. | Method and controller for clustering applications in a software-defined network |
US20150317169A1 (en) * | 2014-05-04 | 2015-11-05 | Midfin Systems Inc. | Constructing and operating high-performance unified compute infrastructure across geo-distributed datacenters |
US20160050132A1 (en) * | 2014-08-18 | 2016-02-18 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system to dynamically collect statistics of traffic flows in a software-defined networking (sdn) system |
US9674057B2 (en) * | 2014-10-07 | 2017-06-06 | At&T Intellectual Property I, L.P. | Method and system to monitor a network |
US9584477B2 (en) * | 2015-02-26 | 2017-02-28 | International Business Machines Corporation | Packet processing in a multi-tenant software defined network (SDN) |
US20170048126A1 (en) * | 2015-08-11 | 2017-02-16 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system for debugging in a software-defined networking (sdn) system |
US10320644B1 (en) * | 2015-09-14 | 2019-06-11 | Amazon Technologies, Inc. | Traffic analyzer for isolated virtual networks |
US20170230267A1 (en) * | 2016-02-08 | 2017-08-10 | Ciena Corporation | Traffic-adaptive network control systems and methods |
US20190166013A1 (en) * | 2016-04-29 | 2019-05-30 | Sanctum Networks Limited | A data driven intent based networking approach using a light weight distributed SDN controller for delivering intelligent consumer experience |
US10440082B1 (en) * | 2016-06-21 | 2019-10-08 | Amazon Technologies, Inc. | Adjusting parameter settings for bitrate selection algorithms |
US20190036780A1 (en) * | 2017-07-31 | 2019-01-31 | Cisco Technology, Inc. | Generating a data model for a virtualized software-defined network |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10721146B2 (en) * | 2012-07-31 | 2020-07-21 | Micro Focus Llc | Monitoring for managed services |
US20190036842A1 (en) * | 2017-07-31 | 2019-01-31 | Cisco Technology, Inc. | Traffic steering in fastpath |
US20190036814A1 (en) * | 2017-07-31 | 2019-01-31 | Cisco Technology, Inc. | Traffic steering with path ordering |
US11792075B2 (en) * | 2017-09-22 | 2023-10-17 | Open Text Inc. | State-based entity behavior analysis |
US20210250237A1 (en) * | 2017-09-22 | 2021-08-12 | Webroot, Inc. | State-based entity behavior analysis |
US11949588B2 (en) * | 2018-08-23 | 2024-04-02 | Agora Lab, Inc. | Large-scale real-time multimedia communications |
US20210203594A1 (en) * | 2018-08-23 | 2021-07-01 | Agora Lab, Inc. | Large-Scale Real-Time Multimedia Communications |
US11216742B2 (en) | 2019-03-04 | 2022-01-04 | Iocurrents, Inc. | Data compression and communication using machine learning |
US11468355B2 (en) | 2019-03-04 | 2022-10-11 | Iocurrents, Inc. | Data compression and communication using machine learning |
US20220078122A1 (en) * | 2019-04-24 | 2022-03-10 | Huawei Technologies Co., Ltd. | Method and apparatus for accessing gateway |
US10951490B2 (en) | 2019-05-09 | 2021-03-16 | Cisco Technology, Inc. | Intelligent tunnel assignment and dynamic SLA threshold configuration to increase availability and utilization of SD-WAN tunnels |
CN115174424A (en) * | 2019-05-31 | 2022-10-11 | 瞻博网络公司 | Method and apparatus for network management |
US10979316B2 (en) | 2019-05-31 | 2021-04-13 | Juniper Networks, Inc. | Dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection |
EP3745644A1 (en) * | 2019-05-31 | 2020-12-02 | Juniper Networks, Inc. | Dynamic application sla metric generation, distribution, and intentbased sd-wan link selection |
EP4145796A1 (en) * | 2019-05-31 | 2023-03-08 | Juniper Networks, Inc. | Dynamic application sla metric generation, distribution, and intent-based sd-wan link selection |
CN112019371A (en) * | 2019-05-31 | 2020-12-01 | 瞻博网络公司 | Dynamic application SLA metric generation, distribution and intent-based SD-WAN link selection |
US11831520B2 (en) | 2019-05-31 | 2023-11-28 | Juniper Networks, Inc. | Dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection |
US11146463B2 (en) | 2019-06-05 | 2021-10-12 | Cisco Technology, Inc. | Predicting network states for answering what-if scenario outcomes |
US20220345399A1 (en) * | 2019-09-30 | 2022-10-27 | Orange | Method for monitoring a data stream associated with a process within a shared network |
CN115516829A (en) * | 2020-06-02 | 2022-12-23 | 思科技术公司 | Provisioning configuration changes with deployment freeze options |
CN111917645A (en) * | 2020-08-20 | 2020-11-10 | 深圳多拉多通信技术有限公司 | SDN-based path optimization method and system for mobile network |
US20210311780A1 (en) * | 2020-11-30 | 2021-10-07 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and system for arranging business process, computing device, and non-transitory computer readable storage medium |
Also Published As
Publication number | Publication date |
---|---|
US11405279B2 (en) | 2022-08-02 |
US20210184932A1 (en) | 2021-06-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10721165B2 (en) | Controlling a software-defined network | |
US11405279B2 (en) | Virtualized software-defined network | |
US20190036780A1 (en) | Generating a data model for a virtualized software-defined network | |
US11201817B2 (en) | Traffic steering in fastpath | |
US10819564B2 (en) | Network hub site redundancy and failover | |
US20210243095A1 (en) | Routing network traffic | |
US20190036814A1 (en) | Traffic steering with path ordering | |
US11018937B2 (en) | Determining an effect of a network configuration change | |
CA3063088C (en) | Routing network traffic based on performance | |
US11722403B2 (en) | Routing network traffic based on DNS | |
US11658898B2 (en) | Routing network traffic based on destination | |
US20190036842A1 (en) | Traffic steering in fastpath | |
US10944733B2 (en) | Dynamic disassociated channel encryption key distribution |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CISCO TECHNOLOGY, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BAJAJ, SANDEEP;REEL/FRAME:046822/0108 Effective date: 20180725 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |