WO2015147859A1 - Reconciling information in a controller and a node - Google Patents
Reconciling information in a controller and a node Download PDFInfo
- Publication number
- WO2015147859A1 WO2015147859A1 PCT/US2014/032154 US2014032154W WO2015147859A1 WO 2015147859 A1 WO2015147859 A1 WO 2015147859A1 US 2014032154 W US2014032154 W US 2014032154W WO 2015147859 A1 WO2015147859 A1 WO 2015147859A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- controller
- virtual
- node
- information
- entities
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0866—Checking the configuration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/1886—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast with traffic restrictions for efficiency improvement, e.g. involving subnets or subdomains
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/085—Retrieval of network configuration; Tracking network configuration history
- H04L41/0853—Retrieval of network configuration; Tracking network configuration history by actively collecting configuration information or by backing up configuration information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/508—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement
- H04L41/5096—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement wherein the managed service relates to distributed or central networked applications
Definitions
- a network can be made up of a network infrastructure (including various network entities) that is used by devices to communicate with each other through the network.
- network entities include switches, routers, configuration servers (e.g. Dynamic Host Configuration Protocol or DHCP servers), and so forth.
- the network infrastructure of a particular network is owned by a network operator.
- an enterprise such as a business concern, educational organization or government agency, can operate a network for use by users (e.g. employees, customers, etc.) of the enterprise.
- the network infrastructure of such network is owned by the enterprise.
- the network operator can instead pay to use networking entities provided by a third party service provider.
- the service provider provides an infrastructure that includes various network entities accessible by customers (also referred to as "tenants") of the service provider.
- customers also referred to as "tenants”
- a network operator would not have to invest in various components of a network infrastructure, and would not have to be concerned with maintenance of the network infrastructure.
- FIG. 1 is a block diagram of an example arrangement that includes a network cloud infrastructure and tenant systems, according to some
- Fig. 2 is a flow diagram of a synchronization process according to some implementations.
- Fig. 3 is a schematic diagram of a hierarchical arrangement of types of network entities, according to some examples.
- FIG. 4 is a flow diagram of a process of a reconciler, according to further implementations.
- FIG. 5 is a block diagram of a controller incorporating some
- Fig. 1 is a block diagram of an example arrangement that includes a network cloud infrastructure 100, which may be operated and/or owned by a network service provider.
- the network cloud infrastructure 100 has customers (also referred to as "tenants") that operate respective tenant systems 102.
- Each tenant system 102 can include a network deployment that uses network entities of the network cloud infrastructure 100.
- the provision of network entities of a network cloud infrastructure by a network service provider to a tenant is part of a cloud-service model that is sometimes referred to as network as a service (NaaS) or infrastructure as a service (laaS).
- NaaS network as a service
- laaS infrastructure as a service
- virtual network entities 104 are hosted by managed nodes 106 (e.g. server computers, physical switches, etc.).
- a virtual network entity being hosted by a managed node 106 refers to the virtual network entity being executable by the managed node 106 (in cases where the virtual network entity is implemented as machine-readable instructions).
- a virtual network entity hosted by a managed node 106 can refer to some partition of the managed node, where the partition can include hardware as well as machine-readable instructions.
- Examples of virtual network entities 104 include any or some combination of the following network resources: a switch for switching data between devices at layer 2 of the Open Systems Interconnection (OSI) model; a router for routing data at layer 3 of the OSI model; a port of a transport protocol such as the Transmission Control Protocol (TCP); an Internet Protocol (IP) address (which identifies an endpoint of a network); a configuration server, such as a Dynamic Host
- OSI Open Systems Interconnection
- IP Internet Protocol
- DHCP Dynamic Configuration Protocol
- a security enforcement entity for enforcing a security policy
- a network a subnet of a network (note that a network can include multiple subnets that make up different segments of the network); and so forth.
- a virtual network entity can also refer to a network service, such as any or some combination of the following: switching service, routing service, DHCP configuration service, security service, and so forth.
- the managed nodes 106 hosting respective virtual network entities 104 are managed by a controller 108.
- Each managed node 106 can include one or multiple virtual network entities 104.
- the controller 108 is able to direct which virtual network entities 104 are hosted on which managed nodes 106. Although just one controller 108 is shown in Fig. 1 , it is noted that in other examples, the network cloud infrastructure 100 can include multiple controllers 108 for managing the managed nodes 106.
- Fig. 1 The arrangement shown in Fig. 1 in which the controller 108 manages managed nodes 106 can be part of a software-defined networking (SDN)
- SDN software-defined networking
- the controller 108 is part of a control plane, while the managed nodes 106 are part of a user plane through which traffic is communicated. Tenant or user traffic does not have to be communicated through the control plane.
- the controller 108 is responsible for determining where (which of the managed nodes 106) a virtual network entity 104 is to be hosted, while a managed node is responsible for deploying a specific network resource or service.
- the controller 108 has a service database 1 10 that contains information of virtual network entities deployed at various managed nodes 106.
- each managed node 106 has a respective service database 1 12 that contains information of virtual network entities 104 hosted by the respective managed node 106.
- the service database 1 10 can be stored on a storage medium within the controller 108, or on a storage medium that is outside the controller 108 but that is accessible by the controller 108.
- the service database 1 12 can be stored on a storage medium within a respective managed node 106, or on a storage medium outside the respective managed node 106 but that is accessible by one or multiple managed nodes 106.
- the information stored in the service database 1 10 or 1 12 can include state information and configuration information.
- State information can specify a status of the virtual network entity 104, such as if the virtual network entity 104 is operational or not operational (e.g. whether a router is forwarding traffic or the router is not forwarding traffic), whether a port is up or down, whether a network is operational or not operational, whether a subnet is operational or not operational, and so forth.
- Configuration information can specify a capability or configuration of a virtual network entity.
- the configuration information can identify subnets of a network, specify values of parameters that affect operation of a virtual network entity (e.g. parameters of a switch, router, or DHCP server), indicate whether or not a virtual network entity can be accessed by a tenant system, and so forth.
- a cloud infrastructure can include virtual entities 104 that are useable in a tenant system 102.
- virtual entities can include processing resources, storage resources, software (in the form of machine-readable
- the view (as represented by the information in the service database 1 10) of the controller 108 can become inconsistent with the view (as represented by the information in a service database 1 12) of a particular managed node 106.
- the particular managed node may experience a fault
- the controller 108 may experience a fault
- communication between the particular managed node 106 and the controller 108 may have been temporarily lost.
- managed nodes 106 can periodically poll the controller 108 for any changes relating to virtual network entities 104 hosted by the respective managed nodes since the last update. Such polling can be performed using
- OpenStack components in the managed nodes 106 and the controller 108 where OpenStack provides an open source cloud computing platform for cloud systems.
- An issue associated with managed nodes 106 periodically polling the controller 108 is that such polling can result in a large number of messages communicated between the managed nodes 106 and the controller 108. If the network cloud infrastructure 100 has a relatively large number of managed nodes, the increase in polling messages can consume control plane bandwidth in the network cloud infrastructure 100 that can lead to reduced performance or delayed response in the network cloud infrastructure 100. Also, the network cloud infrastructure 100 becomes less scalable, since increasing the number of managed nodes leads to an even larger number of polling messages.
- the controller 108 does not rely on periodic polling by managed nodes 106.
- the controller 108 includes a reconciler 1 14 that is able to reconcile any inconsistencies between the service database 1 10 of the controller 108 and service databases 1 12 of the managed nodes 106.
- the reconciler 1 14 is able to perform a cleanup phase and a synchronization phase.
- the cleanup phase the reconciler 1 14 can remove stale virtual network entities from managed nodes 106.
- a stale virtual network entity can refer to a virtual network entity that currently exists but that should have been removed previously.
- the reconciler 1 14 can perform updates to achieve consistency between the views of managed nodes 106 and the controller 108.
- Fig. 2 is a flow diagram of a synchronization process of a synchronization phase performed by the reconciler 1 14 in the controller 108, in accordance with some implementations.
- the reconciler 1 14 receives (at 202) information from a particular one of the managed nodes 106 in the network cloud infrastructure 100.
- the received information relates to a characteristic (e.g. state and/or configuration) of a virtual network entity hosted by the particular managed node.
- the received information can include information contained or derived from information stored in the service database 1 12 of the particular managed node.
- the reconciler 1 14 determines (at 204) whether the received information from the particular managed node is inconsistent with information maintained by the controller relating to the characteristic of the virtual network entity hosted by the particular managed node.
- the information maintained by the controller 108 includes information retrieved from or derived from information in the service database 1 10. If the received information is not inconsistent with information maintained by the controller 108, then no reconciliation action is performed by the reconciler 1 14.
- the reconciler 1 14 reconciles (at 206) an inconsistency between the received information and the information maintained by the controller 108.
- the reconciling includes one or some combination of the following: updating information relating to a virtual network entity in the service database 1 10 or 1 12 (and possibly updating a version of the virtual network entity in a managed node 106), or adding information relating to a virtual network entity to the service database 1 10 or 1 12 (and possibly adding the virtual network entity to a managed node 106), or removing information relating to a virtual network entity from the service database 1 10 or 1 12 (and possibly removing the virtual network entity from a managed node 106).
- FIG. 3 There can be multiple different types of virtual network entities. Some example types of virtual network entities are depicted in Fig. 3. The different types of virtual network entities have a hierarchical relationship, as depicted in the
- the root of the hierarchical arrangement shown in the example of Fig. 3 is a network (which is one type of virtual network entity).
- Another type of virtual network entity is a subnet, which depends on the network. In other words, a subnet is part of the network (a network can include multiple subnets).
- Other virtual network entity types shown in Fig. 3 include a port (which depends on the network), a router (which depends on the subnet and the port), and a floating IP address (which depends on the router).
- another virtual network entity is a security enforcement service, which depends on the port.
- a floating IP address refers to an IP address that can be dynamically added to a virtual network entity, which in the case of Fig. 3 is the router. In some examples, floating IP addresses are supported in an OpenStack
- floating IP addresses can be used in other architectures.
- a first network virtual entity depending on a second virtual network entity refers to the first virtual network entity having a relationship with the second virtual network entity where the first virtual network entity's operation or existence is in the context of the second virtual network entity.
- Fig. 3 the root (network) is at the top of the hierarchical arrangement, while a leaf is at the lowest level of the hierarchical arrangement.
- Fig. 4 is a flow diagram of a process performed by the reconciler 1 14, according to further implementations.
- the reconciler 1 14 determines (at 402) if loss of communication with a managed node N (managed node N is one of the managed nodes 106) has occurred. Loss of communication between the controller 108 and managed node N can occur due to various reasons, including fault at the controller 108, fault at managed node N, or fault in the communications link between the controller 108 and managed node N. Note that the process of Fig. 4 is performed for each of the multiple managed nodes 106 in the network cloud infrastructure 100 that are associated with the controller 108.
- the reconciler 1 14 determines (at 404) if communication has been re-established with managed node N. If not, the reconciler 1 14 continues to wait for re-establishment of communication with managed node N. However, if communication has been re-established with managed node N, then the reconciler 1 14 proceeds to perform a cleanup phase 406.
- the cleanup phase 406 iteratively performs tasks for each of multiple virtual network entity types (assuming managed node N has multiple types of virtual network entities).
- the iterative performance of tasks in the cleanup phase 406 starts at the virtual network entity type at the lowest level (the level closest to a leaf in Fig. 3) of the hierarchical arrangement. More generally, the iterative performance of tasks in the cleanup phase 406 iterates in an order of dependency such that any virtual network entity type A that depends upon virtual network entity type B appears earlier in the iteration than virtual network entity type B.
- the reconciler 1 14 compares (at 408) a listing (X) of a virtual network entities received from managed node N with a listing (Y) of virtual network entities at the controller 108.
- the listing (X) is retrieved or derived from the service database 1 12 of managed node N, while the listing (Y) of virtual network entities is retrieved or derived from the service database 1 10 of the controller 108.
- the listing (X) can be received from managed node N in response to a query from the controller 108 to the managed node N.
- the listing (X) of virtual network entities and the listing (Y) of virtual network entities identify virtual network entities of type T, which is the virtual network entity type that is currently considered in the iterative cleanup phase 406.
- the reconciler 1 14 removes (at 410) a given virtual network entity of type T from managed node N if the given virtual network entity is not in listing (Y) of the controller 108.
- Tasks 408 and 410 are iteratively performed for each of different virtual network entity types (T) in managed node N.
- the synchronization stage 412 iteratively performs tasks for each of multiple virtual network entity types in managed node N, starting at the highest level of the hierarchical arrangement (the level closest to the root in Fig. 3, for example). More generally, the iterative performance of tasks in the
- synchronization phase 412 iterates in an order of dependency such that any virtual network entity type A that depends upon virtual network entity type B appears later in the iteration than virtual network entity type B.
- the reconciler 1 14 compares (at 414) a listing (X) of virtual network entities obtained from managed node (N) with a listing (Y) of virtual network entities of the controller 108. For each virtual network entity (E) of type T that is identified in the listing (X), if E exists in the listing (Y) but does not match a version stored in the listing (Y), then E is updated (at 416) in the listing (X). Updating E in the listing (X) also causes the version of the virtual network entity E to be updated at managed node N.
- a virtual network entity may exist but not match if there exists a virtual network entity where the characteristics (e.g.
- listing (X) and listing (Y) may each identify a virtual network entity E that shares a common unique ID. However, listing (X) and listing (Y) may have at least one attribute of virtual network entity E that differ in value, which means that the versions of virtual network entity E identified in listing (X) and listing (Y) are inconsistent.
- Tasks 414, 416, and 418 are performed for each of the other virtual network entity types that exist in managed node N.
- the listing (X) and the listing (Y) may be cached in cache memory accessible by the reconciler 1 14, so that such listings can be retrieved more quickly.
- reconciliation between views of virtual network entities maintained by managed nodes 106 and the controller 108 occurs when a fault occurs (e.g. loss of communication detected at 402 in Fig. 4).
- a fault e.g. loss of communication detected at 402 in Fig. 4
- messaging associated with such reconciliation is communicated when the fault occurs, which reduces network traffic since faults occur relatively infrequently.
- Fig. 5 is a block diagram of an arrangement of the controller 108 according to some implementations.
- the controller 108 can include one or multiple processors 502, which can be coupled to one or multiple network interfaces 504 (to allow the controller 108 to communicate over a network), and to a non-transitory machine-readable or computer-readable storage medium 506 (or multiple storage media).
- the storage medium or storage media 506 can store the reconciler 1 14 in the form of machine-readable instructions, as well as the service database 1 10.
- the reconciler 1 14 can be loaded from the storage medium or storage media 506 for execution on the processor(s) 502.
- a processor can include a microprocessor, microcontroller, processor module or subsystem, programmable integrated circuit, programmable gate array, or another control or computing device.
- the storage medium (or storage media) 506 can include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices.
- semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories
- magnetic disks such as fixed, floppy and removable disks
- other magnetic media including tape optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices.
- CDs compact disks
- DVDs digital video disks
- Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture).
- An article or article of manufacture can refer to any manufactured single component or multiple components.
- the storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A controller, which manages nodes that provide virtual entities, receives information from a first of the nodes, where the received information relates to a characteristic of a virtual entity provided by the first node. The controller determines that the received information is inconsistent with information, maintained by the controller, relating to the characteristic of the virtual entity provided by the first node. In response to the determining, the controller reconciles an inconsistency between the received information and the information maintained by the controller.
Description
RECONCILING INFORMATION IN A CONTROLLER AND A NODE Background
[0001 ] A network can be made up of a network infrastructure (including various network entities) that is used by devices to communicate with each other through the network. Examples of network entities include switches, routers, configuration servers (e.g. Dynamic Host Configuration Protocol or DHCP servers), and so forth.
[0002] Traditionally, the network infrastructure of a particular network is owned by a network operator. For example, an enterprise, such as a business concern, educational organization or government agency, can operate a network for use by users (e.g. employees, customers, etc.) of the enterprise. The network infrastructure of such network is owned by the enterprise.
[0003] In an alternative arrangement, instead of using a network operator's own network infrastructure to implement a network, the network operator can instead pay to use networking entities provided by a third party service provider. The service provider provides an infrastructure that includes various network entities accessible by customers (also referred to as "tenants") of the service provider. By using the infrastructure of the service provider, a network operator would not have to invest in various components of a network infrastructure, and would not have to be concerned with maintenance of the network infrastructure.
Brief Description Of The Drawings
[0004] Some implementations are described with respect to the following figures.
[0005] Fig. 1 is a block diagram of an example arrangement that includes a network cloud infrastructure and tenant systems, according to some
implementations.
[0006] Fig. 2 is a flow diagram of a synchronization process according to some implementations.
[0007] Fig. 3 is a schematic diagram of a hierarchical arrangement of types of network entities, according to some examples.
[0008] Fig. 4 is a flow diagram of a process of a reconciler, according to further implementations.
[0009] Fig. 5 is a block diagram of a controller incorporating some
implementations.
Detailed Description
[0010] Fig. 1 is a block diagram of an example arrangement that includes a network cloud infrastructure 100, which may be operated and/or owned by a network service provider. The network cloud infrastructure 100 has customers (also referred to as "tenants") that operate respective tenant systems 102. Each tenant system 102 can include a network deployment that uses network entities of the network cloud infrastructure 100. The provision of network entities of a network cloud infrastructure by a network service provider to a tenant is part of a cloud-service model that is sometimes referred to as network as a service (NaaS) or infrastructure as a service (laaS).
[001 1 ] The network entities in the network cloud infrastructure that are available for use by tenant systems 102 are referred to as "virtual network entities." The network entities are "virtual" in the sense that the network entities are not physical entities within a network deployment of a respective tenant system 102, but rather entities that can be logically implemented in the network deployment. As shown in Fig. 1 , virtual network entities 104 are hosted by managed nodes 106 (e.g. server computers, physical switches, etc.). A virtual network entity being hosted by a managed node 106 refers to the virtual network entity being executable by the managed node 106 (in cases where the virtual network entity is implemented as machine-readable instructions). Alternatively, a virtual network entity hosted by a managed node 106 can refer to some partition of the managed node, where the partition can include hardware as well as machine-readable instructions.
[0012] Examples of virtual network entities 104 include any or some combination of the following network resources: a switch for switching data between devices at layer 2 of the Open Systems Interconnection (OSI) model; a router for routing data at layer 3 of the OSI model; a port of a transport protocol such as the Transmission Control Protocol (TCP); an Internet Protocol (IP) address (which identifies an endpoint of a network); a configuration server, such as a Dynamic Host
Configuration Protocol (DHCP) server used for setting network configuration parameters such as IP addresses for devices that communicate over a network; a security enforcement entity (for enforcing a security policy); a network; a subnet of a network (note that a network can include multiple subnets that make up different segments of the network); and so forth. Although example network resources are provided above, it is noted that virtual network entities can also include other types of network resources that are useable in a network deployment of a tenant system 102. Additionally or alternatively, a virtual network entity can also refer to a network service, such as any or some combination of the following: switching service, routing service, DHCP configuration service, security service, and so forth.
[0013] The managed nodes 106 hosting respective virtual network entities 104 are managed by a controller 108. Each managed node 106 can include one or multiple virtual network entities 104. The controller 108 is able to direct which virtual network entities 104 are hosted on which managed nodes 106. Although just one controller 108 is shown in Fig. 1 , it is noted that in other examples, the network cloud infrastructure 100 can include multiple controllers 108 for managing the managed nodes 106.
[0014] The arrangement shown in Fig. 1 in which the controller 108 manages managed nodes 106 can be part of a software-defined networking (SDN)
arrangement, in which machine-readable instructions executed by the controller 108 perform management of the managed nodes 106. In the SDN arrangement, the controller 108 is part of a control plane, while the managed nodes 106 are part of a user plane through which traffic is communicated. Tenant or user traffic does not have to be communicated through the control plane. The controller 108 is
responsible for determining where (which of the managed nodes 106) a virtual network entity 104 is to be hosted, while a managed node is responsible for deploying a specific network resource or service.
[0015] The controller 108 has a service database 1 10 that contains information of virtual network entities deployed at various managed nodes 106. In addition, each managed node 106 has a respective service database 1 12 that contains information of virtual network entities 104 hosted by the respective managed node 106. The service database 1 10 can be stored on a storage medium within the controller 108, or on a storage medium that is outside the controller 108 but that is accessible by the controller 108. Similarly, the service database 1 12 can be stored on a storage medium within a respective managed node 106, or on a storage medium outside the respective managed node 106 but that is accessible by one or multiple managed nodes 106.
[0016] The information stored in the service database 1 10 or 1 12 can include state information and configuration information. State information can specify a status of the virtual network entity 104, such as if the virtual network entity 104 is operational or not operational (e.g. whether a router is forwarding traffic or the router is not forwarding traffic), whether a port is up or down, whether a network is operational or not operational, whether a subnet is operational or not operational, and so forth. Configuration information can specify a capability or configuration of a virtual network entity. For example, the configuration information can identify subnets of a network, specify values of parameters that affect operation of a virtual network entity (e.g. parameters of a switch, router, or DHCP server), indicate whether or not a virtual network entity can be accessed by a tenant system, and so forth.
[0017] More generally, a cloud infrastructure can include virtual entities 104 that are useable in a tenant system 102. Such virtual entities can include processing resources, storage resources, software (in the form of machine-readable
instructions), and so forth.
[0018] In the ensuing discussion, reference is made to virtual network entities of a network cloud infrastructure. However, techniques or mechanisms according to some implementations can be applied to other types of virtual entities provided by nodes of a cloud infrastructure.
[0019] Occasionally, the view (as represented by the information in the service database 1 10) of the controller 108 can become inconsistent with the view (as represented by the information in a service database 1 12) of a particular managed node 106. For example, the particular managed node may experience a fault, the controller 108 may experience a fault, or communication between the particular managed node 106 and the controller 108 may have been temporarily lost.
[0020] In some examples, managed nodes 106 can periodically poll the controller 108 for any changes relating to virtual network entities 104 hosted by the respective managed nodes since the last update. Such polling can be performed using
OpenStack components in the managed nodes 106 and the controller 108, where OpenStack provides an open source cloud computing platform for cloud systems. An issue associated with managed nodes 106 periodically polling the controller 108 is that such polling can result in a large number of messages communicated between the managed nodes 106 and the controller 108. If the network cloud infrastructure 100 has a relatively large number of managed nodes, the increase in polling messages can consume control plane bandwidth in the network cloud infrastructure 100 that can lead to reduced performance or delayed response in the network cloud infrastructure 100. Also, the network cloud infrastructure 100 becomes less scalable, since increasing the number of managed nodes leads to an even larger number of polling messages.
[0021 ] In accordance with some implementations, the controller 108 does not rely on periodic polling by managed nodes 106. The controller 108 includes a reconciler 1 14 that is able to reconcile any inconsistencies between the service database 1 10 of the controller 108 and service databases 1 12 of the managed nodes 106. In some implementations, the reconciler 1 14 is able to perform a cleanup phase and a synchronization phase. In the cleanup phase, the reconciler
1 14 can remove stale virtual network entities from managed nodes 106. A stale virtual network entity can refer to a virtual network entity that currently exists but that should have been removed previously. In the synchronization phase, the reconciler 1 14 can perform updates to achieve consistency between the views of managed nodes 106 and the controller 108.
[0022] Fig. 2 is a flow diagram of a synchronization process of a synchronization phase performed by the reconciler 1 14 in the controller 108, in accordance with some implementations. The reconciler 1 14 receives (at 202) information from a particular one of the managed nodes 106 in the network cloud infrastructure 100. The received information relates to a characteristic (e.g. state and/or configuration) of a virtual network entity hosted by the particular managed node. The received information can include information contained or derived from information stored in the service database 1 12 of the particular managed node.
[0023] The reconciler 1 14 determines (at 204) whether the received information from the particular managed node is inconsistent with information maintained by the controller relating to the characteristic of the virtual network entity hosted by the particular managed node. The information maintained by the controller 108 includes information retrieved from or derived from information in the service database 1 10. If the received information is not inconsistent with information maintained by the controller 108, then no reconciliation action is performed by the reconciler 1 14.
However, if the received information is determined to be inconsistent with the information maintained by the controller 108, then the reconciler 1 14 reconciles (at 206) an inconsistency between the received information and the information maintained by the controller 108. The reconciling includes one or some combination of the following: updating information relating to a virtual network entity in the service database 1 10 or 1 12 (and possibly updating a version of the virtual network entity in a managed node 106), or adding information relating to a virtual network entity to the service database 1 10 or 1 12 (and possibly adding the virtual network entity to a managed node 106), or removing information relating to a virtual network
entity from the service database 1 10 or 1 12 (and possibly removing the virtual network entity from a managed node 106).
[0024] There can be multiple different types of virtual network entities. Some example types of virtual network entities are depicted in Fig. 3. The different types of virtual network entities have a hierarchical relationship, as depicted in the
hierarchical arrangement shown in Fig. 3. The root of the hierarchical arrangement shown in the example of Fig. 3 is a network (which is one type of virtual network entity). Another type of virtual network entity is a subnet, which depends on the network. In other words, a subnet is part of the network (a network can include multiple subnets). Other virtual network entity types shown in Fig. 3 include a port (which depends on the network), a router (which depends on the subnet and the port), and a floating IP address (which depends on the router). As further shown in Fig. 3, another virtual network entity is a security enforcement service, which depends on the port. A floating IP address refers to an IP address that can be dynamically added to a virtual network entity, which in the case of Fig. 3 is the router. In some examples, floating IP addresses are supported in an OpenStack
architecture. In other examples, floating IP addresses can be used in other architectures.
[0025] A first network virtual entity depending on a second virtual network entity refers to the first virtual network entity having a relationship with the second virtual network entity where the first virtual network entity's operation or existence is in the context of the second virtual network entity. In the example hierarchical
arrangement shown in Fig. 3, the root (network) is at the top of the hierarchical arrangement, while a leaf is at the lowest level of the hierarchical arrangement. Two leaves exist in the hierarchical arrangement of Fig. 3: floating IP address and security enforcement service.
[0026] Fig. 4 is a flow diagram of a process performed by the reconciler 1 14, according to further implementations. The reconciler 1 14 determines (at 402) if loss of communication with a managed node N (managed node N is one of the managed nodes 106) has occurred. Loss of communication between the controller 108 and
managed node N can occur due to various reasons, including fault at the controller 108, fault at managed node N, or fault in the communications link between the controller 108 and managed node N. Note that the process of Fig. 4 is performed for each of the multiple managed nodes 106 in the network cloud infrastructure 100 that are associated with the controller 108.
[0027] In response to determining that loss of communication with managed node N has occurred, the reconciler 1 14 determines (at 404) if communication has been re-established with managed node N. If not, the reconciler 1 14 continues to wait for re-establishment of communication with managed node N. However, if communication has been re-established with managed node N, then the reconciler 1 14 proceeds to perform a cleanup phase 406.
[0028] The cleanup phase 406 iteratively performs tasks for each of multiple virtual network entity types (assuming managed node N has multiple types of virtual network entities). The iterative performance of tasks in the cleanup phase 406 starts at the virtual network entity type at the lowest level (the level closest to a leaf in Fig. 3) of the hierarchical arrangement. More generally, the iterative performance of tasks in the cleanup phase 406 iterates in an order of dependency such that any virtual network entity type A that depends upon virtual network entity type B appears earlier in the iteration than virtual network entity type B.
[0029] In the cleanup phase 406, the reconciler 1 14 compares (at 408) a listing (X) of a virtual network entities received from managed node N with a listing (Y) of virtual network entities at the controller 108. The listing (X) is retrieved or derived from the service database 1 12 of managed node N, while the listing (Y) of virtual network entities is retrieved or derived from the service database 1 10 of the controller 108.
[0030] The listing (X) can be received from managed node N in response to a query from the controller 108 to the managed node N. The listing (X) of virtual network entities and the listing (Y) of virtual network entities identify virtual network
entities of type T, which is the virtual network entity type that is currently considered in the iterative cleanup phase 406.
[0031 ] Based on the comparing, the reconciler 1 14 removes (at 410) a given virtual network entity of type T from managed node N if the given virtual network entity is not in listing (Y) of the controller 108.
[0032] Tasks 408 and 410 are iteratively performed for each of different virtual network entity types (T) in managed node N.
[0033] After the cleanup phase 406, the reconciler 1 14 proceeds to a
synchronization phase 412. The synchronization stage 412 iteratively performs tasks for each of multiple virtual network entity types in managed node N, starting at the highest level of the hierarchical arrangement (the level closest to the root in Fig. 3, for example). More generally, the iterative performance of tasks in the
synchronization phase 412 iterates in an order of dependency such that any virtual network entity type A that depends upon virtual network entity type B appears later in the iteration than virtual network entity type B.
[0034] For a current virtual network entity type T that is currently considered in the iterative synchronization phase 412, the reconciler 1 14 compares (at 414) a listing (X) of virtual network entities obtained from managed node (N) with a listing (Y) of virtual network entities of the controller 108. For each virtual network entity (E) of type T that is identified in the listing (X), if E exists in the listing (Y) but does not match a version stored in the listing (Y), then E is updated (at 416) in the listing (X). Updating E in the listing (X) also causes the version of the virtual network entity E to be updated at managed node N. A virtual network entity may exist but not match if there exists a virtual network entity where the characteristics (e.g. a unique identifier) that uniquely identify the virtual network entity match, but the remaining fields of the virtual network entity (other associated data) do not match. For example, listing (X) and listing (Y) may each identify a virtual network entity E that shares a common unique ID. However, listing (X) and listing (Y) may have at least one attribute of
virtual network entity E that differ in value, which means that the versions of virtual network entity E identified in listing (X) and listing (Y) are inconsistent.
[0035] Moreover, for each virtual network entity (E) identified in listing (Y), if E does not exist in listing (X), then the virtual network entity (E) is added (at 418) to the listing (X) (note that the virtual network entity (E) is also added to the managed node N).
[0036] Tasks 414, 416, and 418 are performed for each of the other virtual network entity types that exist in managed node N.
[0037] In further implementations, to improve performance, the listing (X) and the listing (Y) may be cached in cache memory accessible by the reconciler 1 14, so that such listings can be retrieved more quickly.
[0038] By employing techniques or mechanisms according to some
implementations, reconciliation between views of virtual network entities maintained by managed nodes 106 and the controller 108 occurs when a fault occurs (e.g. loss of communication detected at 402 in Fig. 4). As a result, messaging associated with such reconciliation is communicated when the fault occurs, which reduces network traffic since faults occur relatively infrequently.
[0039] Fig. 5 is a block diagram of an arrangement of the controller 108 according to some implementations. The controller 108 can include one or multiple processors 502, which can be coupled to one or multiple network interfaces 504 (to allow the controller 108 to communicate over a network), and to a non-transitory machine-readable or computer-readable storage medium 506 (or multiple storage media). The storage medium or storage media 506 can store the reconciler 1 14 in the form of machine-readable instructions, as well as the service database 1 10. The reconciler 1 14 can be loaded from the storage medium or storage media 506 for execution on the processor(s) 502. A processor can include a microprocessor, microcontroller, processor module or subsystem, programmable integrated circuit, programmable gate array, or another control or computing device.
[0040] The storage medium (or storage media) 506 can include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices. Note that the instructions discussed above can be provided on one computer- readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly plural nodes. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components. The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.
[0041 ] In the foregoing description, numerous details are set forth to provide an understanding of the subject disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and variations from the details discussed above. It is intended that the appended claims cover such modifications and variations.
Claims
What is claimed is: 1 . A method comprising:
receiving, by a controller in an infrastructure in which the controller manages nodes that provide virtual entities, information from a first of the nodes, the received information relating to a characteristic of a virtual entity provided by the first node; determining, by the controller, that the received information is inconsistent with information, maintained by the controller, relating to the characteristic of the virtual entity provided by the first node; and
in response to the determining, reconciling, by the controller, an inconsistency between the received information and the information maintained by the controller.
2. The method of claim 1 , further comprising:
selecting, by the controller, nodes on which respective virtual entities are deployed.
3. The method of claim 1 , further comprising:
querying, by the controller, a particular one of the nodes for identification of virtual entities on the particular node;
determining, by the controller, whether a given one of the virtual entities on the particular node is not identified by information maintained by the controller regarding virtual entities on the particular node; and
in response to determining that the given virtual entity on the particular node is not identified by the information maintained by the controller regarding virtual entities on the particular node, cause, by the controller, removal of the given virtual entity from the particular node.
4. The method of claim 3, wherein the querying comprises querying the particular node for identification of virtual entities of a first type from among a plurality of types of virtual entities on the particular node.
5. The method of claim 4, further comprising:
querying, by the controller, the particular node for identification of virtual entities of a second of the plurality of types on the particular node;
determining, by the controller, whether a given one of the second type of virtual entities on the particular node is not identified by the information maintained by the controller regarding virtual entities on the particular node; and
in response to determining that the given one of the second type of virtual entities on the particular node is not identified by the information maintained by the controller regarding virtual entities on the particular node, cause, by the controller, removal of the given one of the second type of virtual entities from the particular node.
6. The method of claim 5, wherein the virtual entities of the plurality of types on the particular node have hierarchical dependencies such that the first type of virtual entities depends on the second type of virtual entities, and wherein the querying for identification of virtual entities of the first type occurs before the querying for identification of virtual entities of the second type based on the hierarchical dependency between the first type of virtual entities and the second type of virtual entities.
7. The method of claim 1 , wherein determining that the received information is inconsistent with the information maintained by the controller comprises determining that at least one attribute in the received information does not match at least one attribute in the information maintained by the controller, the at least one attribute in the received information relating to the characteristic of the virtual entity provided by the first node.
8. The method of claim 1 , further comprising providing at least a subset of the virtual entities for use in a deployment of a tenant system associated with a customer of a provider of the infrastructure.
9. The method of claim 1 , further comprising removing a virtual entity from the first node in response to detecting an inconsistency.
10. A controller to control managed nodes in a network infrastructure, comprising: at least one processor to:
receive information of virtual network entities hosted by a first of the managed nodes, the received information relating to characteristics of the virtual network entities;
compare the received information to information maintained by the controller for the virtual network entities hosted by the first managed node; and
in response to the comparing indicating inconsistency between the received information and the information maintained by the controller, cause a change at the first managed node.
1 1 . The controller of claim 10, wherein the change at the first managed node includes at least one of updating at least one virtual network entity at the first managed node, removing at least one virtual network entity from the first managed node, and adding at least one virtual network entity to the first managed node.
12. The controller of claim 10, wherein the at least one processor is to perform the receiving, the comparing, and the causing the change in response to detecting a loss of communication between the controller and at least one of the managed nodes.
13. The controller of claim 10, wherein the receiving, the comparing, and the causing the change are part of a synchronization phase in which the change includes at least one of updating a version of a virtual network entity at the first managed node and adding a virtual network entity to the first managed node.
14. The controller of claim 13, wherein the at least one processor is to further perform a cleanup phase to remove a virtual network entity from the first managed node in response to detecting inconsistency between information of the first managed node and information maintained by the controller.
15. An article comprising at least one machine-readable storage medium storing instructions that upon execution cause a controller to:
receive information from a first of a plurality of nodes of an infrastructure that are managed by the controller, the received information relating to a characteristic of a virtual entity provided by the first node;
determine whether the received information is inconsistent with information, maintained by the controller, relating to the characteristic of the virtual entity provided by the first node;
in response to determining that the received information is inconsistent with the information maintained by the controller, reconcile an inconsistency between the received information and the information maintained by the controller; and
in response to determining that the received information is not inconsistent with the information maintained by the controller, determining to not perform the reconciling.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2014/032154 WO2015147859A1 (en) | 2014-03-28 | 2014-03-28 | Reconciling information in a controller and a node |
US15/117,466 US10742505B2 (en) | 2014-03-28 | 2014-03-28 | Reconciling information in a controller and a node |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2014/032154 WO2015147859A1 (en) | 2014-03-28 | 2014-03-28 | Reconciling information in a controller and a node |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015147859A1 true WO2015147859A1 (en) | 2015-10-01 |
Family
ID=54196174
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2014/032154 WO2015147859A1 (en) | 2014-03-28 | 2014-03-28 | Reconciling information in a controller and a node |
Country Status (2)
Country | Link |
---|---|
US (1) | US10742505B2 (en) |
WO (1) | WO2015147859A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105872120A (en) * | 2015-12-14 | 2016-08-17 | 乐视云计算有限公司 | Public network IP processing method and device |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10820285B2 (en) * | 2018-11-26 | 2020-10-27 | Hitron Technologies Inc. | Method for network synchronization |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060190454A1 (en) * | 2005-02-21 | 2006-08-24 | Hiroki Kato | Network system, element thereof and network visualization method |
US20110051622A1 (en) * | 2007-03-09 | 2011-03-03 | Anne-Marie Cristina Bosneag | System, Method and Network Node for Checking the Consistency of Node Relationship Information in the Nodes of a Strongly Connected Network |
WO2013020126A1 (en) * | 2011-08-04 | 2013-02-07 | Midokura Pte. Ltd. | System and method for implementing and managing virtual networks |
US20130060819A1 (en) * | 2010-07-06 | 2013-03-07 | W. Andrew Lambeth | Distributed network control system with one master controller per logical datapath set |
US20130301658A1 (en) * | 2011-01-28 | 2013-11-14 | Nec Corporation | Communication system, control device, forwarding node, communication control method, and program |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8554750B2 (en) * | 2009-01-15 | 2013-10-08 | Bmc Software, Inc. | Normalization engine to manage configuration management database integrity |
US9430494B2 (en) * | 2009-12-28 | 2016-08-30 | Oracle International Corporation | Spatial data cartridge for event processing systems |
US9202225B2 (en) * | 2010-05-28 | 2015-12-01 | Red Hat, Inc. | Aggregate monitoring of utilization data for vendor products in cloud networks |
US8452675B2 (en) * | 2010-08-06 | 2013-05-28 | Fisher-Rosemount Systems, Inc. | Automatic receiving, inspection, inventory management and reporting using wireless communications |
US20120151040A1 (en) * | 2010-12-13 | 2012-06-14 | Mouravyov Sergei | Computer inventory data consolidation |
US9479323B2 (en) | 2011-01-28 | 2016-10-25 | Nec Corporation | Communication system, forwarding node, control device, communication control method, and program |
EP2690831A4 (en) | 2011-03-23 | 2014-12-10 | Nec Corp | Communication control system, switch node, and communication control method |
US20130114607A1 (en) * | 2011-11-09 | 2013-05-09 | Jeffrey S. McGovern | Reference Architecture For Improved Scalability Of Virtual Data Center Resources |
US8995272B2 (en) | 2012-01-26 | 2015-03-31 | Brocade Communication Systems, Inc. | Link aggregation in software-defined networks |
-
2014
- 2014-03-28 WO PCT/US2014/032154 patent/WO2015147859A1/en active Application Filing
- 2014-03-28 US US15/117,466 patent/US10742505B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060190454A1 (en) * | 2005-02-21 | 2006-08-24 | Hiroki Kato | Network system, element thereof and network visualization method |
US20110051622A1 (en) * | 2007-03-09 | 2011-03-03 | Anne-Marie Cristina Bosneag | System, Method and Network Node for Checking the Consistency of Node Relationship Information in the Nodes of a Strongly Connected Network |
US20130060819A1 (en) * | 2010-07-06 | 2013-03-07 | W. Andrew Lambeth | Distributed network control system with one master controller per logical datapath set |
US20130301658A1 (en) * | 2011-01-28 | 2013-11-14 | Nec Corporation | Communication system, control device, forwarding node, communication control method, and program |
WO2013020126A1 (en) * | 2011-08-04 | 2013-02-07 | Midokura Pte. Ltd. | System and method for implementing and managing virtual networks |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105872120A (en) * | 2015-12-14 | 2016-08-17 | 乐视云计算有限公司 | Public network IP processing method and device |
Also Published As
Publication number | Publication date |
---|---|
US10742505B2 (en) | 2020-08-11 |
US20160359669A1 (en) | 2016-12-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11463316B2 (en) | Topology explorer | |
US10574513B2 (en) | Handling controller and node failure scenarios during data collection | |
US10868833B2 (en) | DNS or network metadata policy for network control | |
US10198338B2 (en) | System and method of generating data center alarms for missing events | |
CN110710161B (en) | Generating device-level logical models of a network | |
CN110710160B (en) | Method and system for generating network-wide logic model for network policy analysis | |
US9621512B2 (en) | Dynamic network action based on DHCP notification | |
US10820212B2 (en) | System and method for partitioned wireless telecommunication network deployment | |
WO2014123831A1 (en) | Method and apparatus for providing migration of cloud components across address domains | |
EP3643014A1 (en) | Identifying mismatches between a logical model and node implementation | |
WO2018236706A1 (en) | Network node memory utilization analysis | |
US11722356B2 (en) | Enabling integration of solutions with software-defined networking platform | |
WO2018236569A1 (en) | Identifying components for removal in a network configuration | |
US20160092537A1 (en) | Polling based synchronization in managed networks | |
US11736410B1 (en) | Synchronizing device resources for element management systems | |
US9060027B2 (en) | Assigning location identifiers to nodes in a distributed computer cluster network environment | |
US20140297821A1 (en) | System and method providing learning correlation of event data | |
US9985840B2 (en) | Container tracer | |
US20220358108A1 (en) | Historical graph database | |
US10742505B2 (en) | Reconciling information in a controller and a node | |
US11108854B2 (en) | Peer-to-peer network for internet of things resource allocation operation | |
US20150170037A1 (en) | System and method for identifying historic event root cause and impact in a data center | |
EP1479192B1 (en) | Method and apparatus for managing configuration of a network | |
US20200092255A1 (en) | Enhanced communication of service status information in a computing environment | |
US20170329672A1 (en) | Methods for providing global spare data storage device management and devices thereof |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14887494 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15117466 Country of ref document: US |
|
NENP | Non-entry into the national phase | ||
122 | Ep: pct application non-entry in european phase |
Ref document number: 14887494 Country of ref document: EP Kind code of ref document: A1 |