WO2016106486A1 - Method and apparatus for configuring a communication system - Google Patents
Method and apparatus for configuring a communication system Download PDFInfo
- Publication number
- WO2016106486A1 WO2016106486A1 PCT/CN2014/095280 CN2014095280W WO2016106486A1 WO 2016106486 A1 WO2016106486 A1 WO 2016106486A1 CN 2014095280 W CN2014095280 W CN 2014095280W WO 2016106486 A1 WO2016106486 A1 WO 2016106486A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- configuration
- cluster
- clusters
- node
- common
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/04—Network management architectures or arrangements
- H04L41/044—Network management architectures or arrangements comprising hierarchical management structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/04—Network management architectures or arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/74—Address processing for routing
- H04L45/745—Address table lookup; Address filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
Definitions
- the present invention relates to a method and apparatus for configuring a communication system, for example relating to large system configuration.
- the configuration of a communication system which comprises a plurality of nodes and servers, and in particular nodes and servers which are organised into clusters, is a complex task.
- the updating of a service oriented behaviour usually requires some parameters to be configured in each node of the communication system. This not only causes long maintenance windows for such updates, but is also prone to error.
- the application nodes and configuration servers are usually tightly coupled or closely related, which means that it is difficult to support dynamic scaling-out for business needs. It is difficult to make a configuration change on all nodes of a cluster, or on all clusters in a system. It is also difficult to ensure configuration consistency between various nodes of a cluster, or between multiple clusters.
- a method in a configuration server for configuring a communication system comprises configuring a plurality of nodes in a first cluster, and configuring a plurality of nodes in at least a second cluster, such that the configuration server acts as a common configuration server for at least first and second clusters.
- the configuration server comprises a processing module adapted to configure a plurality of nodes in a first cluster, and configure a plurality of nodes in at least a second cluster, such that the configuration server acts as a common configuration server for at least first and second clusters.
- a method in a node of a communication system wherein the node is associated with a cluster comprising a plurality of nodes.
- the method comprises determining a configuration service address relating to a common configuration server that is adapted to configure the cluster associated with the node and at least one other cluster.
- the method comprises sending a request for configuration information to the common configuration server, and receiving configuration information from the common configuration server.
- a computer program comprising instructions which, when executed on at least one processor, causes the at least one processor to carry out the method according to any one of the appended claims.
- a carrier comprising the computer program as described above, wherein the carrier is one of an electronic signal, optical signal, radio signal or computer readable storage medium.
- Figure 1 shows an example of a communication system
- Figure 2 shows an example of a method according to an embodiment of the present invention
- Figure 3 shows an example of a communication system according to an embodiment of the present invention
- Figure 4 illustrates an example of how clusters of a communication system may be grouped in a hierarchical structure
- Figure 5 provides further detail of how clusters of a communication system may be grouped in a hierarchical structure
- Figure 6 shows an example of a method according to an embodiment of the present invention
- Figure 7 illustrates an example of an application of an embodiment of the present invention
- Figure 8 shows an example of a method according to an embodiment of the present invention
- Figure 9 illustrates another example of an application of an embodiment of the present invention.
- Figure 10 shows an example of a configuration server according to an embodiment of the present invention.
- a cluster system normally contains many servers (or nodes) .
- the servers may be of the same type, or of different types.
- a cluster system could contain 2 Load Balancing Servers (LBSs) , 40 Traffic Processing Servers (TPSs) , and 1 Configuration Server (CS) .
- LBSs Load Balancing Servers
- TPSs Traffic Processing Servers
- CS Configuration Server
- the configuration server manages the configuration of each server in its cluster separately.
- each load balancing server or traffic processing server can have its own configuration in the configuration server, although most of the configurations are the same across all load balancing servers and most of the configurations are the same across all traffic processing servers. The reason is two-fold. Different types of server require different sets of configuration parameters, and each server typically has its own distinct values for some specific configuration parameters, for example, the IP address that a service should start on.
- a configuration that is intended to have the same value across servers or nodes will be referred to as a "common configuration”
- a configuration that is intended to have a distinct value for different servers or nodes will be referred to as a "specific configuration” .
- the operator needs to generate the new configuration for the traffic processing server by copying the configuration of an existing traffic processing server, and carrying out changes for the specific configuration of the new traffic processing server.
- an operator needs to add or change some configuration specifically on one type of server, and add or change some configuration specifically on another type of server. For example, in order to support traffic processing for TCP port 8080, the operator needs to add "8080" to a configuration parameter of all load balancing servers, and also add "8080" to another configuration parameter of all traffic processing servers.
- Each cluster has its own configuration server, with its own copy of a configuration. While a small portion of a configuration will be specific for each cluster (e.g. due to local province regulation) , most of the configurations will be the same across all clusters.
- a new cluster In a scenario where a new cluster is to be added to the system, typically an operator will need to deploy a new configuration server, login to a running configuration server of an existing cluster, and copy all of that configuration to the configuration of the new cluster; and make slight changes to reflect some specific requirements of the new cluster. After the configuration server of the new cluster is populated with a proper configuration, only then can the servers that process traffic or tasks (e.g. load balancing servers and traffic processing servers) be started. In this process, many human interventions are required, for example when deploying a configuration server, accessing another configuration server, copying an existing configuration, making specific configuration changes, all of which are inefficient and also error prone.
- process traffic or tasks e.g. load balancing servers and traffic processing servers
- FIG. 1 shows a configuration of a system in which an operator has four clusters 103 1 to 103 4 .
- each cluster 103 is shown as comprising a plurality of nodes of a first type 107 and a plurality of nodes of a second type 109, and a configuration server 105 for managing the configuration of each node within the cluster.
- a cluster may contain further nodes, including further nodes of other types, but for ease of reference only first and second node types 107, 109 are shown.
- the dotted lines 111 represent manual configuration of the configuration servers 105, for example by a system administrator 101.
- the solid lines between the configuration servers 105 and their respective nodes 107, 109 represent, for example, private communication via a cluster-internal network, which is not directly accessible from outside.
- a first cluster 103 1 and second cluster 103 2 may be provided for a particular service, e.g. Web Optimization, while a third cluster 103 3 and a fourth cluster 103 4 may be provided for another service, e.g. Video Optimization.
- the configuration of each node 107, 109 in each cluster 103 is distinct, for example due to each configuration containing a local configuration, such as an IP address to start the respective Web Optimization or Video Optimization services.
- the configuration between each node 107, 109 is difficult to synchronize. In a scenario where a common configuration for all clusters 103 needs to be updated, the system administrator 101 has to log into the configuration cluster 105 of each cluster 103 one by one to repeatedly make the same change. This is not only a tedious action but also error prone.
- each node 107, 109 has a copy of the common configuration.
- the administrator has to maintain n copies of the common configuration.
- the administrator has to maintain m*n copies of the common configuration.
- This existence of multiple copies of the common configuration has several drawbacks. For example, it is difficult to make changes on common configurations. It is also difficult to ensure the consistency of the common configurations in different nodes. Memory space is also wasted, due to the need to store a copy of the common configuration of each node.
- the configurations of clusters in a multiple cluster system are managed by one configuration server.
- the configurations of these clusters are managed in a hierarchical way, where the lower level clusters can inherit configurations from higher level clusters, further details of which will be described later in the application.
- a new representation of a node specific configuration is provided, whereby a node specific configuration is elevated to be a common configuration of a cluster.
- This representation approach supports a node level configuration localization (or translation) from the common configuration, further details of which will also be described later in the application.
- Figure 2 shows a method in a configuration server for configuring a communication system, according to a first embodiment of the present invention.
- the method comprises configuring a plurality of nodes in a first cluster (step 201) .
- the method in the configuration server further comprises the step of configuring a plurality of nodes in at least a second cluster (step203) , such that the configuration server acts as a common configuration server for at least first and second clusters.
- the method comprises configuring a plurality of clusters, comprising at least the first cluster and the second cluster, in a hierarchical structure.
- An advantage of the hierarchical structure of clusters is that only one configuration server is required to support multiple clusters, which provides efficiency on resource usage. Such a hierarchical structure also makes it easier to make a common configuration change, for example whereby a one time change can take effect on all lineal-descendant clusters.
- the hierarchical structure of clusters also provides flexibility to introduce a new variant of cluster.
- the new variant can be placed in a proper hierarchy, with the specific configuration parameters of the cluster then being set. All the other configurations may use inherited values.
- the hierarchical structure also helps ensure configuration consistency by its nature, as the configurations that are intended to be common only have one copy.
- Figure 3 illustrates an example of a hierarchical structure in a system in which an operator has four real clusters 103 1 to 103 4 , (which for example have nodes taking traffic) .
- First and second clusters 103 1 and 103 2 are for a service, e.g. Web Optimization, while third and fourth clusters 103 3 and 103 4 are for another service, e.g. Video Optimization.
- FIG. 3 shows a common configuration server 113.
- the common configuration server 113 may be controlled or administered, for example, by a system administrator 101.
- the four real clusters 103 1 to 103 4 are grouped in this example into two logical clusters: Cluster 1 for service 1 (relating to real clusters 103 1 and 103 2 , which will also be referred to herein as Cluster 1.1 and Cluster 1.2 respectively) and Cluster 2 for service 2 (relating to real clusters 103 3 and 103 4 , which will also be referred to herein as Cluster 2.1 and Cluster 2.2 respectively) .
- the clusters may be grouped logically, for example, based on configuration parameters they have in common in view of their common service. Other methods may also be used to logically group clusters.
- the two logical clusters, i.e. Cluster 1 and Cluster 2 are further grouped to one logical cluster, the root cluster, Cluster 0. It is noted that further layers and further clusters may be provided in a hierarchical structure, but the example of Figure 3 has been simplified for ease of reference.
- Figure 4 shows another representation of the hierarchical structure of the clusters illustrated in the example of Figure 3.
- Cluster 0, at root level comprises a root configuration, Configuration 0;
- Cluster 1 at a first hierarchical level below root level, comprises a first level configuration, Configuration 1;
- Cluster 1.1 at a second hierarchical level below root level, comprises a second level configuration, Configuration 1.1;
- Cluster 1.2 at a second hierarchical level below root level, comprises a second level configuration, Configuration 1.2;
- Cluster 2 at a first hierarchical level below root level, comprises a first level configuration, Configuration 2;
- Cluster 2.1 at a second hierarchical level below root level, comprises a second level configuration, Configuration 2.1;
- Cluster 2.2 at a second hierarchical level below root level, comprises a second level configuration, Configuration 2.2;
- ⁇ Cluster 0 is the parent of Cluster 1 and Cluster 2.
- ⁇ Cluster 1 is the parent of Ccluster 1.1 and Cluster 1.2.
- ⁇ Cluster 2 is the parent of Cluster 2.1 and Cluster 2.2.
- Figure 5 provides an example of how the configuration parameters may interrelate.
- Configuration parameters that are intended to be common for all clusters are configured on the root cluster, Cluster 0 (this being the highest level in the hierarchical structure) .
- configuration parameters p1, p2 and p3 are shown as being common to all clusters (and shown as comprising values “a” , “b” and “c” , respectvely) .
- Configuration parameters that are intended to be common between Cluster 1.1 and Cluster 1.2 are configured on Cluster 1 (this being at a first level in the hierarchical structure) .
- configuration parameters p4 and p5 are shown as being common to clusters at this level, i.e. common to Cluster 1.1 and Cluster 1.2 (and shown as comprising values “d” and “e” , respectively) .
- configuration parameters that are common between Cluster 2.1 and Cluster 2.2 are configured on Cluster 2 (this being at a first level in the hierarchical structure) .
- configuration parameters p4 and p5 are shown as being common to clusters at this level, i.e. common to Cluster 2.1 and Cluster 2.2 (and shown as comprising values “h” and “i” , respectively) .
- Configuration parameter (s) that are intended to be specific for a cluster are configured on the lowest level clusters, in the example of Figure 5 the lowest level being the second level below root level, and comprising Cluster 1.1, Cluster 1.2, Cluster 2.1 and Cluster 2.2.
- configuration parameter p6 is shown as being specific to each cluster (and shown as comprising values “f” , “g” , “j” and “k” , respectively, for each cluster) .
- reference “p” represents a particular configuration parameter, e.g. IP address to start an application, and that a value, e.g. “f” , is the value of that configuration parameter, for example the IP start address specific for Cluster 1.1.
- a value e.g. “f”
- the configuration parameter “p4” for example, both clusters 1.1 and 1.2 share the same value for this configuration parameter, i.e. “d” .
- configuration parameters that might be shared between clusters. Examples of configuration parameters that might be shared between clusters include, for exampe:
- timeout values for different communication protocols e.g. TCP connection timeout, HTTP request timeout, etc. ;
- -IP addresses of external nodes e.g. black-listed IP addresses, IP address of billing system, etc.
- Figure 5 therefore shows the actual configuration of each cluster, Cluster 1.1, Cluster 1.2, Cluster 1.3 and Cluster 1.4 after combining its own specifc configuration parameter (s) with inherited configuration parameters.
- Cluster 1.1 comprises configuration parameter p6 (having value “f” which is specifc to Cluster 1.1) , configuration parameters p4 and p5 (having values “d” and “e” respectively) which have been inherited from parent Cluster 1 at a higher hierarchical level, and configuration parameters p1, p2 and p3 (having values “a” , “b” and “c” respectively) which have been inherited from grandparent Cluster 0, or the root cluster.
- configuration parameter p6 having value “f” which is specifc to Cluster 1.1
- configuration parameters p4 and p5 having values “d” and “e” respectively
- configuration parameters p1, p2 and p3 having values “a” , “b” and “c” respectively
- a method according to an embodiment of the present invention comprises configuring a plurality of clusters in a hierarchical structure by grouping one or more clusters into logical groups of clusters.
- Clusters in a logical group of clusters comprise at least one configuration parameter in common.
- the common configuration server is provided at a root level in the hierarchical structure, and wherein the root level comprises a configuration (e.g. Configuration 0) comprising one or more configuration parameters which are common to all clusters being configured by the common configuration server.
- a configuration e.g. Configuration 0
- clusters in a logical group of clusters at a first level below the root level share one or more configuration parameters common to that logical group of clusters, and inherit one or more configuration parameters from the root level.
- the hierarchical structure comprises a plurality of levels, wherein clusters in a logical group of clusters at an intermediate hierarchical level share one or more configuration parameters common to that logical group of clusters, and inherit one or more configuration parameters from a parent hierarchical level.
- the one or more configuration parameters common to a logical group of clusters at the intermediate hierarchical level may be passed down to one or more child clusters.
- a cluster at a lowest hierarchical level (e.g. Clusters 1.1, 1.2, 2.1, 2.2) comprises at least one configuration parameter specific to that cluster, and inherits one or more configuration parameters from a parent cluster, and any grandparent clusters or any higher clusters in the hierarchy. It is noted that the lowest level cluster, e.g. Cluster 1.1, may have more than just one configuration parameter which is specific to that cluster.
- the hierarchical structure comprises a plurality of hierarchical levels, and wherein one or more clusters at a lower hierarchical level are adapted to inherit one or more configuration parameters from one or more clusters at a higher hierarchical level.
- the actual configuration of a cluster may be a combination of its own configuration specifc to that cluster, and the actual configuration of its parent, and whereby its own configuration takes precedence.
- a method performed in a common configuration server that is adpated to configure a plurality of clusters in this way may further comprise the steps of receiving a request from a first node to receive configuration information for the first node, step 601.
- the method comprises determining from cluster identity information received with the request, which cluster the first node is associated with, step 603.
- Configuration information is retrieved for the identified cluster, step 605, and the configuration information then sent to the first node, 607.
- Figure 7 shows an example of how a node (or node instance in a cloud environment) is able to obtain configuration information from a common configuration server 113 according to an embodiment of the present invention.
- a new node (Node A) may be bootstrapped with information comprising a cluster identity (being the identity of the cluster the node is to be associated with) and an address of a configuration service (for example an IP address and Port associated with the common configuration server described above) , as shown by step 1.
- Other information may also be bootstrapped to the node during a set-up or initialization procedure.
- the node sends a request for configuration information to the common configuration server 113.
- the address of the common configuration server is determined, for example, from an address of the configuration service that has been obtained in a bootstrap procedure as described above.
- the request sent in step 2 may comprise, for example, the sending of a Hypertext Transfer Protocol, HTTP, GET request for the configuration of Node A.
- the interface between the configuration service and the node may comprise, for example, a RESTful interface (i.e. a Representational State Transfer architecture that complies to certain architectural constraints) .
- the HTTP GET request comprises the cluster identity, i.e. the identity of the cluster that the new node is to be associated with.
- the configuration service e.g. common configuration server 113 returns the configuration information for this request according to the cluster identity.
- the configuration information may comprise, for example, configuration files represented in JavaScript Object Notation (JSON) files.
- step 4 according to one example the configuration information downloaded from the configuration service is then modified or localized to a node level configuration, and the configuration parameters can then be applied to a local node application. Further details about the translation at local level will be described later in the application.
- the configuration of multiple clusters can be managed by the configuration service, i.e. a common configuration server 113.
- the clusters are organized in a hierarchical way. Each cluster can have its own configuration.
- a configuration of a cluster is inherited by its children clusters, and a child configuration can have its own configuration value.
- the actual configuration of a cluster is a combination of its own configuration and the actual configuration of its parent (and any higher clusters in the hierarchy) , whereby its own configuration takes precedence.
- a method in a node of a communication system wherein the node is associated with a cluster comprising a plurality of nodes.
- the method comprises determining a configuration service address relating to a common configuration server that is adapted to configure the cluster associated with the node and at least one other cluster, step 801.
- a request for configuration information is sent to the common configuration server.
- configuration information is received from the common configuration server.
- a node can perform localization or translation of a common configuration received form a configuration server or common configuration service, into a configuration local to that node.
- the localization may be used in conjunction with the embodiments described herein concerning hierarchical configuration using a common configuration server, or alone in a single cluster comprising a plurality of nodes.
- a large system could have thousands of nodes.
- Each node may need to have an unique configuration on the configuration server. This is due to the fact that the configuration usually contains some node specific configuration values, e.g. an IP address on which an application is to be started.
- a node corresponding to cluster 1.1 may comprise a node specific configuration comprising configuration parameter p6 having value “f” (e.g. “f” being the starting IP address of an application at that node) .
- Managing thousands of unique configurations involves complexity on changing a common configuration on all nodes, ensuring consistency of common configuration across all nodes, and adding configuration for new nodes in the cluster.
- a method to elevate a node specific configuration to become a common configuration This has an advantage that a cluster then only needs to have one common configuration, whereby all nodes in the cluster can then use this common configuration.
- the embodiment proposes to set an alias configuration parameter as the value.
- Each node can be adapted to perform a localization procedure to translate the alias configuration parameter to its real node specific value, i.e. the specific configuration parameter for the node.
- an alias configuration parameter can follow a pattern such as:
- the “ ⁇ Separator>Tag” may be used to inform the node that a localization or translation procedure is to be performed.
- the above text is translated to the value of Attribute of Object.
- the specific configuration parameter may be set instead to a value such as "#LOCALIZE#ipv4#eth1#".
- the node receives this alias representation, the node performs a localization procedure to translate the alias to its first ipv4 address of interface 'eth1' (that is, “1.2.3.4” ) , before really applying the value to an application.
- a method in a node upon receiving configuration information from a configuration server, such as a common configuration server, a method in a node further comprises performing a translation procedure on the received configuration information to derive a node level configuration.
- Performing a translation procedure may comprise determining an alias configuration parameter in the configuration information received from the configuration server, and mapping the alias configuration parameter to a specific configuration parameter which is specific to the local node.
- Mapping information may be stored at the node, for example wherein the mapping information provides a relationship between an alias configuration parameter received from a configuration server and a specific configuration parameter for the node.
- Figure 9 is provided as another example illustrating how the translation (localization) procedure may be provided within an overall configuring procedure.
- a first step 1 it is assumed that Node A is starting up.
- the Node A performs a local IP discovery. This may involve determining a starting IP address, which identifies the IP address of the common configuration service.
- the Node A sends a request for configuration information to a configuration service (such as a common configuration server) , and in step 4 receives configuration information files.
- a configuration service such as a common configuration server
- step 5 the Node A translates an alias configuration parameter to a node specific value or values.
- the Node A applies the configuration to an application.
- the system configuration may be provided as a service by a common configuration server.
- the common configuration server is a node within a communication system, or part of another node, wherein the configuration service runs on such a node.
- the common configuration service is provided as a cloud based service or platform, for example a web based service. Any node that has permission to use this service can connect to this server and use the configuration service to get proper configuration.
- Figure 10 shows an example of a configuration server 113, according to an embodiment of the present invention, for configuring a communication system.
- the configuration server 113 comprising a processing module 115 adapted to configure a plurality of nodes in a first cluster, and configure a plurality of nodes in at least a second cluster, such that the configuration server 113 acts as a common configuration server for at least first and second clusters.
- the processing module 115 may be adapted to perform any of the steps described herein for the common configuration server.
- a node can use a protocol such as HTTP, for example, to download its configuration from such a configuration server.
- a protocol such as HTTP
- a node specific configuration is elevated to be a common configuration, such that a cluster can have only one copy of a configuration, which all nodes in the cluster can share.
- the clusters can be organized into logical groups. The configuration of a cluster may be inherited by its children clusters. Therefore a common configuration of a set of clusters can be put to a higher level cluster, and the operator just needs to maintain only one copy of the common configuration across clusters.
- configuration service and node are loosely coupled, which has an advantage in that both ends can expand or evolve with minimal impact to the other.
- a node may be bootstrapped with information including cluster identity, configuration service address, and some other information.
- the configuration service address can be used by a node to contact the configuration service.
- the cluster identity enables the node to retrieve the configuration files that are available to that cluster.
- the embodiments of the present invention enable the configuration management of a large system to be carried out much simpler and more efficiently.
- the embodiments also provide a better support to the high elasticity requirement in cloud computing environment.
- high elasticity it is meant that it should be easy to adapt the number of servers in a cluster depending on the traffic profile. For example, it is advantageous not to require any extra configuration to add servers to a cluster at a busy period of traffic, or to remove servers from a cluster during low traffic.
- the embodiments above have the advantage of providing simple and efficient configuration management. For example, an operator just needs to perform service oriented configuration change for a cluster. The complexity of the mapping between service oriented configuration and configurations of various types of nodes are transparent to the operator.
- the hierarchical configuration architecture keeps minimized copies of a common configuration across clusters. Thus, it is comparatively easier to make a common configuration change to all clusters. The consistency of configurations across clusters is also improved due to there being only one copy of the common configuration in existence.
- node specific configurations can be elevated to be a common configuration, all the nodes in a cluster can share the same copy of the common configuration, which has an advantage in that an operator does not need to do any configuration change when the cluster is scaling-out, i.e. becoming larger.
- the embodiments of the invention may be platform independent, and may be used for a bare-metal machine, a virtualization environment and a Cloud environment.
- a “bare-metal” machine is a traditional server, on which the operating system has direct access to the hardware (e.g. a Linux server, or Windows on a laptop, etc. ) .
- the hardware e.g. a Linux server, or Windows on a laptop, etc.
- hypervisor e.g. a software for a virtual server
- the operating system of the virtual server has no direct access to the hardware. All calls are handled via the hypervisor.
- a method for configuring a communication system comprising one or more clusters, each cluster comprising a plurality of associated communication nodes of one or more type, the method comprising controlling the configuration of the plurality of clusters using a common, or single, configuration server.
- the plurality of clusters may be controlled in a hierarchical structure. Any configuration parameters that are common to a group of first level clusters of a first level in the hierarchy are inherited from a common cluster at second level, the second level being higher than the first level, and any parameters common to clusters at the second level are inherited from a common cluster at a third level, the third level being higher than the second level, and so on until a root level is reached where the configuration parameters of the root level are common to all clusters in the system.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
- Stored Programmes (AREA)
Abstract
A method in a configuration server for configuring a communication system comprises configuring a plurality of nodes in a first cluster, and configuring a plurality of nodes in at least a second cluster, such that the configuration server acts as a common configuration server for at least first and second clusters. A method in a node associated with a cluster comprises determining a configuration service address relating to a common configuration server that is adapted to configure the cluster associated with the node and at least one other cluster, sending a request for configuration information to the common configuration server, and receiving configuration information from the common configuration server.
Description
The present invention relates to a method and apparatus for configuring a communication system, for example relating to large system configuration.
The configuration of a communication system which comprises a plurality of nodes and servers, and in particular nodes and servers which are organised into clusters, is a complex task. For example, the updating of a service oriented behaviour usually requires some parameters to be configured in each node of the communication system. This not only causes long maintenance windows for such updates, but is also prone to error.
Furthermore, in existing communication systems the application nodes and configuration servers are usually tightly coupled or closely related, which means that it is difficult to support dynamic scaling-out for business needs. It is difficult to make a configuration change on all nodes of a cluster, or on all clusters in a system. It is also difficult to ensure configuration consistency between various nodes of a cluster, or between multiple clusters.
Summary
It is an aim of the present invention to provide a method and apparatus which obviate or reduce at least one or more of the disadvantages mentioned above.
According to a first aspect of the present invention there is provided a method in a configuration server for configuring a communication system. The method comprises configuring a plurality of nodes in a first cluster, and configuring a plurality of nodes in at least a second cluster, such that the configuration server acts as a common configuration server for at least first and second clusters.
According to another aspect of the present invention there is provided a configuration server for configuring a communication system. The configuration server comprises a processing module adapted to configure a plurality of nodes in a first cluster, and configure a plurality of nodes in at least a second cluster, such that the configuration server acts as a common configuration server for at least first and second clusters.
According to another aspect of the present invention there is provided a method in a node of a communication system, wherein the node is associated with a cluster comprising a plurality of nodes. The method comprises determining a configuration service address relating to a common configuration server that is adapted to configure the cluster associated with the node and at least one other cluster. The method comprises sending a request for configuration information to the common configuration server, and receiving configuration information from the common configuration server.
According to another aspect of the present invention, there is provided a computer program, comprising instructions which, when executed on at least one processor, causes the at least one processor to carry out the method according to any one of the appended claims.
According to another aspect of the present invention, there is provided a carrier comprising the computer program as described above, wherein the carrier is one of an electronic signal, optical signal, radio signal or computer readable storage medium.
Brief description of the drawings
For a better understanding of examples of the present invention, and to show more clearly how the examples may be carried into effect, reference will now be made, by way of example only, to the following drawings in which:
Figure 1 shows an example of a communication system;
Figure 2 shows an example of a method according to an embodiment of the present invention;
Figure 3 shows an example of a communication system according to an embodiment of the present invention;
Figure 4 illustrates an example of how clusters of a communication system may be grouped in a hierarchical structure;
Figure 5 provides further detail of how clusters of a communication system may be grouped in a hierarchical structure;
Figure 6 shows an example of a method according to an embodiment of the present invention;
Figure 7 illustrates an example of an application of an embodiment of the present invention;
Figure 8 shows an example of a method according to an embodiment of the present invention;
Figure 9 illustrates another example of an application of an embodiment of the present invention; and
Figure 10 shows an example of a configuration server according to an embodiment of the present invention.
A cluster system normally contains many servers (or nodes) . The servers may
be of the same type, or of different types. As an example, a cluster system could contain 2 Load Balancing Servers (LBSs) , 40 Traffic Processing Servers (TPSs) , and 1 Configuration Server (CS) . The configuration server manages the configuration of each server in its cluster separately. For example, each load balancing server or traffic processing server can have its own configuration in the configuration server, although most of the configurations are the same across all load balancing servers and most of the configurations are the same across all traffic processing servers. The reason is two-fold. Different types of server require different sets of configuration parameters, and each server typically has its own distinct values for some specific configuration parameters, for example, the IP address that a service should start on.
In order to simplify the description herein, a configuration that is intended to have the same value across servers or nodes will be referred to as a "common configuration" , and a configuration that is intended to have a distinct value for different servers or nodes will be referred to as a "specific configuration" .
In a scenario where a common configuration is to be changed, a system administrator needs to repeat the same change on every server or node.
Although the tedious can be reduced by batch processing, there is still a concern over the consistency of the common configuration across all servers, since every operation has the possibility of failure.
In a scenario where the capacity of a cluster is to be expanded by adding a new traffic processing server, the operator needs to generate the new configuration for the traffic processing server by copying the configuration of an existing traffic processing server, and carrying out changes for the specific configuration of the new traffic processing server.
In a scenario where a new service is to be added, an operator needs to add or change some configuration specifically on one type of server, and add or
change some configuration specifically on another type of server. For example, in order to support traffic processing for TCP port 8080, the operator needs to add "8080" to a configuration parameter of all load balancing servers, and also add "8080" to another configuration parameter of all traffic processing servers.
Large telecom operators normally have multiple same-purpose cluster systems, which can be seen as one system with multiple clusters. Each cluster has its own configuration server, with its own copy of a configuration. While a small portion of a configuration will be specific for each cluster (e.g. due to local province regulation) , most of the configurations will be the same across all clusters.
In a scenario where a configuration parameter that is intended to have an equal value across all clusters is to be changed, the operator needs to repeat the same change on every cluster. Although the tedious can be reduced by batch processing, there is still a concern over the consistency of the common configuration across all clusters, since every operation has the possibility of failure.
In a scenario where a new cluster is to be added to the system, typically an operator will need to deploy a new configuration server, login to a running configuration server of an existing cluster, and copy all of that configuration to the configuration of the new cluster; and make slight changes to reflect some specific requirements of the new cluster. After the configuration server of the new cluster is populated with a proper configuration, only then can the servers that process traffic or tasks (e.g. load balancing servers and traffic processing servers) be started. In this process, many human interventions are required, for example when deploying a configuration server, accessing another configuration server, copying an existing configuration, making specific configuration changes, all of which are inefficient and also error prone.
To help explain an example of a typical configuration, Figure 1 shows a configuration of a system in which an operator has four clusters 1031 to 1034. In the example each cluster 103 is shown as comprising a plurality of nodes of a first type 107 and a plurality of nodes of a second type 109, and a configuration server 105 for managing the configuration of each node within the cluster. It is noted that a cluster may contain further nodes, including further nodes of other types, but for ease of reference only first and second node types 107, 109 are shown. The dotted lines 111 represent manual configuration of the configuration servers 105, for example by a system administrator 101. The solid lines between the configuration servers 105 and their respective nodes 107, 109 represent, for example, private communication via a cluster-internal network, which is not directly accessible from outside.
In an example, a first cluster 1031 and second cluster 1032 may be provided for a particular service, e.g. Web Optimization, while a third cluster 1033 and a fourth cluster 1034 may be provided for another service, e.g. Video Optimization. The configuration of each node 107, 109 in each cluster 103 is distinct, for example due to each configuration containing a local configuration, such as an IP address to start the respective Web Optimization or Video Optimization services. The configuration between each node 107, 109 is difficult to synchronize. In a scenario where a common configuration for all clusters 103 needs to be updated, the system administrator 101 has to log into the configuration cluster 105 of each cluster 103 one by one to repeatedly make the same change. This is not only a tedious action but also error prone.
As can be seen from the above, existing configuration solutions expose the technical complexity to operators. As a business service is typically provided by multiple nodes of different types in a cluster system, an operator usually needs to set the different parameters in each type of node while following a strict procedure.
As another example, seven different nodes need to be updated in order to add a new service to the third cluster 1033 in Figure 1. This is unnecessarily complex and also error prone.
As mentioned earlier, each node 107, 109 has a copy of the common configuration. Thus, for a cluster with n nodes, the administrator has to maintain n copies of the common configuration. For a multiple-cluster system with m clusters, the administrator has to maintain m*n copies of the common configuration. This existence of multiple copies of the common configuration has several drawbacks. For example, it is difficult to make changes on common configurations. It is also difficult to ensure the consistency of the common configurations in different nodes. Memory space is also wasted, due to the need to store a copy of the common configuration of each node.
There are also drawbacks when adding a new node to such a system, since this means a new copy of the configuration and some customization must be prepared. Adding a new cluster means that configuration and customization for all nodes in the new cluster must be prepared. The data preparation involves many manual steps, which again is prone to error.
The embodiments described below propose new concepts for organizing and managing system configuration.
According to a first aspect of the invention, the configurations of clusters in a multiple cluster system are managed by one configuration server. The configurations of these clusters are managed in a hierarchical way, where the lower level clusters can inherit configurations from higher level clusters, further details of which will be described later in the application.
According to a second aspect of the present invention, a new representation of a node specific configuration is provided, whereby a node specific configuration
is elevated to be a common configuration of a cluster. This representation approach supports a node level configuration localization (or translation) from the common configuration, further details of which will also be described later in the application.
Figure 2 shows a method in a configuration server for configuring a communication system, according to a first embodiment of the present invention. The method comprises configuring a plurality of nodes in a first cluster (step 201) . The method in the configuration server further comprises the step of configuring a plurality of nodes in at least a second cluster (step203) , such that the configuration server acts as a common configuration server for at least first and second clusters.
By providing a common configuration server which is able to configure nodes in at least first and second separate clusters, this has the benefit of removing the need to have a configuration server in each cluster. Also, this allows the system to be scaled more easily.
In one example, the method comprises configuring a plurality of clusters, comprising at least the first cluster and the second cluster, in a hierarchical structure.
An advantage of the hierarchical structure of clusters is that only one configuration server is required to support multiple clusters, which provides efficiency on resource usage. Such a hierarchical structure also makes it easier to make a common configuration change, for example whereby a one time change can take effect on all lineal-descendant clusters.
The hierarchical structure of clusters also provides flexibility to introduce a new variant of cluster. For example, the new variant can be placed in a proper hierarchy, with the specific configuration parameters of the cluster then being
set. All the other configurations may use inherited values.
The hierarchical structure also helps ensure configuration consistency by its nature, as the configurations that are intended to be common only have one copy.
Figure 3 illustrates an example of a hierarchical structure in a system in which an operator has four real clusters 1031 to 1034, (which for example have nodes taking traffic) . First and second clusters 1031 and 1032 are for a service, e.g. Web Optimization, while third and fourth clusters 1033 and 1034 are for another service, e.g. Video Optimization.
Rather than having separate configuration servers 1051 to 1054 for each of the respective clusters 1031 to 1034 as with Figure 1, the embodiment of Figure 3 shows a common configuration server 113. The common configuration server 113 may be controlled or administered, for example, by a system administrator 101.
The four real clusters 1031 to 1034 are grouped in this example into two logical clusters: Cluster 1 for service 1 (relating to real clusters 1031 and 1032, which will also be referred to herein as Cluster 1.1 and Cluster 1.2 respectively) and Cluster 2 for service 2 (relating to real clusters 1033 and 1034, which will also be referred to herein as Cluster 2.1 and Cluster 2.2 respectively) . The clusters may be grouped logically, for example, based on configuration parameters they have in common in view of their common service. Other methods may also be used to logically group clusters. The two logical clusters, i.e. Cluster 1 and Cluster 2, are further grouped to one logical cluster, the root cluster, Cluster 0. It is noted that further layers and further clusters may be provided in a hierarchical structure, but the example of Figure 3 has been simplified for ease of reference.
Figure 4 shows another representation of the hierarchical structure of the
clusters illustrated in the example of Figure 3.
Cluster 0, at root level, comprises a root configuration, Configuration 0;
Cluster 1.1, at a second hierarchical level below root level, comprises a second level configuration, Configuration 1.1;
Cluster 1.2, at a second hierarchical level below root level, comprises a second level configuration, Configuration 1.2;
Cluster 2.1, at a second hierarchical level below root level, comprises a second level configuration, Configuration 2.1;
Cluster 2.2, at a second hierarchical level below root level, comprises a second level configuration, Configuration 2.2;
Thus, in this example:
·Cluster 0 is the parent of Cluster 1 and Cluster 2.
·Cluster 1 is the parent of Ccluster 1.1 and Cluster 1.2.
·Cluster 2 is the parent of Cluster 2.1 and Cluster 2.2.
For the configurations relating to each cluster, Figure 5 provides an example of how the configuration parameters may interrelate.
Configuration parameters that are intended to be common for all clusters are configured on the root cluster, Cluster 0 (this being the highest level in the hierarchical structure) . In the example above, configuration parameters p1, p2 and p3 are shown as being common to all clusters (and shown as comprising values “a” , “b” and “c” , respectvely) .
Configuration parameters that are intended to be common between Cluster 1.1 and Cluster 1.2 are configured on Cluster 1 (this being at a first level in the hierarchical structure) . In the example of Figure 5, configuration parameters p4 and p5 are shown as being common to clusters at this level, i.e. common to Cluster 1.1 and Cluster 1.2 (and shown as comprising values “d” and “e” , respectively) .
Similarly, configuration parameters that are common between Cluster 2.1 and Cluster 2.2 are configured on Cluster 2 (this being at a first level in the hierarchical structure) . In the example of Figure 5, configuration parameters p4 and p5 are shown as being common to clusters at this level, i.e. common to Cluster 2.1 and Cluster 2.2 (and shown as comprising values “h” and “i” , respectively) .
Configuration parameter (s) that are intended to be specific for a cluster are configured on the lowest level clusters, in the example of Figure 5 the lowest level being the second level below root level, and comprising Cluster 1.1, Cluster 1.2, Cluster 2.1 and Cluster 2.2. In the example, configuration parameter p6 is shown as being specific to each cluster (and shown as comprising values “f” , “g” , “j” and “k” , respectively, for each cluster) .
From the above it can be seen that reference “p” represents a particular configuration parameter, e.g. IP address to start an application, and that a value, e.g. “f” , is the value of that configuration parameter, for example the IP start address specific for Cluster 1.1. Likewise, for the configuration parameter “p4” ,
for example, both clusters 1.1 and 1.2 share the same value for this configuration parameter, i.e. “d” .
Examples of configuration parameters that might be shared between clusters include, for exampe:
-timeout values for different communication protocols, e.g. TCP connection timeout, HTTP request timeout, etc. ;
-enabled features, e.g. header enrichment enabled, content compression
enabled, etc. ;
-IP addresses of external nodes, e.g. black-listed IP addresses, IP address of billing system, etc.
Figure 5 therefore shows the actual configuration of each cluster, Cluster 1.1, Cluster 1.2, Cluster 1.3 and Cluster 1.4 after combining its own specifc configuration parameter (s) with inherited configuration parameters.
For example, Cluster 1.1 comprises configuration parameter p6 (having value “f” which is specifc to Cluster 1.1) , configuration parameters p4 and p5 (having values “d” and “e” respectively) which have been inherited from parent Cluster 1 at a higher hierarchical level, and configuration parameters p1, p2 and p3 (having values “a” , “b” and “c” respectively) which have been inherited from grandparent Cluster 0, or the root cluster.
From the above it can be seen that a method according to an embodiment of the present invention comprises configuring a plurality of clusters in a hierarchical structure by grouping one or more clusters into logical groups of clusters.
Clusters in a logical group of clusters comprise at least one configuration parameter in common.
In one embodiment, the common configuration server is provided at a root level in the hierarchical structure, and wherein the root level comprises a configuration (e.g. Configuration 0) comprising one or more configuration parameters which are common to all clusters being configured by the common configuration server.
In one example, clusters in a logical group of clusters at a first level below the root level (e.g. Cluster 1; Cluster 2) share one or more configuration parameters common to that logical group of clusters, and inherit one or more configuration parameters from the root level.
In one example, the hierarchical structure comprises a plurality of levels, wherein clusters in a logical group of clusters at an intermediate hierarchical level share one or more configuration parameters common to that logical group of clusters, and inherit one or more configuration parameters from a parent hierarchical level.
The one or more configuration parameters common to a logical group of clusters at the intermediate hierarchical level may be passed down to one or more child clusters.
A cluster at a lowest hierarchical level (e.g. Clusters 1.1, 1.2, 2.1, 2.2) comprises at least one configuration parameter specific to that cluster, and inherits one or more configuration parameters from a parent cluster, and any grandparent clusters or any higher clusters in the hierarchy. It is noted that the lowest level cluster, e.g. Cluster 1.1, may have more than just one configuration parameter which is specific to that cluster.
According to one embodiment the hierarchical structure comprises a plurality of hierarchical levels, and wherein one or more clusters at a lower hierarchical level are adapted to inherit one or more configuration parameters from one or
more clusters at a higher hierarchical level. The actual configuration of a cluster may be a combination of its own configuration specifc to that cluster, and the actual configuration of its parent, and whereby its own configuration takes precedence.
Referring to Figure 6, a method performed in a common configuration server (such as the common configuration server 113 of Figure 3) that is adpated to configure a plurality of clusters in this way may further comprise the steps of receiving a request from a first node to receive configuration information for the first node, step 601. The method comprises determining from cluster identity information received with the request, which cluster the first node is associated with, step 603. Configuration information is retrieved for the identified cluster, step 605, and the configuration information then sent to the first node, 607.
Figure 7 shows an example of how a node (or node instance in a cloud environment) is able to obtain configuration information from a common configuration server 113 according to an embodiment of the present invention. During a set-up or initialization procedure, for example, a new node (Node A) may be bootstrapped with information comprising a cluster identity (being the identity of the cluster the node is to be associated with) and an address of a configuration service (for example an IP address and Port associated with the common configuration server described above) , as shown by step 1. Other information may also be bootstrapped to the node during a set-up or initialization procedure.
In order to obtain configuration information, in step 2 of Figure 7, the node (Node A) sends a request for configuration information to the common configuration server 113. The address of the common configuration server is determined, for example, from an address of the configuration service that has been obtained in a bootstrap procedure as described above. The request sent in step 2 may comprise, for example, the sending of a Hypertext Transfer
Protocol, HTTP, GET request for the configuration of Node A. The interface between the configuration service and the node may comprise, for example, a RESTful interface (i.e. a Representational State Transfer architecture that complies to certain architectural constraints) . The HTTP GET request comprises the cluster identity, i.e. the identity of the cluster that the new node is to be associated with.
In step 3 of Figure 7, the configuration service (e.g. common configuration server 113) returns the configuration information for this request according to the cluster identity. The configuration information may comprise, for example, configuration files represented in JavaScript Object Notation (JSON) files.
In step 4, according to one example the configuration information downloaded from the configuration service is then modified or localized to a node level configuration, and the configuration parameters can then be applied to a local node application. Further details about the translation at local level will be described later in the application.
The configuration of multiple clusters can be managed by the configuration service, i.e. a common configuration server 113. The clusters are organized in a hierarchical way. Each cluster can have its own configuration. A configuration of a cluster is inherited by its children clusters, and a child configuration can have its own configuration value. The actual configuration of a cluster is a combination of its own configuration and the actual configuration of its parent (and any higher clusters in the hierarchy) , whereby its own configuration takes precedence.
Thus, according to another embodiment of the invention as shown in Figure 8, there is provided a method in a node of a communication system, wherein the node is associated with a cluster comprising a plurality of nodes. The method comprises determining a configuration service address relating to a common
configuration server that is adapted to configure the cluster associated with the node and at least one other cluster, step 801. In step 803, a request for configuration information is sent to the common configuration server. In step 805, configuration information is received from the common configuration server. Next, it will be described in further detail how a node can perform localization or translation of a common configuration received form a configuration server or common configuration service, into a configuration local to that node. The localization may be used in conjunction with the embodiments described herein concerning hierarchical configuration using a common configuration server, or alone in a single cluster comprising a plurality of nodes.
A large system could have thousands of nodes. Each node may need to have an unique configuration on the configuration server. This is due to the fact that the configuration usually contains some node specific configuration values, e.g. an IP address on which an application is to be started. For example, in Figure 5 above, a node corresponding to cluster 1.1 may comprise a node specific configuration comprising configuration parameter p6 having value “f” (e.g. “f” being the starting IP address of an application at that node) .
Managing thousands of unique configurations involves complexity on changing a common configuration on all nodes, ensuring consistency of common configuration across all nodes, and adding configuration for new nodes in the cluster.
According to a second aspect of the present invention, there is provided a method to elevate a node specific configuration to become a common configuration. This has an advantage that a cluster then only needs to have one common configuration, whereby all nodes in the cluster can then use this common configuration.
Thus, according to this aspect of the invention, instead of setting a node specific value for a configuration parameter, the embodiment proposes to set an alias configuration parameter as the value. Each node can be adapted to perform a localization procedure to translate the alias configuration parameter to its real node specific value, i.e. the specific configuration parameter for the node.
For example, an alias configuration parameter can follow a pattern such as:
<Separator>Tag<Separator>Attribute<Separator>Object<Separator>
The “<Separator>Tag” may be used to inform the node that a localization or translation procedure is to be performed. In the translation procedure, the above text is translated to the value of Attribute of Object.
For example, instead of setting a value of a specific configuration parameter of the node to an IP address, for example “1.2.3.4” , the specific configuration parameter may be set instead to a value such as "#LOCALIZE#ipv4#eth1#". When the node receives this alias representation, the node performs a localization procedure to translate the alias to its first ipv4 address of interface 'eth1' (that is, “1.2.3.4” ) , before really applying the value to an application.
Thus, in one embodiment, upon receiving configuration information from a configuration server, such as a common configuration server, a method in a node further comprises performing a translation procedure on the received configuration information to derive a node level configuration.
Performing a translation procedure may comprise determining an alias configuration parameter in the configuration information received from the configuration server, and mapping the alias configuration parameter to a specific configuration parameter which is specific to the local node.
Mapping information may be stored at the node, for example wherein the mapping information provides a relationship between an alias configuration parameter received from a configuration server and a specific configuration parameter for the node.
Figure 9 is provided as another example illustrating how the translation (localization) procedure may be provided within an overall configuring procedure.
In a first step 1, it is assumed that Node A is starting up. In step 2 the Node A performs a local IP discovery. This may involve determining a starting IP address, which identifies the IP address of the common configuration service. In step 3 the Node A sends a request for configuration information to a configuration service (such as a common configuration server) , and in step 4 receives configuration information files. It step 5 the Node A translates an alias configuration parameter to a node specific value or values. In step 6, the Node A applies the configuration to an application.
In the embodiments above the system configuration may be provided as a service by a common configuration server. In one example the common configuration server is a node within a communication system, or part of another node, wherein the configuration service runs on such a node. In another example, the common configuration service is provided as a cloud based service or platform, for example a web based service. Any node that has permission to use this service can connect to this server and use the configuration service to get proper configuration.
Figure 10 shows an example of a configuration server 113, according to an embodiment of the present invention, for configuring a communication system. The configuration server 113 comprising a processing module 115 adapted to configure a plurality of nodes in a first cluster, and configure a plurality of nodes
in at least a second cluster, such that the configuration server 113 acts as a common configuration server for at least first and second clusters.
The processing module 115 may be adapted to perform any of the steps described herein for the common configuration server.
A node can use a protocol such as HTTP, for example, to download its configuration from such a configuration server.
In some embodiments described above, it can be seen that a node specific configuration is elevated to be a common configuration, such that a cluster can have only one copy of a configuration, which all nodes in the cluster can share. Furthermore, in some embodiments the clusters can be organized into logical groups. The configuration of a cluster may be inherited by its children clusters. Therefore a common configuration of a set of clusters can be put to a higher level cluster, and the operator just needs to maintain only one copy of the common configuration across clusters.
The embodiments described herein remove the necessity for each cluster to have its own configuration server.
In the examples described above the configuration service and node are loosely coupled, which has an advantage in that both ends can expand or evolve with minimal impact to the other.
A node may be bootstrapped with information including cluster identity, configuration service address, and some other information. The configuration service address can be used by a node to contact the configuration service. The cluster identity enables the node to retrieve the configuration files that are available to that cluster.
The embodiments described above help reduce the complexity of configuration for large system.
The embodiments of the present invention enable the configuration management of a large system to be carried out much simpler and more efficiently. The embodiments also provide a better support to the high elasticity requirement in cloud computing environment. By high elasticity it is meant that it should be easy to adapt the number of servers in a cluster depending on the traffic profile. For example, it is advantageous not to require any extra configuration to add servers to a cluster at a busy period of traffic, or to remove servers from a cluster during low traffic.
The embodiments above have the advantage of providing simple and efficient configuration management. For example, an operator just needs to perform service oriented configuration change for a cluster. The complexity of the mapping between service oriented configuration and configurations of various types of nodes are transparent to the operator.
The hierarchical configuration architecture keeps minimized copies of a common configuration across clusters. Thus, it is comparatively easier to make a common configuration change to all clusters. The consistency of configurations across clusters is also improved due to there being only one copy of the common configuration in existence.
Since node specific configurations can be elevated to be a common configuration, all the nodes in a cluster can share the same copy of the common configuration, which has an advantage in that an operator does not need to do any configuration change when the cluster is scaling-out, i.e. becoming larger.
Also, when new clusters are added, an operator just needs to do minimized
configuration changes since most of the configuration can be inherited from higher level clusters in the hierarchical architecture.
The embodiments of the invention may be platform independent, and may be used for a bare-metal machine, a virtualization environment and a Cloud environment. As will be understood by a person skilled in the art, a “bare-metal” machine is a traditional server, on which the operating system has direct access to the hardware (e.g. a Linux server, or Windows on a laptop, etc. ) . In a virtualization environment and a cloud environment the operating system of a virtual server runs on top of a software, which is called hypervisor. The operating system of the virtual server has no direct access to the hardware. All calls are handled via the hypervisor.
According to one embodiment there is provided a method for configuring a communication system comprising one or more clusters, each cluster comprising a plurality of associated communication nodes of one or more type, the method comprising controlling the configuration of the plurality of clusters using a common, or single, configuration server.
The plurality of clusters may be controlled in a hierarchical structure. Any configuration parameters that are common to a group of first level clusters of a first level in the hierarchy are inherited from a common cluster at second level, the second level being higher than the first level, and any parameters common to clusters at the second level are inherited from a common cluster at a third level, the third level being higher than the second level, and so on until a root level is reached where the configuration parameters of the root level are common to all clusters in the system.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended
claims. The word “comprising” does not exclude the presence of elements or steps other than those listed in a claim, “a” or “an” does not exclude a plurality, and a single processor or other unit may fulfil the functions of several units recited in the claims. Any reference signs in the claims shall not be construed so as to limit their scope.
Claims (24)
- A method in a configuration server for configuring a communication system, the method comprising:configuring a plurality of nodes in a first cluster; andconfiguring a plurality of nodes in at least a second cluster, such that the configuration server acts as a common configuration server for at least first and second clusters.
- A method as claimed in claim 1, comprising the step of configuring a plurality of clusters, comprising at least the first cluster and the second cluster, using a hierarchical structure.
- A method as claimed in claim 2, wherein configuring a plurality of clusters using a hierarchical structure comprises grouping one or more clusters into logical groups of clusters.
- A method as claimed in claim 3, wherein clusters in a logical group of clusters comprise at least one configuration parameter in common.
- A method as claimed in any one of claims 2 to 4, wherein the common configuration server is provided at a root level in the hierarchical structure, and wherein the root level comprises a configuration comprising one or more configuration parameters which are common to all clusters being configured by the common configuration server.
- A method as claimed in claim 5, wherein clusters in a logical group of clusters at a first level below the root level share one or more configuration parameters common to that logical group of clusters, and inherit one or more configuration parameters from the root level.
- A method as claimed in claim 6, wherein the hierarchical structure comprises a plurality of levels, and wherein clusters in a logical group of clusters at an intermediate hierarchical level share one or more configuration parameters common to that logical group of clusters, and inherit one or more configuration parameters from a parent hierarchical level.
- A method as claimed in claim 7, wherein the one or more configuration parameters common to a logical group of clusters at the intermediate hierarchical level are passed down to one or more child clusters.
- A method as claimed in claim 6 or 7, wherein a cluster at a lowest hierarchical level comprises at least one configuration parameter specific to that cluster, and inherits one or more configuration parameters from a parent cluster, and any grandparent clusters and any clusters higher in the hierarchy.
- A method as claimed in claim 6 or 7, comprising the step of providing an alias configuration parameter in place of the at least one configuration parameter specific to a cluster, whereby the alias configuration parameter is for use by a node having the at least one specific configuration parameter, for translating the alias configuration parameter into a local configuration parameter specific to the respective node.
- A method as claimed in any one of claims 2 to 4, wherein the hierarchical structure comprises a plurality of hierarchical levels, and wherein one or more clusters at a lower hierarchical level are adapted to inherit one or more configuration parameters from one or more clusters at a higher hierarchical level.
- A method as claimed in any one of claims 2 to 11, wherein the actual configuration of a cluster is a combination of its own configuration specifc to that cluster, and the actual configuration of its parent cluster and any higher level clusters in the hierarchy, and whereby its own configuration takes precedence.
- A method as claimed in any one of claims 1 to 12, further comprising:receiving a request from a first node to receive configuration information for the first node;determining from cluster identity information received with the request, which cluster the first node is associated with;retrieving configuration information for the identified cluster; andsending the configuration information to the first node.
- A configuration server (113) for configuring a communication system, the configuration server comprising a processing module (115) adapted to configure a plurality of nodes in a first cluster, and configure a plurality of nodes in at least a second cluster, such that the configuration server (113) acts as a common configuration server for at least first and second clusters.
- A configuration server (113) as claimed in claim 14, wherein the processing module (115) is adapted to perform the steps according to any one of claims 2 to 13.
- A method in a node of a communication system, wherein the node is associated with a cluster comprising a plurality of nodes, the method comprising:determining a configuration service address relating to a common configuration server that is adapted to configure the cluster associated with the node and at least one other cluster;sending a request for configuration information to the common configuration server; andreceiving configuration information from the common configuration server.
- A method as claimed in claim 16, wherein the sending step comprises sending cluster identity information, the cluster identity information identifying the cluster which the node is associated with.
- A method as claimed in claim 16 or 17 wherein, upon receiving configuration information from the common configuration server, the method further comprises performing a translation procedure on the received configuration information to derive a node level configuration.
- A method as claimed in claim 18, wherein performing a translation procedure comprises:determining an alias configuration parameter in the configuration information received from the configuration server; andmapping the alias configuration parameter to a specific configuration parameter which is specific to the node.
- A method as claimed in claim 19, further comprising storing mapping information at the node, wherein the mapping information provides a relationship between an alias configuration parameter received from a configuration server and a specific configuration parameter for the node.
- A method as claimed in any one of claims 17 to 20, comprising receiving and storing the cluster identity and/or configuration service address at the node during an initial set-up procedure for the node.
- A method as claimed in any one of the preceding claims, wherein a cluster comprises a plurality of nodes of one or more types.
- A computer program, comprising instructions which, when executed on at least one processor, causes the at least one processor to carry out the method according to any one of claims 1 to 13, or 16 to 22.
- A carrier comprising the computer program of claim 23, wherein the carrier is one of an electronic signal, optical signal, radio signal or computer readable storage medium.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP14909299.1A EP3241306A4 (en) | 2014-12-29 | 2014-12-29 | Method and apparatus for configuring a communication system |
US14/435,875 US9847903B2 (en) | 2014-12-29 | 2014-12-29 | Method and apparatus for configuring a communication system |
CN201480084451.6A CN107210924B (en) | 2014-12-29 | 2014-12-29 | Method and apparatus for configuring a communication system |
PCT/CN2014/095280 WO2016106486A1 (en) | 2014-12-29 | 2014-12-29 | Method and apparatus for configuring a communication system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2014/095280 WO2016106486A1 (en) | 2014-12-29 | 2014-12-29 | Method and apparatus for configuring a communication system |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016106486A1 true WO2016106486A1 (en) | 2016-07-07 |
Family
ID=56283776
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2014/095280 WO2016106486A1 (en) | 2014-12-29 | 2014-12-29 | Method and apparatus for configuring a communication system |
Country Status (4)
Country | Link |
---|---|
US (1) | US9847903B2 (en) |
EP (1) | EP3241306A4 (en) |
CN (1) | CN107210924B (en) |
WO (1) | WO2016106486A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109002332A (en) * | 2017-06-05 | 2018-12-14 | 阿里巴巴集团控股有限公司 | A kind of process initiation, configuration method and device, system |
WO2021126027A1 (en) * | 2019-12-18 | 2021-06-24 | Telefonaktiebolaget Lm Ericsson (Publ) | First node, second node, and methods performed thereby for handling configuration of resources |
US11212175B2 (en) * | 2016-06-22 | 2021-12-28 | EMC IP Holding Company, LLC | Configuration management for cloud storage system and method |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108156009B (en) * | 2016-12-06 | 2021-03-26 | 北京金山云网络技术有限公司 | Service calling method and device |
CN109981371B (en) * | 2019-03-31 | 2021-05-18 | 联想(北京)有限公司 | Processing method and electronic equipment |
CN110413371B (en) * | 2019-06-18 | 2023-08-15 | 平安科技(深圳)有限公司 | Port adding method, device, equipment and storage medium for multi-activity load balancing application |
CN110635953A (en) * | 2019-10-17 | 2019-12-31 | 厦门网宿有限公司 | Configuration information management method and device |
CN111064626B (en) * | 2020-01-02 | 2023-05-26 | 广州虎牙科技有限公司 | Configuration updating method, device, server and readable storage medium |
CN112383424B (en) * | 2020-11-11 | 2022-10-25 | 杭州和利时自动化有限公司 | Operator station management and control method, device, equipment and readable storage medium |
CN114489720A (en) * | 2021-12-31 | 2022-05-13 | 医渡云(北京)技术有限公司 | Service deployment method, device, storage medium and equipment based on cluster |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101512507A (en) * | 2006-08-30 | 2009-08-19 | 微软公司 | Dynamically configuring, allocating and deploying computing systems |
CN102970376A (en) * | 2012-12-05 | 2013-03-13 | 曙光信息产业(北京)有限公司 | Cluster configuration method and device |
CN103731825A (en) * | 2013-12-20 | 2014-04-16 | 北京理工大学 | Bridge-type-based wireless sensing network key management scheme |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7617292B2 (en) | 2001-06-05 | 2009-11-10 | Silicon Graphics International | Multi-class heterogeneous clients in a clustered filesystem |
US20030074436A1 (en) | 2001-10-04 | 2003-04-17 | Adc Broadband Access Systems Inc. | Management information base object model |
US7739541B1 (en) * | 2003-07-25 | 2010-06-15 | Symantec Operating Corporation | System and method for resolving cluster partitions in out-of-band storage virtualization environments |
US7617501B2 (en) * | 2004-07-09 | 2009-11-10 | Quest Software, Inc. | Apparatus, system, and method for managing policies on a computer having a foreign operating system |
US7937455B2 (en) * | 2004-07-28 | 2011-05-03 | Oracle International Corporation | Methods and systems for modifying nodes in a cluster environment |
US8582470B2 (en) * | 2007-03-12 | 2013-11-12 | Telefonaktiebolaget L M Ericsson (Publ) | Arrangement and method relating to network management |
US8874705B1 (en) * | 2008-03-07 | 2014-10-28 | Symantec Corporation | Method and apparatus for identifying an optimal configuration of a resource |
US8825789B2 (en) * | 2008-12-16 | 2014-09-02 | Netapp, Inc. | Method and apparatus to implement a hierarchical cache system with pNFS |
US9658895B2 (en) * | 2012-08-07 | 2017-05-23 | Advanced Micro Devices, Inc. | System and method for configuring boot-time parameters of nodes of a cloud computing system |
CN104219290B (en) * | 2014-08-19 | 2017-05-31 | 南京邮电大学 | A kind of multimode cloud application elasticity collocation method |
-
2014
- 2014-12-29 CN CN201480084451.6A patent/CN107210924B/en active Active
- 2014-12-29 US US14/435,875 patent/US9847903B2/en active Active
- 2014-12-29 WO PCT/CN2014/095280 patent/WO2016106486A1/en active Application Filing
- 2014-12-29 EP EP14909299.1A patent/EP3241306A4/en not_active Withdrawn
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101512507A (en) * | 2006-08-30 | 2009-08-19 | 微软公司 | Dynamically configuring, allocating and deploying computing systems |
CN102970376A (en) * | 2012-12-05 | 2013-03-13 | 曙光信息产业(北京)有限公司 | Cluster configuration method and device |
CN103731825A (en) * | 2013-12-20 | 2014-04-16 | 北京理工大学 | Bridge-type-based wireless sensing network key management scheme |
Non-Patent Citations (1)
Title |
---|
See also references of EP3241306A4 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11212175B2 (en) * | 2016-06-22 | 2021-12-28 | EMC IP Holding Company, LLC | Configuration management for cloud storage system and method |
CN109002332A (en) * | 2017-06-05 | 2018-12-14 | 阿里巴巴集团控股有限公司 | A kind of process initiation, configuration method and device, system |
CN109002332B (en) * | 2017-06-05 | 2022-06-14 | 阿里巴巴集团控股有限公司 | Process starting and configuring method, device and system |
WO2021126027A1 (en) * | 2019-12-18 | 2021-06-24 | Telefonaktiebolaget Lm Ericsson (Publ) | First node, second node, and methods performed thereby for handling configuration of resources |
Also Published As
Publication number | Publication date |
---|---|
CN107210924B (en) | 2020-07-28 |
EP3241306A1 (en) | 2017-11-08 |
CN107210924A (en) | 2017-09-26 |
US20160352559A1 (en) | 2016-12-01 |
US9847903B2 (en) | 2017-12-19 |
EP3241306A4 (en) | 2017-11-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9847903B2 (en) | Method and apparatus for configuring a communication system | |
CN110198231B (en) | Container network management method and system for multiple tenants and middleware | |
US10771344B2 (en) | Discovery of hyper-converged infrastructure devices | |
US20220078092A1 (en) | Provisioning a service | |
CN107579931B (en) | A kind of block chain, that is, Service Source adaptation method based on Kubernetes | |
US20190342229A1 (en) | Multi-cluster resource management | |
US10560345B2 (en) | Consistent placement between private and public cloud deployments of application services | |
CN105376303B (en) | Docker implementation system and communication method thereof | |
US10222986B2 (en) | Tenant-level sharding of disks with tenant-specific storage modules to enable policies per tenant in a distributed storage system | |
CN109194502B (en) | Management method of multi-tenant container cloud computing system | |
US20180295029A1 (en) | Managing groups of servers | |
CN106302771A (en) | Domain name configuration method of application created based on Docker container | |
US11169787B2 (en) | Software acceleration platform for supporting decomposed, on-demand network services | |
US10693728B2 (en) | Storage isolation domains for converged infrastructure information handling systems | |
US11438280B2 (en) | Handling IP network addresses in a virtualization system | |
CN104468150A (en) | Method for realizing fault migration through virtual host and virtual host service device | |
US20180139175A1 (en) | Accessing nodes deployed on an isolated network | |
CN112187958A (en) | Method and device for registering, discovering and forwarding microservice | |
US11489814B1 (en) | Customized domain name resolution for virtual private clouds | |
US20200019637A1 (en) | External data management in a remote network management platform | |
US12028314B2 (en) | Providing persistent external internet protocol address for extra-cluster services | |
US9923865B1 (en) | Network address management | |
CN112351114B (en) | Information processing method and device and storage medium | |
US11922200B2 (en) | Remote network control for network virtualization | |
US11212136B2 (en) | Infrastructure support in cloud environments |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 14435875 Country of ref document: US |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14909299 Country of ref document: EP Kind code of ref document: A1 |
|
REEP | Request for entry into the european phase |
Ref document number: 2014909299 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |