EP3120496B1 - Génération de configuration sur la base d'une estimation de disponibilité - Google Patents

Génération de configuration sur la base d'une estimation de disponibilité Download PDF

Info

Publication number
EP3120496B1
EP3120496B1 EP15714288.6A EP15714288A EP3120496B1 EP 3120496 B1 EP3120496 B1 EP 3120496B1 EP 15714288 A EP15714288 A EP 15714288A EP 3120496 B1 EP3120496 B1 EP 3120496B1
Authority
EP
European Patent Office
Prior art keywords
service
component
type
availability
sis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP15714288.6A
Other languages
German (de)
English (en)
Other versions
EP3120496A1 (fr
Inventor
Parsa POURALI
Maria Toeroe
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telefonaktiebolaget LM Ericsson AB
Original Assignee
Telefonaktiebolaget LM Ericsson AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget LM Ericsson AB filed Critical Telefonaktiebolaget LM Ericsson AB
Publication of EP3120496A1 publication Critical patent/EP3120496A1/fr
Application granted granted Critical
Publication of EP3120496B1 publication Critical patent/EP3120496B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0793—Remedial or corrective actions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0823—Configuration setting characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability
    • H04L41/0836—Configuration setting characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability to enhance reliability, e.g. reduce downtime
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/008—Reliability or availability analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751—Error or fault detection not based on redundancy
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/079—Root cause analysis, i.e. error or fault diagnosis

Definitions

  • Embodiments of the invention relate to the generation of a system configuration based on the estimation of availability.
  • SA Service Availability
  • SA is defined as the percentage of time the service is provided even in the presence of inevitable failures in the system.
  • SA is typically achieved by using redundant resources (e.g., components) in the system and managing these redundant resources through repair and recovery mechanisms.
  • the Service Availability Forum (SA Forum), a consortium of communications and computing companies has developed a set of standard application programming interfaces (APIs) to enable the development of commercial-off-the-shelf (COTS) components based highly available systems.
  • APIs application programming interfaces
  • the Availability Management Framework (AMF) has the role of ensuring the availability of services of a component-based clustered system.
  • the AMF achieves this task by managing the redundant resources and re-assigning the workload of a faulty component to healthy ones.
  • AMF uses a configuration (i.e., AMF configuration) that describes the logical organization of the application components and their services.
  • AMF configuration i.e., AMF configuration
  • the design of an AMF configuration is a complex and error prone task.
  • Document US 2009/172168 A1 may be construed to disclose a technique pertaining to server management data which describe observed operating condition of a pool of spare servers.
  • a dynamic allocation period is determined as a period during which the target system needs additional server resources to handle an expected demand.
  • a set of allocation candidates are nominated from the spare server pool, by eliminating therefrom spare servers which are likely to fail during the dynamic allocation period.
  • An appropriate allocation candidate is then selected for allocation to the target system, such that the selected candidate will satisfy a specified requirement during its allocation period.
  • Document US 2012/221900 A1 may be construed to disclose a system deployment determination system that can appropriately define the number of information processing apparatuses that satisfies availability defined in an SLA as the number of information processing apparatuses used in a target system to be configured.
  • the list generating means generates a list including information processing apparatuses of which failure rates are less than the failure rate defined, and searches an information processing apparatus used independently or an information processing apparatus group forming a cluster of which cost is the lowest.
  • the apparatus number determining means calculates a number of information processing apparatuses required to satisfy the amount of requested processing during normal operation on the basis of the amount of requested processing during normal operation and an amount of processing performed by the searched information processing apparatus used independently or the searched information processing apparatus group forming the cluster.
  • the number of information processing apparatuses thus calculated can be allocated from the searched information processing apparatus used independently or the searched information processing apparatus group forming the cluster, the number of information processing apparatuses thus calculated is adopted as the number of information processing apparatuses used in the target system to be configured.
  • a computer-implemented method for generating a configuration for a service provider system to provide a highly available (HA) service according to claim 1 is provided.
  • a computer-readable medium according to claim 6 is provided.
  • a system adapted to generate a configuration for a service provider system to provide an HA service according to claim 7 is provided.
  • Embodiments of the invention provide methods and systems for generating configurations that support a requested level of service availability.
  • an availability-estimate based method is provided. Using the method in the configuration generation process, entity prototypes that cannot provide the requested service availability are eliminated to reduce the number of configurations to be considered in the configuration generation.
  • the availability-estimate based method is a quantitative method that estimates service availability based on partial configuration information. This estimate can then be used at earlier stages of the configuration generation process to prioritize or eliminate configuration options.
  • the method eliminates the prototypes arrangements (also referred to as type stack(s)) that would lead to configurations that cannot satisfy the availability requirements. Subsequently, the configuration generation process proceeds only for the type stack(s) that has the potentiality of achieving the requested service availability. Thus, only those configurations that are expected to guarantee the requested availability characteristics are generated.
  • the method avoids the use of full-fledged availability analysis methods and tools that are resource and time consuming and may not be solvable for complex models.
  • An AMF implementation manages the availability of application services based on a configuration, which is the logical organization of the application's services and components.
  • a component is the smallest logical entity in the system, on which AMF performs error detection, isolation and repair.
  • a component represents a specific resource such as a process, which is capable of providing a set of functionalities of an application.
  • AMF assigns a component service instance (CSI) to the component which represents an associated workload.
  • CSI component service instance
  • Higher level services are combinations of these functionalities and they are represented in the AMF configuration as service instances (SIs). Accordingly, one or more components that collaborate to provide an SI are placed together to compose a service unit (SU).
  • SI is the workload assigned to an SU at the runtime.
  • An SI can be assigned to an SU in the active or standby state. Active means that the SU is the primary service provider for that SI, whereas standby means that the SU is acting as secondary (redundant) entity for that SI.
  • An SU may have no or many assignments. If it has no assignment it is a spare SU. Redundant SUs protecting the same set of SIs are grouped together to form a service group (SG) to guarantee the service availability. Each SG is constructed according to a redundancy model. There are five different redundancy models: 2N, N+M, NWay, NWay-Active, and No-Redundancy.
  • each SI has their own characteristics with respect to the number of active and standby assignments an SI may have and the distribution of these assignments among the service units within the service group. For example, in a 2N redundancy model, each SI has an active and a standby assignment and only one SU takes all the active assignments and another one the standbys. Whereas in the NWay-Active redundancy model, the SIs have many active, but no standby assignments assigned to different SUs. A set of SGs form the AMF Application. The SUs are deployed on nodes that together form a cluster.
  • a type reflects and encapsulates the common characteristics that its instances share due to the common software implementation used to instantiate them.
  • the AMF types are primarily derived from the prototypes that one or more software vendors provide to describe the features and limitations of their implementation.
  • the AMF types respect the constraints imposed by the prototype from which they are derived.
  • the software vendor describes the software in terms of prototypes in an Entity Types File (ETF) . This description may include such characteristics as the component capability with respect to each functionality, the minimum and maximum number of component instances a SU may contain, and the different dependencies.
  • ETF Entity Types File
  • Component Service Type (CSType): The CSType specifies the attributes that characterize a particular workload that can be assigned to a component.
  • SvcType specifies how CSTypes can be combined.
  • Component Type (CompType): A CompType describes the characteristics of a specific version of a software implementation that can be used to instantiate components. Most importantly, the CompType defines the CSTypes such components can provide along with the dependencies among them. The CompType also specifies some other characteristics, for example, if such a component can be restarted to recover its services.
  • An SUType limits the combination of CompTypes, and accordingly it lists the SvcTypes that can be provided using instances derived from it. Any limitation on the number of instances of a particular CompType within the SUType is defined as a range value.
  • the SUType may also specify that in case of a component failure the entire SU is to fail.
  • the SGType limits what SUTypes can be used to build a SG of this SG prototype. It also determines the redundancy model for its instances.
  • the vendor can recommend the attribute settings for the failure escalation policies; e.g., the maximum number of times the components of an SU can fail within the probation period before the whole SU is restarted, and the maximum number of times that an SU can be restarted before it is failed over.
  • AppType The AppType limits the SGTypes that can be used in its instances.
  • An ETF describing a software implementation contains at least the CompTypes and the CSTypes.
  • the other prototypes are used when the software vendor wishes to constraint the way the software is deployed (e.g., CompTypes A and B are to be collocated in the SUType). Otherwise all combinations and attribute settings are allowed.
  • FIG. 1 illustrates a configuration generation process 100 according to one embodiment.
  • the input for generating an AMF configuration includes configuration requirements 101, ETF 102 and cluster information 103.
  • the configuration requirements 101 describe the services to be provided the system, the ETF 102 describes the available software, and the cluster information 103 describes the deployment cluster.
  • the process 100 After validating the input, the process 100 generates SIs and CSIs (block 110) and proceeds to the following configuration generation steps: ETF prototypes selection (block 120); creation of AMF types from the selected ETF prototypes (block 130); creation of AMF entities for the AMF types (block 140) and setting the attributes of AMF entities and their allocation to cluster nodes (block 150).
  • the output of the process 100 is an AMF configuration 104.
  • the first step in the configuration generation process 100 is the ETF prototype selection (block 120). This step finds all the ETF prototypes that can satisfy the input configuration requirements. For instance, for a given administrative domain, one or more application prototypes may provide the requested services through one or more SG prototypes. In this case for each possible combination of prototypes that satisfies the requirements, a stack of prototypes is built that contains a set of component prototypes at the bottom. The component prototypes may be grouped by SU prototypes that in turn may belong to SG prototypes that finally may belong to some application prototypes. Based on how many suitable candidate prototypes are available at each level, multiple type stacks may be built. Some of the stacks may be incomplete; that is, contain only the prototypes for the lower layers.
  • the available collection of ETF prototypes can be represented as a disconnected non-ordered directed acyclic graph (DAG) 200 as shown in the example of Figure 2 .
  • the DAG 200 represents an ETF model.
  • the nodes of the DAG 200 represent ETF prototypes, whereas the edges represent the relations among these prototypes.
  • the Root 210 represents the requested service.
  • the DAG 200 is constructed from the ETF files (e.g., the ETF 102 of Figure 1 ), each file describing one or more component prototypes with their services and their possible organization into an application prototype.
  • any prototype may have the Root 210 as its parent; e.g., Orphan SGT3 220.
  • an AppType has only one parent, the Root 210, other prototypes may have more than one; e.g., SUT3 can be used to build instances of SGT2 or SGT3. Some prototype may be missing; e.g., SGT3 can be used for any AppType other than AppT1, which specifies that only SGT1 and SGT2 can be used in its instances.
  • Figure 3 illustrates an example of three of these different type stacks. These type stacks are passed onto the next step of the configuration generation process, the AMF type creation.
  • the second step of the configuration generation process 100 is AMF type creation (block 130).
  • the selected ETF prototype stacks are processed and the corresponding AMF types are derived. If there was no prototype selected for an AMF type, one is created.
  • the attributes of the AMF types are constrained by the attributes of the ETF prototypes from which they are derived. In this respect some attributes are to be set as provided in the prototype, while others are changeable. Accordingly, if the ETF prototype specifies a particular value, this value is used in the AMF type. Changeable attributes are set according to applicable configuration design patterns. If there is no applicable configuration design pattern for a changeable attribute, the default provided by the ETF prototype can be used. If no default is specified, the applicable default defined by the AMF specification can be used. It is understood that other attribute settings may also be valid and may be used.
  • the third step of the configuration generation process 100 is AMF entities creation (block 140). Once the AMF types have been created, their corresponding AMF entities are created and configured for the system. For this, first the number of instances for each AMF entity type is determined. When the AMF entities are created, for each AMF type the number of entities is not fixed, but can be selected from a range. This number takes into account the configuration requirements as well as all the constraints imposed by their AMF type.
  • the fourth step of the configuration generation process 100 is completing the configuration attributes (block 150).
  • This last step of the AMF configuration generation process sets the attributes that determine the distribution of the AMF entities (e.g., SUs) among the nodes of the cluster.
  • the step aims at providing an even distribution so that at runtime the load will be balanced in the AMF cluster.
  • multiple ETF prototypes may be available to provide the same service and can be selected in the first step (block 120); from the same ETF prototype, multiple AMF types can be derived in the second step (block 130); different numbers of AMF entities can be created for the same AMF type in the third step (block 140) and the fourth step (block 150); and the AMF entities can be distributed in many ways in the fourth step (block 150).
  • the difficulty of the configuration generation task lies in that in each step the choices are made based on some criteria, but not all the criteria. Accordingly, some options selected by some criteria may lead to no result as they are pruned later by other criteria, other options that are pruned early by some criteria may improve significantly by later choices and become the solution if they were not pruned.
  • the availability-estimate based method estimates the service availability that can be achieved by a configuration. This estimate is used to eliminate prototypes that cannot meet the availability requirements for the system, and to generate only the configurations that satisfy the requested level of availability. For those configurations that satisfy the requested level of availability, the method determines the number of redundant entities in the configuration. Thus, the method is applied at two steps of the configuration generation process 100. In the first step (block 120) at the type selection, the type stacks that cannot guarantee the requested service availability are eliminated based on the availability estimates. In the third step (block 140) of AMF entities creation, AMF entities are generated based on these estimates such that the generated entities satisfy the requested service availability.
  • MTTR Mean Time To Repair
  • MTTF Mean Time To Failure
  • the MTTF is the average time between two consecutive failures, whereas the MTTR refers to the average time to repair the component, so that it can provide the service again.
  • the MTTF of a component is considered as a constant failure rate provided by the vendor for the CompType from which the component is derived. Thus, only the MTTR is to be estimated.
  • repair means the repair of the service rather than the component.
  • the recommended recovery of the CompTypes and other configuration attributes determine how the service is recovered after a component failure.
  • the recommended recovery actions of the components are analyzed in the context of the configuration to determine what actual recovery actions will be taken by AMF at runtime in case of a failure. Based on this actual recovery action, the time needed to complete the associated procedures is estimated. With the MTTR and the failure rates of the components involved, the service availability can be estimated.
  • the assumptions (A1-A5) include the following: (A1) it is assumed that in each SU there is one component for each CompType that is needed to provide the service. (A2) It is assumed that there is an identical (i.e., derived from the same SUType) redundant SU hosted on a redundant node that can take over the service assignment when necessary. (A3) It is also assumed that there is more than one assignment for an SI protected by the NWay-Active redundancy model. Thus, no outage time for such a service is considered.
  • One important configuration attribute that determines the actual recovery action taken by AMF in response to a component failure is the recovery action configured for the faulty component. This is configured based on the recommendation provided by the vendor in the corresponding CompType. For a CompType, there may be any of the following recommended recovery actions defined in the AMF specifications: No Recommendation, Component Restart, Component Failover, Container Restart, Node Switchover, Node Failover, Node Failfast, Application Restart, or Cluster Reset.
  • AMF detects a component failure, it checks the applicable recommended recovery action of the component, and it also checks all the other configuration attributes that can alter this recommendation. Then AMF determines the actual recovery action that it will execute.
  • the component restart recommendation is altered if the restart option for the component is disabled for the component itself or any of its parenting entity (e.g., SG). If AMF cannot perform the component restart and the actual recovery action will be escalated to the parent SU level (i.e., SU restart or SU failover).
  • the recovery recommendations are analyzed for all the CompTypes in the context of their type stack and the applicable actual recovery actions are determined.
  • the time it takes to complete this recovery action can be estimated.
  • the availability-estimate based method determines the procedures it involves and their timing to estimate the needed time for completing the recovery action.
  • AMF accomplishes all its tasks through managing the components of the system under its control. Accordingly, all recovery actions except for node failfast can be decomposed into a combination of recovery actions applicable to components, which are: component restart, component failover, and the component switchover. These component level recovery actions can be further decomposed into a sequence of component life-cycle and API callback operations. Table I presents the decomposition of the different component level recovery actions. TABLE I.
  • An actual recovery action is first decomposed into the operations in Table I for estimating the time for its completion. All the operations in Table I are guarded by timers. These timers determine the maximum time that AMF waits before it considers the operation unsuccessful. Therefore, the timeout values of these timers are used to estimate the recovery time.
  • AOT Average outage time of component instantiation AOTIWOD Average outage time for instantiation attempts without delay
  • AOTIWD Average outage time for instantiation attempts with delay
  • ClT Clean-up timeout for a component saAmfCompCleanupTimeout, saAmfCtClcCliTimeout, defaultClcCliTimeOut
  • CSS Timeout of the CSI set callback to the component saAmfCompCSISetTimeout, saAmfCtCallbackTimeout, defaultCallbackTimeOut
  • Delay between instantiation attempts saAmfCompDelayBetweenInstantiateAttempts, saAmfDelayBetweenInstantiateAttempts
  • FOT Failover time of the faulty component IT Instantiation timeout for a component saAmfCompInstantiateTimeout, saAmfCtClcCliTimeout, defaultC
  • a first actual recovery action is the component restart.
  • Figure 4 illustrates a procedure of a component restart recovery action according to one embodiment. As illustrated, the component restart recovery action starts with cleaning up the faulty component. If the cleanup is successful, then AMF tries to re-instantiate the component. Once the component is instantiated successfully, AMF sets the component's assignments as requested to recover the services to complete the component restart recovery action. If unsuccessful, AMF will attempt to instantiate the component a number of times without any delay between the attempts, followed by attempts with delays between them. In the availability estimation, it is assumed that the cleanup and instantiation commands have their own success probability. The number of instantiation attempts without delay (NIWOD) and the number of instantiation attempts with delay (NIWD) are expected input to the configuration generation process.
  • NIWOD number of instantiation attempts without delay
  • NIWD number of instantiation attempts with delay
  • a component restart recovery action fails if AMF fails to clean up the component or if the instantiation fails after all the allowed attempts. In these cases, it is assumed that AMF reboots the node hosting the component and all the assignments of the node are failed over to the standbys nodes (i.e., the nodes on which the components with the standby assignments reside).
  • Figure 5 illustrates an example of the possible (i.e., alternative) cases of a component restart recovery action according to one embodiment.
  • a single attempt is allowed for instantiation without delay as well as with delay.
  • the recovery time for each case is determined using a tree diagram, with each case represented by a leaf of the tree.
  • the tree for the example has five leaves.
  • the leaves representing Case 2 and Case 4 are the cases where the component restart recovery action is successful.
  • the other leaves, Case 1, Case 3 and Case 5 correspond to the cases where the component restart recovery action fails and the failure is escalated to the node level. As a result, all the assignments of the node are to be failed over.
  • the availability estimate calculation is performed by a system executing an embodiment of the availability-estimate based method. Embodiments of the system will be described in connection with Figure 16 and Figure 17 .
  • the system calculates the value of its probability based on the probabilities of the individual operations involved and the time that it takes based on their associated timeouts.
  • the probability of Case 2 is equal to PCS ⁇ PIS , which represents the probability of a successful cleanup action multiplied by a successful instantiation attempt.
  • the time needed to complete each case is the time it takes for all of the (unsuccessful) consecutive actions prior to that case, and the time needed for the current action (e.g., CSS, NCF, IT).
  • the average recovery time i.e., outage time
  • the component restart recovery action is equal to the summation of all of the probable cases.
  • NCF NST + Max 1 ⁇ j ⁇ N CSSj
  • Eq. (2) below can be used to calculate the average outage time due to the component instantiation attempts without delay, as part of the component restart action estimation.
  • PCS i ⁇ PINS (i-l) ⁇ PIS represents the probability of the i th successful case while (PCS i ⁇ PINS (i - l) ⁇ PCNS) represents the i th unsuccessful case.
  • these probabilities are multiplied by the respective time values, which are expressed by the terms ((i x (ClT + IT)) + CSS) and ((i ⁇ ClT) + ((i-l) x IT) + NCF.
  • Eq. (3) calculates the average outage time due to the component instantiation attempts with delay.
  • PINS NIWOD calculates the probability that all instantiation attempts without delay were unsuccessful.
  • the average outage time can be calculated using Eq. (4) according to one embodiment.
  • OIF PC S n ⁇ PIN S NIWOD ⁇ PINS D NIWD ⁇ n ⁇ ClT + IT + NIWD ⁇ d + NCF
  • the average outage time caused by the component restart recovery action is the sum of the partial time estimates defined by Eq. (2), Eq. (3) and Eq. (4).
  • the estimated total recovery time for the component restart is given in Eq. (5) according to one embodiment.
  • MTTR AOTIWOD + AOTIWD + OIF
  • a second actual recovery action is the component failover.
  • a component failover recovery action includes two operations: cleaning up the faulty component, and assigning its assignments to its healthy peers.
  • the other components of the SU of the faulty component may be subject to a switchover or a failover of their assignments.
  • Figure 6 illustrates an activity diagram for the component failover recovery action according to one embodiment. The activity diagram illustrates the case when other components are switched over; i.e., AMF first changes their active assignments to quiesced, then AMF moves the active assignments to the corresponding peer components.
  • Figure 7 illustrates an example of a component failover recovery action according to one embodiment.
  • Case 2 represents the successful failover recovery action
  • Case 1 corresponds to the failure of the component failover resulting in to node failfast, when all assignments of the node are failed over while the node is abruptly rebooted.
  • Figure 7 only depicts the failover action as a part of the whole failover recovery action. It does not reflect the switchover action that may be applied to the collocated components.
  • a third actual recovery action is the SU restart.
  • the SU restart actual recovery action is the first level of escalation of the component restart recovery recommendation.
  • the SU restart is the restart of all its components.
  • the actions need to be synchronized across the SU; that is, all the components are cleaned up first simultaneously, and if successful, then simultaneously AMF re-instantiates all of them and sets their state assignments as the components had them before the failure.
  • Figure 8 illustrates an activity diagram for an SU restart actual recovery action according to one embodiment.
  • Eq. (10) does not include the first cleanup which ensures the removal of the assignments, and therefore needs to be synchronized. This is taken into account by Eq. (9).
  • Eq. (11) calculates the average outage time of the instantiation attempts with delay for each CompType.
  • AOT denote the average outage time of a component instantiation for a given CompType.
  • the AOT can be calculated by Eq. (13), which is the sum of the portions calculated by Eq. (10), Eq. (11) and Eq. (12).
  • AOT AOTIWOD + AOTIWD + OIF
  • a fourth actual recovery action is the SU failover.
  • the SU failover actual recovery action includes the following operations: first, AMF cleans up all the components of the SU, then, if the cleanup was successful, AMF reassigns their CSI assignments to appropriate healthy peer components.
  • Figure 9 illustrates an activity diagram of an SU failover actual recovery action according to one embodiment.
  • a first step is to calculate the cleanup time of each CompType used in the SU using Eq. (9).
  • the reassignments of the CSIs are guarded by the CSI set timeout (i.e., CSS).
  • a next step is to add up the maximum of cleanup times and the maximum of the reassignment time, since each of these operations are executed concurrently as shown in Eq. (15).
  • MTTR Max 1 ⁇ j ⁇ N CleaupTim e j + Max 1 ⁇ j ⁇ N PC S j ⁇ CS S j where j iterates through the CompTypes in the SU (i.e., N ) .
  • the remaining recovery actions that can be recommended for a component include: node failfast, node failover, node switchover, application restart and cluster reset. These actions are to be taken into account for the recovery estimation as well.
  • the node failover recovery action is similar to the SU failover recovery action for which the estimate is given by Eq. (15). For the node all the CompTypes hosted on the node are considered. In case of node switchover, AMF fails over the faulty component but switches over the others hosted on the node. Thus, the recovery time can be estimated similar to the component failover given in Eq. (8).
  • the node failfast recovery action has been considered in Eq. (1).
  • the application restart recovery action is defined as terminating the whole application and starting it again, which is the cleanup of all of the components first and then instantiating them again.
  • This action is equivalent to the SU restart recovery action performed simultaneously on all SUs of the application and coordinated across the cluster; i.e., Eq. (15) can be used to estimate the recovery time with the modifications that j iterates through the CompTypes of all SUTypes of the application.
  • the cluster reset recommended recovery means that AMF simultaneously reboots all the nodes in the cluster and restarts the applications on them.
  • AMF starts to assign the assignments when either all required components have instantiated successfully or when the saAmfClusterStartupTimeout expires.
  • the description above presents the service recovery time (i.e., MTTR) for the different actual recovery actions AMF may take after a component failure. Furthermore, every CompType has a failure rate (i.e., MTTF) provided as an input. Hence, by using the recovery time estimate and the failure rate of a CompType, the system (which performs the availability-estimate based method) estimates the availability of the CSTypes that the CompType may provide. The system further combines the availability of the CSTypes to estimate the availability of the SvcType. With this estimate it can be determined whether the type stack cannot provide the service with the availability requested and therefore does not need to be considered for configuration generation.
  • MTTR failure rate
  • MTTFj is the failure rate of CompTypej that has been included in the type stack to provide some CSTypes of the SI's SvcType.
  • MTTRj is the recovery time estimate for CompType j based on the applicable actual recovery action in case of the failure.
  • NP is the number of CompTypes in the type stack that participate in providing the SvcType.
  • the service availability is estimated for each SvcType ( STSA ) for each type stack, and the resulting STSA is compared with the requested service availability specified in the configuration requirements. If a type stack cannot provide at least the requested service availability for any of the SvcTypes, it is eliminated from the list of type stacks.
  • Instantiation-Level dependency one of the complexities of estimating the service availability is the instantiation level dependency among the components of the same SU. According to this dependency, the instantiation of a component is a prerequisite for the instantiation of the other component. This type of dependency is applicable only when instantiating or terminating an SU. The following explains the instantiation-level dependency and the way it can be handled in the availability estimation.
  • the instantiation level dependency considers the following factors:
  • AMF instantiates all components with the same instantiation level in parallel. Moreover, AMF instantiates the components of a given instantiation level, only when all components with a lower instantiation level have been instantiated successfully.
  • the AMF terminates all components with the same instantiation level in parallel. Furthermore, AMF only terminates the components of a given instantiation level only when all components with a higher instantiation level have been terminated.
  • the instantiation level dependency is only applicable during the instantiation and termination of an SU. Therefore, it is applicable for the SU restart recovery action.
  • the system uses Eq. (18) instead of Eq. (14) to calculate the recovery time of an SU restart recovery action.
  • M the highest level of the dependency among the CompTypes of the SUType
  • i varies from one to the M, and j iterates through the number of CompTypes that have the same instantiation level in the SUType.
  • Max 1 ⁇ j ⁇ N ( CleaupTime ij ) in the equation above represent the maximum time that is needed to clean up the CompTypes of the i th instantiation level.
  • Proxy-Proxied dependency according to this dependency, if the proxy component fails, the AMF finds another proxy component to take over the proxying work. In the meanwhile, the proxied component(s) can continue providing its services. Only if the proxy component and its proxied components fail at the same time, the proxied component cannot provide its service. No additional calculations are considered for this type of dependency, as the failure of the proxy component does not indicate the failure of the proxied component(s).
  • CSI-CSI dependency due to the CSI-CSI dependency, the components will be assigned in a defined order. It was assumed that the CSI assignments were done in parallel, However, if there is a CSI-CSI dependency, the CSIs assignments are done sequentially. It means that first the sponsor CSI is assigned to the component, and then its dependent CSI(s). In such a case, the CSS of the CompType providing the dependent CSI is increased in a way that it includes the CSS of the CompType providing the sponsor CSI. The same logic applies for CSIs removal. It means that the sponsor CSI is removed from a component, only when its dependent CSI(s) is removed from the assigned component(s).
  • SI-SI dependency an SI may be configured to depend on other SIs, in the sense that an SU can only be assigned as active for the dependent SI if all its sponsor SIs are assigned.
  • the AMF defines the tolerance time as a configurable attribute of a dependency between SIs. For example, if the SI1 depends on the SI2, the tolerance time indicates for how long SI1 can tolerate SI2 being in the unassigned state. If this time passes before SI2 becomes assigned again, the AMF will remove the assignments of SI1 from the SU. Accordingly, the estimation calculations will be impacted in a way that, if the MTTR of the sponsor service is more than the tolerance time of the dependent service, the availability of the dependent service will be reduced by the availability of its sponsor. Thus, the availability estimation of the dependent service is reduced as calculated by Algorithm I.
  • Figure 10 illustrates an example of selecting a type stack based on estimated service availability, where the ETF prototypes selected to provide SvcTypeX are arranged into two type stacks. While TypeStack1 uses CompType1 and CompType2 together to deliver a service, TypeStack2 uses only CompType3 to provide the same service. The availability requested for the SI of SvcTypeX in the configuration requirements is 0.99999 (or 99.999%).
  • Table III provides the attributes for the CompTypes in the type stacks which are used as input for the estimation calculations. Table III.
  • the input to the system includes the results of the actual recovery analysis for each of the CompTypes.
  • NST 8 is given as input also
  • the NCF for TypeStack1 is the maximum between the CSS of CompType1 and CompType2.
  • CompType2 The actual recovery action of CompType2 is component failover.
  • the system estimates its recovery time as follows. First, the system uses Eq. (7) to estimate the time needed to switch over the CSIs of other components (i.e., the component of the prototype CompTypel) in the SU (prototype).
  • TypeStack2 has only one CompType, namely CompType3, whose actual recovery action is SU failover.
  • the system can calculate the service availability estimate for SvcType_X for each of the type stacks.
  • NP denotes the number of CompTypes
  • MTTFj and MTTRj refer to the failure rate and the recovery time estimate of each component.
  • TypeStack1 the service availability is calculated as 0.999969, while for TypeStack2 it is 0.999991. Hence, in this example only TypeStack2 is considered further in the configuration generation process.
  • the following description explains an embodiment of the availability-estimate based method applied to the third step (block 140) of the configuration generation process 100 ( Figure 1 ), which is the AMF entities creation.
  • the existing AMF configuration generation method expects as input the number of SUs, the number of SGs and their redundancy model. These parameters depend on the software available; that is, the ETF prototypes selected to provide the services, their composition and the required level of service availability. Rather than expecting these parameters as input, the method described herein generates a configuration for a type stack that is expected to satisfy the requested level of availability according to the availability estimates.
  • a system performing the availability-estimate based entities creation method automatically calculates first the number of components in each SU with respect to the level of service availability requested for the SI-template. Once the SU is known, the system can calculate the number of SUs for the SG ensuring that the number of SUs does not exceed the cluster size (i.e., the number of nodes). This constraint is added to prevent hosting more than one SU of an SG on the same node.
  • the redundancy model and the number of SUs determine the capacity of an SG; hence the system can calculate the number of SGs needed to protect the SIs of the SI-template.
  • AMF entities are created based on the AMF types created in the second step (block 130) of the configuration generation process 100 ( Figure 1 ). Therefore, unless otherwise noted, in the following description with respect to the availability-estimate based entities creation method, entity types such as component types (CTs), component service types (CSTs), service unit types (SU types) and service group types (SG types) refer to AMF types and are abbreviated herein in respective parentheses.
  • CTs component types
  • CSTs component service types
  • SU types service unit types
  • SG types service group types
  • the system calculates the number of components of a given CT in a SU with respect to the required service availability.
  • the calculations for estimating the availability due to components failures have been covered in the previous paragraphs.
  • an assumption (A1) was made that there is a single component of each required CT in each SU.
  • the assumption (A1) is removed; it is determined how many components are needed to provide an SI and how many of them can be placed together in an SU while still meeting the availability requirements for the SIs.
  • the availability-estimate based entities creation method described herein ensures the number of created entities in the configuration meets the availability requirements. Two assumptions are made in the calculations: (1) only one CT in the SU type can provide a particular CST; and (2) the different service types can be provided by different sets of CTs. This can be true because the separation of CSTs design pattern is applied prior to this step.
  • Each type stack includes a CT for each component service type (CST) required by a service type (SvcType) of an SI-template (SIT) of the configuration requirements.
  • a CT may be included to provide different CST, but for the same CST only a single CT is included.
  • the SvcType of a SIT determines the CST to be provided by the CTs of a SU type.
  • Table IV List of Acronyms for Availability Estimate-Based Entities Creation Method Abbreviation Definition SUMaxCapPerCT Maximum number of components of a given CT that can be put in the SU NoOfCompsPerCST Number of components of a CT that can provide a particular CST ActiveCapabilityPerCST Component capability to handle active assignments (per CST) StandbyCapabilityPerCST Component capability to handle standby assignments (per CST) NoOfSIs Number of SIs of the SI Template NoOfCSIsPerCST Number of CSIs of a CST in each SI of the SI template NoOfActiveAssignments Number of active assignments per each SI NoOfStandbyAssignments Number of standby assignments per each SI NoOfNodes Number of nodes in the cluster
  • the number of components of a given CT in an SU is calculated with respect to the requested availability for the SIs.
  • the availability estimation of a service (type) assuming that there is only one component of each CT.
  • AMF CT AMF CT
  • the system multiplies the deliverable availability of the components that are participating to provide the service. Due to the multiplication of the components' deliverable availability, the larger the number of components, the lower the service availability. Therefore, to calculate the number of components in an SU, the system cannot increase the number of components to the point that the multiplication of the components' deliverable availability becomes lower than the requested service availability. For this goal, the system starts with putting a minimum number of components of each CT in the SU, and then increases the number of components to the point that the availability requirement is not violated.
  • the minimum number of components of a CT to be put in the SU depends on the capability of the CT.
  • Algorithm II defines the steps to calculate the minimum number of components of each CT in the SU, taking into account that the SU is capable of providing a minimum of one SI.
  • Algorithm III defines the steps to calculate the maximum number of components of a CT, taking into account that SU is capable of providing at most all the SIs of the SI template (i.e. NoOfSIs). Algorithm III is used when the maximum number of components of a CT is not specified (i.e. unlimited) in the ETF.
  • the actual number of components (of a CT) that is targeted is between the minimum and maximum number of components.
  • the system uses Algorithm IV to calculate the actual number of components of the CTs in an SU, by taking into account the requested service availability.
  • the logic behind the algorithm above is to start with putting the minimum number of components of each CT in the SU.
  • This minimum number also represents the proportion of the number of components of the CTs in the SU. Therefore, the number of components of a CT is always a coefficient of its minimum number.
  • the system iteratively increases the number of components based on their proportion. For example, consider that the minimum and maximum number of components of the "CT1" are calculated as 4 and 16, respectively. In this case, the number of components of the "CT1" can be 4, 8, 12 or 16. In each iteration, the system increases the number of components based on their proportion, and then calculates the achievable service availability as Eq.
  • N the total number of components (i.e., summation of the number of components of all the CTs) that are participating to provide the service.
  • SISA the calculated service availability
  • the system iterates again by increasing the number of components. This iteration is performed multiple times unless: (a) the SISA becomes lower than the requested service availability, or (b) or the number of components of any of the CTs exceeds the maximum number of components of the CT that is determined by Algorithm III. Note that the discussed calculation is to be applied to all of the service types that are being provided by the SG type, separately.
  • the number of components of a CT in a SU is between the minimum and maximum number of components of the CT calculated by Algorithm II and Algorithm III, respectively.
  • the number of components of a CT in a SU is calculated as the minimum number of components of a CT multiplied by i , where i iterates through one to the number of SIs of the SI template (i.e., NoOfSIs ).
  • the service availability calculation (i.e., Eq. (19)) does not consider the impact of the recovery action of a service on another service.
  • the following example helps in clarifying this. Assume that the SU1 is required to provide two services (i.e., SI templates) which are Svc1 and Svc2. In such a case, the recovery action of Svc1 can affect the availability of Svc2, if the recovery action (of Svc1) is at the SU, Node, Application or Cluster levels. It means that if a component that is providing Svc 1 fails and it has the actual recovery action of the aforementioned levels (e.g.
  • N the total number of components that are participating to provide Svc2
  • M the number of components that are providing the Svc1 and have the actual recovery of the SU, node, application or cluster levels.
  • the number of SGs and SUs calculations that have been proposed are per CST.
  • the calculations are applied per CST of the service type that the SG type protects, separately. Then, the one that resulted in a greater number of SGs can be chosen.
  • the one that has the greater number of SUs can be selected. For example, if there is a service type that has three CSTs, the system will use the calculations below for the all the three CSTs, separately. Then, select the one that resulted in a greater number of SGs.
  • Algorithm V defines the steps to calculate the number of SUs and SGs.
  • the calculations of the number of SGs (of an SG type) and SUs mostly depend on the redundancy model of the SG type. In the following discussion, the calculations for each redundancy model will be explained separately. Again, it is noted that a constraint is imposed on the system that no more than one SU of an SG can reside on the same node. It means that the maximum number of SUs within an SG cannot exceed the number of nodes.
  • No-Redundancy redundancy model In this redundancy model, each SU can be assigned as active for at most one SI. Therefore, to handle all the SIs, the number of SUs is equal to the number of SIs. The system can use an additional spare SU to take the SI assignment in case of an SU failure.
  • the system divides the total number of SIs to the number of nodes minus one. This is because it reserves one node to later put a spare SU on it.
  • the system uses floor because the maximum number of SIs that can be assigned to an SU has to be an integer number. Once the maximum number of SIs that an active SU can handle is found, the system finds the maximum number of SIs that a standby SU is capable of handling, as Eq. (24).
  • MaxNoOfSIsPerOneStdSU floor NoOfCompsPerCST ⁇ StandbyCapabilityPerCST NoOfCSIsPerCST
  • the number of SIs that an SG can handle is the minimum of MaxNoOfSIsPerOneActSU and MaxNoOfSIsPerOneStdSU .
  • the active SU can handle 5 SIs and the standby SU can handle 6 SIs.
  • N+M Redundancy model In this redundancy model, there are a total number of N active SUs and M standby SUs to handle the SIs protected by the SG (i.e., N+M SUs).
  • the system calculates the total number of SUs as NumberOfActSUs + NumberOfStdSUs .
  • the system considers the maximum number of SUs in one SG, equal to the number of nodes. Thus, if the total number of SUs (i.e., NumberOfActSUs + NumberOfStdSUs ) exceeds the number of nodes, the system needs to distribute the SUs on a greater number of SGs. In order to distribute the SUs among SGs fairly, at first the system needs to find the proportion of active and standby SUs that are needed to be put together in an SG. For example, if the total number of active SUs is 100 and the total number of standby SUs is 50, the active proportion is 2 and the standby proportion is 1.
  • the number of SUs in an SG can be between the number of ActProportion + StdProportion , and the NoOfNodes .
  • the system calculates the total number of active and standby SUs that can be put together to be compliant with the NoOfNodes.
  • NumberOfSUsPerOneSG min Ceil floor NoOfNodes ActProportion + StdProportion ⁇ ActProportion + StdProportion , NumberOfActSUs + NumberOfStdSUs
  • NumberOfSGs Ceil NumberOfActSUs + NumberOfStdSUs NumberOfSUsPerOneSG
  • the system calculates the maximum number of SIs that can be assigned to an active SU, by using the Eq. (23).
  • the system divides the NoOfNodes by NoOfActiveAssignments. This is because it is reserving some nodes for the SIs' other assignments. This can be clarified with an example.
  • NumberOfSUsPerOneSG NoOfActiveAssignment ⁇ Ceil MaxNumberOfSIsPerSG MaxNoOfSIsPerOneActSU
  • NWay Redundancy model In the NWay redundancy model, the calculations are more complex than the other redundancy models.
  • the NoOfStandbyAssignments has to be given as an input (i.e., configuration requirements).
  • an SU can be assigned as active for some SIs and in the meanwhile, it can be assigned as standby for the other SIs.
  • active and standby SUs it is meant the SUs that are needed to handle the SI's active and standby assignments, respectively.
  • the same calculations are used as presented for NWay-Active redundancy model.
  • the calculations were shown only for active point of view.
  • the system uses the same calculations, but for both active and standby points of view. Then, the system chooses the result from one of them as the reference result of the calculations.
  • the system uses Eq. (21) to calculate the maximum number of SIs that can be handled by one SU.
  • the system has to reserve NoOfStandbyAssignments nodes (or SUs) in addition to the active SUs, in order to handle the NoOfStandbyAssignments simultaneous failures.
  • NumberOfActiveSGs Ceil NoOfSIs MaxNumberOfSIsPerActiveSG
  • NumberOfActiveSUsPerOneSG Ceil MaxNumberOfSIsPerActiveSG MaxNoOfSIsPerOneActSU + NoOfStandbyAssignments
  • NumberOfStandbySGs Ceil NoOfSIs MaxNumberOfSIsPerStandbySG
  • NumberOfStandbySUsPerOneSG Ceil MaxNumberOfSIsPerStandbySG MaxNoOfSIsPerOneStandbySU + 1
  • the system can choose the reference result from the one that resulted in a greater number of SGs. If both points of view resulted in the same number of SGs, it can choose the one that resulted in a greater number of SUs.
  • Figure 11 illustrates example configuration requirements for the availability estimate-based entities creation method according to one embodiment.
  • the system calculates the number of AMF entities based on the partial configuration requirement information that is illustrated in Figure 11 .
  • Each SI consists of the CSIs of the two CSI templates.
  • each SI should have five CSIs of the FTP CST and five CSIs of the HTTP CST.
  • the minimum level of service availability for the SI template is defined as 0.999.
  • the number of nodes in the cluster is set to five nodes.
  • the system selects the ETF prototypes that can provide the requested service. Then, it derives the corresponding AMF types base on the selected prototypes. Once it has derived the AMF types, it applies the availability estimate-based entities creation method to calculate the number of entities of each AMF type.
  • Figure 12 illustrates the derived AMF types that are used in this example.
  • Algorithm II is used to calculate the minimum number of components of each CT (i.e., FTP_CT and HTTP_CT) to be placed in an SU.
  • Algorithm III is used to calculate the maximum number of components of each CT that can be put in an SU. The results given by using the algorithms is as follows:
  • Algorithm IV is used to calculate the number of components to be put together in the SU, with respect to the requested level of service availability requested (i.e., 0.999).
  • the system sets the number of components of the CTs according to the previous iteration. It means that the number of components of the FTP_CT and HTTP_CT in the SU should be equal to 2 and 4, respectively.
  • NoOfSIs 12 Redundancy model of the SGs that that are going to protect the SI template: N+M
  • NoOfCompsPerCST 2 is the number components of the FTP_CT, which has been calculated above.
  • the number of SUs and SGs is calculated per each CST.
  • the calculations for FTP_CST and HTTP_CST will be presented separately. Then, one of them is selected as the reference result.
  • NumberOfSGs Ceil NumberOfActSUs + NumberOfStdSUs
  • the system needs 3 SGs, and each of them should group 4 SUs.
  • the calculations above were only for FTP_CST. Next, the same calculations are done for the HTTP_CST.
  • MaxNoOfSIsPerOneStdSU floor NoOfCompsPerCST ⁇ StandbyCapabilityPerCST
  • 3 SGs are to be created, with 4 SUs in each of the SG.
  • the system compares the results of the calculations for the CSTs and choose the one that resulted in a greater number of SGs.
  • Figure 14 illustrates the created entities in the example discussed above. It is noted that the distribution of SUs on the nodes and SIs assignments are performed by AMF at runtime. Therefore, Figure 14 shows only one of the possible entity distributions.
  • Embodiments of the invention may be represented as a software product stored in a machine-readable medium (also referred to as a computer-readable medium, a processor-readable medium, or a computer usable medium having a computer readable program code embodied therein).
  • the non-transitory machine-readable medium may be any suitable tangible medium including a magnetic, optical, or electrical storage medium including a diskette, compact disk read only memory (CD-ROM), digital versatile disc read only memory (DVD-ROM) memory device (volatile or non-volatile), or similar storage mechanism.
  • the machine-readable medium may contain various sets of instructions, code sequences, configuration information, or other data, which, when executed, cause a processor to perform steps in a method according to an embodiment of the invention.
  • Those of ordinary skill in the art will appreciate that other instructions and operations necessary to implement the described invention may also be stored on the machine-readable medium.
  • Software running from the machine-readable medium may interface with circuitry to perform the described tasks.
  • Figure 15A is a flow diagram illustrating a method 1500 for eliminating type stacks in the process of generating a configuration for a service provider system to provide an HA service according to one embodiment.
  • the method 1500 begins with identifying type stacks that provide the HA service and one or more component types in each type stack, wherein each type stack is a combination of prototypes that describe features and capabilities of available software providing the HA service (block 1510).
  • the method 1500 further comprises estimating, for each component type in the type stacks, an MTTR of the HA service based on time for completing an actual recovery action in response to a component failure (block 1520); estimating service availability provided by each type stack based on the MTTR and an MTTF of each component type in the type stack (block 1530); and eliminating one or more of the type stacks that do not satisfy a requested service availability before proceeding to subsequent steps of configuration generation (block 1540).
  • Figure 15B is a flow diagram illustrating a method 1550 for creating AMF entities in a configuration generation process for a service provider system to provide an HA service according to one embodiment.
  • the method 1550 begins after elimination of one or more type stacks according to the method 1500 of Figure 15A .
  • the method 1550 comprises creating AMF entities for at least one remaining type stack that has not been eliminated (block 1560).
  • the step of creating the AMF entities further comprises determining, based on the requested service availability, the number of components to be placed in an SU for each component type in the at least one remaining type stack (block 1570).
  • the service provider system may provide a network service, such as a virtualized network function or a chain of virtualized network functions.
  • the service provider system may be any mission critical system where service outage time needs to be minimized.
  • the methods 1500 and 1550 of Figure 15A and Figure 15B may be performed by hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device), or a combination thereof.
  • the methods of Figure 15A and Figure 15B may be performed by a system 1600 of Figure 16 and/or by a system 1700 of Figure 17 .
  • Figure 16 illustrates a system 1600 for generating a configuration for a service provider system for providing an HA service according to one embodiment.
  • the system 1600 performs the methods of Figure 15A and Figure 15B .
  • the system 1600 comprises an identification module 1610 adapted or operative to identify type stacks that provide the HA service and one or more component types in each type stack, wherein each type stack is a combination of prototypes that describe features and capabilities of available software providing the HA service; an MTTR estimation module 1620 adapted or operative to estimate, for each component type in the type stacks, an MTTR of the HA service based on time for completing an actual recovery action in response to a component failure; a service availability estimation module 1630 adapted or operative to estimate service availability provided by each type stack based on the MTTR and an MTTF of each component type in the type stack; and an elimination module 1640 adapted or operative to eliminate one or more of the type stacks that do not satisfy a requested service availability before proceeding to subsequent steps of configuration generation.
  • an identification module 1610 adapted or operative to identify type stacks that provide the HA service and one or more component types in each type stack, wherein each type stack is a combination of prototypes that describe features and capabilities of available software providing the
  • Figure 17 illustrates a diagrammatic representation of a machine in the exemplary form of a system 1700 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.
  • the system 1700 may be part of a network node (e.g., a router, switch, bridge, controller, base station, etc.).
  • the system 1700 may operate in a cloud computing environment where multiple server computers in one or more service centers collectively provide computing services on demand.
  • the computer system 1700 may be a server computer, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
  • the term "machine” shall also be taken to include any collection of machines (e.g., computers) that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
  • the system 1700 includes a processing device 1702.
  • the processing device 1702 represents one or more general-purpose processors, each of which can be: a microprocessor, a central processing unit (CPU), a multicore system, or the like.
  • the processing device 1702 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like.
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • DSP digital signal processor
  • the processing device 1702 is adapted or operative to perform the methods of Figure 15A and Figure 15B .
  • the processing device 1702 is adapted or operative to execute the operations of a configuration generation logic 1722, which contains instructions executable by the processing device 1702 to perform the methods of Figure 15A and Figure 15B .
  • the processor device 1702 is coupled to one or more memory devices such as: a main memory 1704 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), etc.), a secondary memory 1718 (e.g., a magnetic data storage device, an optical magnetic data storage device, etc.), and other forms of computer-readable media, which communicate with each other via a bus or interconnect 1730.
  • the memory devices may also include different forms of read-only memories (ROMs), different forms of random access memories (RAMs), static random access memory (SRAM), or any type of media suitable for storing electronic instructions.
  • the memory devices may store the code and data of the configuration generation logic 1722.
  • the configuration generation logic 1722 may be located in one or more of the locations shown as dotted boxes and labeled by the reference numeral 1722. In alternative embodiments the configuration generation logic 1722 may be located in other location(s) not shown in Figure 17 .
  • the system 1700 may further include a network interface device 1708.
  • a part or all of the data and code of the configuration generation logic 1722 may be transmitted or received over a network 1720 via the network interface device 1708.
  • the configuration generation logic 1722 can be implemented using code and data stored and executed on one or more computer systems (e.g., the system 1700).
  • Such computer systems store and transmit (internally and/or with other electronic devices over a network) code (composed of software instructions) and data using computer-readable media, such as non-transitory tangible computer-readable media (e.g., computer-readable storage media such as magnetic disks; optical disks; read only memory; flash memory) and transitory computer-readable transmission media (e.g., electrical, optical, acoustical or other form of propagated signals - such as carrier waves, infrared signals).
  • a non-transitory computer-readable medium of a given computer system typically stores instructions for execution on one or more processors of that computer system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Hardware Redundancy (AREA)
  • Debugging And Monitoring (AREA)

Claims (14)

  1. Procédé mis en oeuvre par ordinateur pour la génération d'une configuration pour un système de fournisseur de services pour fournir un service de grande disponibilité, HA, le procédé comprenant :
    l'identification de piles de types qui fournissent le service HA et un ou plusieurs types de composants dans chaque pile de types, dans lequel chaque pile de types contient un ensemble de types de composants en bas et est un agencement de prototypes qui i) décrivent des caractéristiques et des capacités de logiciels disponibles fournissant le service HA et ii) sont décrits dans un fichier de types d'entités, ETF, dans lequel le type de composant décrit des caractéristiques d'une version spécifique d'une mise en oeuvre logicielle pouvant être utilisée pour instancier des composants et dans lequel le type de composant est un prototype spécifique défini pour un ETF ;
    l'estimation, pour chaque type de composant dans les piles de types, d'un temps moyen de reprise MTTR, du service HA sur la base du temps d'exécution d'une action de reprise réelle en réponse à une défaillance de composant ;
    l'estimation d'une disponibilité de service fournie par chaque pile de types sur la base du MTTR et d'un temps moyen jusqu'à défaillance, MTTF, de chaque type de composant dans la pile de types ; et
    l'élimination d'une ou plusieurs des piles de types qui ne satisfont pas une disponibilité de service demandée avant de passer aux étapes suivantes de génération de configuration.
  2. Procédé selon la revendication 1, dans lequel :
    - l'estimation de la disponibilité de service comprend en outre : le comptage d'un composant unique pour chaque type de composant dans les piles de types lors de l'estimation de la disponibilité de service ; ou
    - le MTTR est estimé à un ou plusieurs niveaux incluant un niveau de composant, un niveau d'unité de service, SU, un niveau de noeud et un niveau d'application ; ou
    - lorsque l'action de reprise réelle inclut une action de reprise de redémarrage de composant, le procédé comprend en outre l'estimation du MTTR d'un type de composant en tant qu'une somme d'un premier temps moyen d'interruption en raison de tentatives d'instanciation de composant sans retard, d'un deuxième temps moyen d'interruption en raison de tentatives d'instanciation de composant avec retard, et d'un troisième temps moyen d'interruption en raison d'un échec de toutes les tentatives d'instanciation de composant ; ou
    - lorsque l'action de reprise réelle inclut une action de reprise après défaillance de composant, le procédé comprend en outre l'estimation du MTTR d'un type de composant en tant qu'une somme d'un premier élément pour lequel le nettoyage de composant est réussi et d'un deuxième élément pour lequel le nettoyage de composant n'est pas réussi ; ou
    - l'estimation de la disponibilité de service comprend en outre l'identification d'une dépendance à un ou plusieurs niveaux à l'intérieur du service HA ; et l'ajustement d'une estimation de la disponibilité de service pour tenir compte de la dépendance ; ou
    - l'estimation de la disponibilité de service comprend en outre : l'estimation de la disponibilité de service, STSA, d'une pile de types qui inclut N types de composants pour fournir le service HA en tant que : STSA = ∏ j = 1 j ≤ N MTT F j MTT F j + MTT R j ;
    Figure imgb0084
     ou
    - le service HA est d'un premier type de service défini pour une application, et dans lequel l'estimation de la disponibilité de service comprend en outre : l'estimation de la disponibilité de service pour chacun d'une pluralité de types de services définis pour l'application ; et l'élimination d'un sous-ensemble des piles de types qui ne satisfont pas la disponibilité de service demandée pour un ou plusieurs des types de services ; ou
    - le service HA est d'un premier type de service défini pour une application, et dans lequel l'estimation de la disponibilité de service comprend en outre : l'estimation de la disponibilité de service pour chacun d'une pluralité de types de services définis pour l'application ; et la sélection d'un sous-ensemble des piles de types qui fournissent une disponibilité de service pour un ou plusieurs des types de services supérieure à celle du reste des piles de types.
  3. Procédé selon la revendication 1, dans lequel, après l'élimination de l'une ou plusieurs piles de types, le procédé comprend en outre :
    la création d'entités de cadre de gestion de disponibilité, AMF, pour au moins une pile de types restante qui n'a pas été éliminée, dans lequel la création des entités AMF comprend en outre la détermination, sur la base de la disponibilité de service demandée, d'un nombre de composants à placer dans une unité de service, SU, pour chaque type de composant dans l'au moins une pile de types restante.
  4. Procédé selon la revendication 3, dans lequel la détermination du nombre de composant comprend en outre :
    le calcul d'un nombre minimal de composants d'un type de composant à placer dans la SU pour fournir un nombre demandé d'instances de services de composants, CSI, dans une instance de service, SI, unique, dans lequel les CSI sont d'un type de service de composant, CST, fourni par le type de composant ; et
    le calcul d'un nombre maximal de composants du type de composant à placer dans la SU pour garantir que la SU fournisse un nombre demandé de SI pour le service HA.
  5. Procédé selon la revendication 3, comprenant en outre :
    - le calcul, pour chacun d'une pluralité de types de services de composants, CST, du service HA, d'un nombre maximal d'instances de service, SI, pouvant être fournies par une SU ; et le calcul d'un premier nombre de SU pour prendre en charge un nombre demandé de SI sous réserve d'un modèle de redondance donné déterminant des assignations de SI à des SU par groupe de services, SG ; ou
    - le calcul, pour chacun d'une pluralité de types de services de composants, CST, du service HA, d'un nombre de groupes de services, SG, sous réserve d'un modèle de redondance donné déterminant des assignations d'instances de services, SI, à des unités de services, SU, par groupe de services, SG ; et le choix d'un nombre maximal de SG parmi les CST ; ou
    - le calcul, pour un modèle de redondance donné fournissant des assignations redondantes d'instances de services, SI, à des unités de services, SU, d'au moins l'un d'un premier nombre maximal de SI par SU active et d'un deuxième nombre maximal de SI par SU en veille ; et le calcul d'un nombre de groupes de services, SG, sur la base d'un nombre demandé de SI, et de l'au moins un du premier nombre maximal et du deuxième nombre maximal.
  6. Support lisible par ordinateur comprenant des portions de code qui, lorsqu'elles sont exécutées sur un processeur, configurent le processeur pour effectuer toutes les étapes d'un procédé selon l'une quelconque des revendications précédentes de procédé.
  7. Système (1700) apte à générer une configuration pour un système de fournisseur de services pour fournir un service de grande disponibilité, HA, le système comprenant :
    une mémoire (1704, 1718) ; et
    un ou plusieurs processeurs (1702) couplés à la mémoire, les un ou plusieurs processeurs étant aptes à effectuer :
    - l'identification de piles de types qui fournissent le service HA et un ou plusieurs types de composants dans chaque pile de types, dans lequel chaque pile de types contient un ensemble de types de composants en bas et est un agencement de prototypes qui i) décrivent des caractéristiques et des capacités de logiciels disponibles fournissant le service HA et ii) sont décrits dans un fichier de types d'entités, ETF, dans lequel le type de composant décrit des caractéristiques d'une version spécifique d'une mise en oeuvre logicielle pouvant être utilisée pour instancier des composants et dans lequel le type de composant est un prototype spécifique défini pour un ETF ;
    - l'estimation, pour chaque type de composant dans les piles de types, d'un temps moyen de reprise MTTR, du service HA sur la base du temps d'exécution d'une action de reprise réelle en réponse à une défaillance de composant ;
    - l'estimation d'une disponibilité de service fournie par chaque pile de types sur la base du MTTR et d'un temps moyen jusqu'à défaillance, MTTF, de chaque type de composant dans la pile de types ; et
    - l'élimination d'une ou plusieurs des piles de types qui ne satisfont pas une disponibilité de service demandée avant de passer aux étapes suivantes de génération de configuration.
  8. Système selon la revendication 7, dans lequel :
    - les un ou plusieurs processeurs sont en outre aptes à effectuer le comptage d'un composant unique pour chaque type de composant dans les piles de types lors de l'estimation de la disponibilité de service ; ou
    - le MTTR est estimé à un ou plusieurs niveaux incluant un niveau de composant, un niveau d'unité de service, SU, un niveau de noeud et un niveau d'application ; ou
    - lorsque l'action de reprise réelle inclut une action de reprise de redémarrage de composant, les un ou plusieurs processeurs sont en outre aptes à effectuer l'estimation du MTTR d'un type de composant en tant qu'une somme d'un premier temps moyen d'interruption en raison de tentatives d'instanciation de composant sans retard, d'un deuxième temps moyen d'interruption en raison de tentatives d'instanciation de composant avec retard, et d'un troisième temps moyen d'interruption en raison d'un échec de toutes les tentatives d'instanciation de composant ; ou
    - lorsque l'action de reprise réelle inclut une action de reprise après défaillance de composant, les un ou plusieurs processeurs sont en outre aptes à effectuer l'estimation du MTTR d'un type de composant en tant qu'une somme d'un premier élément pour lequel le nettoyage de composant est réussi et d'un deuxième élément pour lequel le nettoyage de composant n'est pas réussi ; ou
    - les un ou plusieurs processeurs sont en outre aptes à effectuer l'identification d'une dépendance à un ou plusieurs niveaux à l'intérieur du service HA ; et l'ajustement d'une estimation de la disponibilité de service pour tenir compte de la dépendance ; ou
    - les un ou plusieurs processeurs sont en outre aptes à effectuer l'estimation de la disponibilité de service, STSA, d'une pile de types qui inclut N types de composants pour fournir le service HA en tant que : STSA = ∏ j = 1 j ≤ N MTT F j MTT F j + MTT R j ;
    Figure imgb0085
     ou
    - le service HA est d'un premier type de service défini pour une application, et dans lequel les un ou plusieurs processeurs sont en outre aptes à effectuer : l'estimation de la disponibilité de service pour chacun d'une pluralité de types de services définis pour l'application ; et l'élimination d'un sous-ensemble des piles de types qui ne satisfont pas la disponibilité de service demandée pour un ou plusieurs des types de services ; ou
    - le service HA est d'un premier type de service défini pour une application, et dans lequel les un ou plusieurs processeurs sont en outre aptes à effectuer : l'estimation de la disponibilité de service pour chacun d'une pluralité de types de services définis pour l'application ; et la sélection d'un sous-ensemble des piles de types qui fournissent une disponibilité de service pour un ou plusieurs des types de services supérieure à celle du reste des piles de types.
  9. Système selon la revendication 7, dans lequel, après l'élimination de l'une ou plusieurs piles de types, les un ou plusieurs processeurs sont en outre aptes à effectuer :
    la création d'entités de cadre de gestion de disponibilité, AMF, pour au moins une pile de types restante qui n'a pas été éliminée ; et
    la détermination, sur la base de la disponibilité de service demandée, d'un nombre de composants à placer dans une unité de service, SU, pour chaque type de composant dans l'au moins une pile de types restante.
  10. Système selon la revendication 9, dans lequel les un ou plusieurs processeurs sont en outre aptes à effectuer :
    le calcul d'un nombre minimal de composants d'un type de composant à placer dans la SU pour fournir un nombre demandé d'instances de services de composants, CSI, dans une instance de service, SI, unique, dans lequel les CSI sont d'un type de service de composant, CST, fourni par le type de composant ; et
    le calcul d'un nombre maximal de composants du type de composant à placer dans la SU pour garantir que la SU fournisse un nombre demandé de SI pour le service HA.
  11. Système selon la revendication 10, dans lequel les un ou plusieurs processeurs sont en outre aptes à effectuer le calcul d'un nombre réel de composants du type de composant à placer dans la SU pour satisfaire la disponibilité de service demandée, dans lequel le nombre réel est un multiple entier du nombre minimal et est inférieur ou égal au nombre maximal.
  12. Système selon la revendication 9, dans lequel les un ou plusieurs processeurs sont en outre aptes à effectuer :
    le calcul, pour chacun d'une pluralité de types de services de composants, CST, du service HA, d'un nombre maximal d'instances de service, SI, pouvant être fournies par une SU ; et
    le calcul d'un premier nombre de SU pour prendre en charge un nombre demandé de SI sous réserve d'un modèle de redondance donné déterminant des assignations de SI à des SU par groupe de services, SG.
  13. Système selon la revendication 12, dans lequel les un ou plusieurs processeurs sont en outre aptes à effectuer :
    le calcul d'un deuxième nombre de SG lorsque le premier nombre de SU dépasse une taille de groupement ; et
    la distribution du premier nombre de SU parmi le deuxième nombre de SG sensiblement uniformément.
  14. Système selon la revendication 9, dans lequel les un ou plusieurs processeurs sont en outre aptes à effectuer :
    - le calcul, pour chacun d'une pluralité de types de services de composants, CST, du service HA, d'un nombre de groupes de services, SG, sous réserve d'un modèle de redondance donné déterminant des assignations d'instances de services, SI, à des unités de services, SU, par groupe de services, SG ; et le choix d'un nombre maximal de SG parmi les CST ; ou
    - le calcul, pour un modèle de redondance donné fournissant des assignations redondantes d'instances de services, SI, à des unités de services, SU, d'au moins l'un d'un premier nombre maximal de SI par SU active et d'un deuxième nombre maximal de SI par SU en veille ; et le calcul d'un nombre de groupes de services, SG, sur la base d'un nombre demandé de SI, et de l'au moins un du premier nombre maximal et du deuxième nombre maximal.
EP15714288.6A 2014-03-19 2015-03-12 Génération de configuration sur la base d'une estimation de disponibilité Active EP3120496B1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201461955535P 2014-03-19 2014-03-19
US201462018788P 2014-06-30 2014-06-30
PCT/IB2015/051823 WO2015140678A1 (fr) 2014-03-19 2015-03-12 Génération de configuration sur la base d'une estimation de disponibilité

Publications (2)

Publication Number Publication Date
EP3120496A1 EP3120496A1 (fr) 2017-01-25
EP3120496B1 true EP3120496B1 (fr) 2019-05-08

Family

ID=52811161

Family Applications (1)

Application Number Title Priority Date Filing Date
EP15714288.6A Active EP3120496B1 (fr) 2014-03-19 2015-03-12 Génération de configuration sur la base d'une estimation de disponibilité

Country Status (3)

Country Link
US (1) US10198308B2 (fr)
EP (1) EP3120496B1 (fr)
WO (1) WO2015140678A1 (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10764323B1 (en) * 2015-12-21 2020-09-01 Amdocs Development Limited System, method, and computer program for isolating services of a communication network in response to a distributed denial of service (DDoS) attack
US10275326B1 (en) * 2014-10-31 2019-04-30 Amazon Technologies, Inc. Distributed computing system failure detection
EP3458954B1 (fr) * 2016-05-19 2020-03-25 Telefonaktiebolaget LM Ericsson (publ) Procédé et système d'évaluation de campagnes de mise à niveau
US11349708B2 (en) 2017-03-09 2022-05-31 Telefonaktiebolaget L M Ericsson (Publ) Configuration generation for virtual network functions (VNFs) with requested service availability
US12026522B2 (en) * 2021-04-06 2024-07-02 International Business Machines Corporation Automatic application dependency management

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6411910B1 (en) * 2000-04-26 2002-06-25 American Power Conversion System and method for estimating power availability
WO2001084313A2 (fr) * 2000-05-02 2001-11-08 Sun Microsystems, Inc. Procede et systeme d'obtention d'une grande disponibilite dans un systeme informatique en reseau
US7830813B1 (en) 2004-09-30 2010-11-09 Avaya Inc. Traffic based availability analysis
US20060129367A1 (en) 2004-11-09 2006-06-15 Duke University Systems, methods, and computer program products for system online availability estimation
CA2504333A1 (fr) 2005-04-15 2006-10-15 Symbium Corporation Programmation et developpement d'infrastructure pour element autonome
EP1772820A1 (fr) * 2005-10-07 2007-04-11 Hewlett-Packard Development Company, L.P. Prévision de conformité de niveaux de service dans des infrastructures de technologie de l'information
WO2008041302A1 (fr) * 2006-09-29 2008-04-10 Fujitsu Limited Programme et procédé de disposition de serveur
US20080244552A1 (en) * 2007-03-27 2008-10-02 Telefonaktiebolaget Lm Ericsson (Publ) Upgrading services associated with high availability systems
CN102597957B (zh) * 2009-10-29 2015-01-21 日本电气株式会社 系统部署确定系统、系统部署确定方法及程序
US8695012B2 (en) * 2010-02-05 2014-04-08 Telefonaktiebolaget L M Ericsson (Publ) Load and backup assignment balancing in high availability systems
US20130297755A1 (en) * 2010-11-23 2013-11-07 Nokia Siemens Networks Oy Network element configuration management
US8549533B2 (en) * 2011-03-18 2013-10-01 Telefonaktiebolaget L M Ericsson (Publ) Ranking service units to provide and protect highly available services using N+M redundancy models
US10229005B2 (en) 2014-01-23 2019-03-12 Telefonaktiebolaget Lm Ericsson (Publ) Pattern based configuration method for minimizing the impact of component failures

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP3120496A1 (fr) 2017-01-25
WO2015140678A1 (fr) 2015-09-24
US20170068588A1 (en) 2017-03-09
US10198308B2 (en) 2019-02-05

Similar Documents

Publication Publication Date Title
EP3593494B1 (fr) Génération de configuration de fonctions de réseau virtuel avec disponibilité de service requise
EP3120496B1 (fr) Génération de configuration sur la base d'une estimation de disponibilité
US11379341B2 (en) Machine learning system for workload failover in a converged infrastructure
EP2532145B1 (fr) Balance de charge et redondance dans un système à forte disponibilité
US8549533B2 (en) Ranking service units to provide and protect highly available services using N+M redundancy models
US8738968B2 (en) Configuration based service availability analysis of AMF managed systems
CN106657167B (zh) 管理服务器、服务器集群、以及管理方法
EP3458954B1 (fr) Procédé et système d'évaluation de campagnes de mise à niveau
US9229839B2 (en) Implementing rate controls to limit timeout-based faults
US20110173616A1 (en) Determination and management of virtual networks
CN115698954A (zh) 管理故障转移区域可用性以实施故障转移服务
WO2023093354A1 (fr) Évitement de duplication de charge de travail parmi des grappes divisées
US11379320B2 (en) Container recovery
WO2016020731A1 (fr) Planificateur à haute disponibilité pour composant(e)s
Pourali et al. Pattern based configuration generation for highly available COTS components based systems
Anand Always on: Architecture for high availability cloud applications
US20240069974A1 (en) Workload scheduler for high availability
US20230244545A1 (en) Reliability-aware resource allocation method and apparatus in disaggregated data centers
Pourali Pattern-Based Generation of AMF Configurations
Pourali et al. Enhanced configuration generation approach for highly available COTS based systems
Ogawa et al. Virtual network allocation for fault tolerance with bandwidth efficiency in a multi-tenant data center
WO2023198276A1 (fr) Gestion de défaillance d'une instance d'application
CN116954970A (zh) 一种容错计算方法、装置、计算机设备及存储介质
MICHIARDI Techniques for efficient and fault-tolerant geo-replication

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20161019

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20180301

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20190128

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1131925

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190515

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602015029806

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190508

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190808

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190908

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190809

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190808

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1131925

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190508

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602015029806

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

26N No opposition filed

Effective date: 20200211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200312

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200312

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200331

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20210325

Year of fee payment: 7

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602015029806

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04L0012240000

Ipc: H04L0041000000

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190508

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190908

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220331

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230329

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240327

Year of fee payment: 10