US20190332415A1 - System and Method for Managing Size of Clusters in a Computing Environment - Google Patents
System and Method for Managing Size of Clusters in a Computing Environment Download PDFInfo
- Publication number
- US20190332415A1 US20190332415A1 US16/504,416 US201916504416A US2019332415A1 US 20190332415 A1 US20190332415 A1 US 20190332415A1 US 201916504416 A US201916504416 A US 201916504416A US 2019332415 A1 US2019332415 A1 US 2019332415A1
- Authority
- US
- United States
- Prior art keywords
- host
- cluster
- hosts
- data structure
- pool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims description 60
- 238000007726 management method Methods 0.000 description 31
- 230000004044 response Effects 0.000 description 15
- 230000001010 compromised effect Effects 0.000 description 12
- 238000001514 detection method Methods 0.000 description 11
- 230000005012 migration Effects 0.000 description 10
- 238000013508 migration Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 6
- 238000007792 addition Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 238000001152 differential interference contrast microscopy Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000017702 response to host Effects 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000009987 spinning Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/505—Clust
Definitions
- a virtual machine is a software abstraction of a physical computing system capable of running one or more applications under the control of a guest operating system, where the guest operating system interacts with an emulated hardware platform, also referred to as a virtual hardware platform.
- a virtual hardware platform is executed on a physical host device, such as a server-class computer.
- VMs are frequently employed in data centers, cloud computing platforms, and other distributed computing systems, and are executed on the physical host devices of such systems.
- these host devices are logically grouped or “clustered” together as a single logical construct.
- the aggregated computing and memory resources of the cluster that are available for running VMs can be provisioned flexibly and dynamically to the various VMs being executed.
- clusters of host devices often include reserved failover capacity, i.e., host devices in the cluster that remain idle during normal operation and are therefore available for executing VMs whenever a host device in the cluster fails.
- reserved failover capacity can make up a significant portion of the resources of a cluster, but are infrequently utilized. For example, for a distributed computing system that includes 50 clusters, where each cluster includes 10 host devices and has a failover capacity of 20%, then the capacity equivalent to 100 host devices are unused in the system until a failure occurs. Because failures are relatively infrequent, the majority of this reserved failover capacity is infrequently utilized, thereby incurring both capital and operational costs for little benefit.
- a number of hosts in a logical cluster is adjusted up or down in an elastic manner.
- a method of adjusting the number of hosts in the cluster includes the steps of tracking membership of hosts in the cluster using a first data structure and tracking membership of hosts in a spare pool using a second data structure, and upon determining that a triggering condition for adding another host is met and that all hosts in the cluster are being used, selecting a host from the spare pool, and programmatically adding an identifier of the selected host to the first data structure and programmatically deleting the identifier of the selected host from the second data structure.
- Non-transitory computer-readable medium that includes instructions that, when executed, enable a computer to implement one or more aspects of the above method, and a system of computers including a management server that is programmed to implement one or more aspects of the above method.
- FIG. 1 illustrates a computing environment, according to one embodiment.
- FIG. 2 is a conceptual diagram that depicts an expanded view of the hardware platform of a computer host of FIG. 1 , according to one or more embodiments.
- FIGS. 3A-3C are conceptual diagrams illustrating the logical removal of an available host from a spare host pool and the logical addition of the available host to one of the clusters, according to one or more embodiments.
- FIG. 4 sets forth a flowchart of method steps carried out by the VM management server of FIG. 1 to address low utilization in a cluster, according to one or more embodiments.
- FIG. 5 sets forth a flowchart of method steps carried out by the VM management server of FIG. 1 to address high utilization in a cluster, according to one or more embodiments.
- FIG. 6 sets forth a flowchart of method steps carried out by the VM management server of FIG. 1 in response to a failure of a host in a particular cluster, according to one or more embodiments.
- FIG. 7 sets forth a flowchart of method steps carried out by the VM management server of FIG. 1 in response to a partial failure of a host in a particular cluster, according to one or more embodiments.
- FIG. 1 illustrates a computing environment 100 , according to one embodiment.
- Computing environment 100 is a virtual data center, also referred to as a software-defined data center, and includes multiple clusters of host devices, or “clusters” 120 , a spare host pool 130 of available physical host devices, or “hosts,” a virtual machine (VM) management server 140 , and, in some embodiments, a failed host pool 150 .
- Computing environment 100 may include multiple virtual data centers residing within a single physical data center. Alternatively or additionally, the virtual components of computing environment 100 may be distributed across multiple physical data centers or locations.
- Each cluster 120 of computing environment 100 includes a plurality of hosts 121 A- 121 N (referred to collectively as hosts 121 ), each configured to execute one more VMs 122 .
- the number N of hosts 121 logically included in a particular cluster 120 can be varied from a predetermined minimum value, for example four, to a maximum implementable value, for example 64 or 128. More specifically, one or more hosts 121 are logically added to and/or removed from a particular cluster 120 in response to utilization of the cluster exceeding a maximum utilization threshold value, falling below a minimum utilization threshold value, and/or detection of a partial or total host failure.
- hosts 121 is described in greater detail below in conjunction with FIG. 2 .
- Spare host pool 130 includes a plurality of available hosts 131 A- 131 M (referred to collectively as hosts 131 ).
- the number M of hosts 131 logically included in spare host pool 130 can be varied dynamically. Specifically, as available hosts 131 are logically removed from spare host pool 130 and logically added to any of clusters 120 as additional hosts 121 , the number M of available hosts 131 in spare host pool 130 decreases. Likewise, as hosts 121 are logically removed from any of cluster 120 and logically added to spare host pool 130 as available hosts 131 , the number M of available hosts 131 in spare host pool 130 increases.
- Each of available hosts 131 may be substantially similar in physical configuration to hosts 121 , which are described in greater detail below in conjunction with FIG. 2 .
- spare host pool 130 may include available hosts 131 that are provisioned to computing environment 100 as cloud-based resources, and consequently are located at one more physical locations and/or data centers that are remote from the hosts 121 included in one or more of clusters 120 . In other embodiments, available hosts 131 are located in the same physical location and/or data center as the hosts 121 included in one or more of clusters 120 .
- VM management server 140 implements a management plane of computing environment 100 and is configured to manage hosts 121 and available hosts 131 associated with computing environment 100 . More specifically, VM management server 140 is configured to manage interactions between hosts 121 , to determine when available hosts 131 are added to a particular cluster 120 , and to determine when hosts 121 are removed from a particular cluster 120 and added to spare host pool 130 . To that end, VM management server 140 includes a high availability (HA) module 141 , a distributed resource scheduler (DRS) module 142 , and a host provisioning module 143 . HA module 141 and DRS module 142 are each configured for a computing environment in which clusters 120 are elastic, i.e., each cluster 120 can be dynamically increased and decreased in size in response to host failure and/or cluster utilization levels.
- HA high availability
- DRS distributed resource scheduler
- VM management server 140 typically runs on a computer host that is separate from hosts 121 and is accessible by a system administrator via a graphical user interface (GUI) or via a command line interface (CLI).
- GUI graphical user interface
- CLI command line interface
- VM management server 140 may be configured to enable a system administrator to perform various management functions for computing system 100 in addition to the automated functionality of HA module 141 and DRS module 142 .
- VM management server 140 may be configured to enable a system administrator to deploy VMs 122 to the different hosts 121 in a particular cluster, monitor the performance of such VMs 122 , and/or manually power up and power down VMs 122 .
- HA module 141 is configured to ensure availability of VMs 121 executing in each cluster 120 in response to a partial or total host failure. For example, when one of hosts 121 in a particular cluster 120 experiences a software crash or hardware fault and is no longer operable, HA module 141 is configured to trigger migration module 144 to migrate VMs executing in the failed host 121 to other hosts 121 within that cluster 120 . In addition, according various embodiments, HA module 141 is configured with a failure detection module 141 A to monitor clusters 120 for host faults. For example, failure detection module 141 A periodically polls each host 121 in computing environment 100 for software crashes and/or hardware faults or evidence thereof. Alternatively or additionally, failure detection module 141 A may be configured to receive fault warnings from hosts 121 in each cluster 120 .
- DRS module 142 is configured to maintain the utilization of hosts 121 in each cluster 120 between a minimum utilization threshold value and a maximum utilization threshold. To that end, DRS module 142 is configured to determine whether utilization of hosts 121 in each cluster 120 is above the minimum utilization threshold value and less than the maximum utilization threshold. For example, resource utilization monitor 142 A in DRS module 142 polls the hosts 121 in each cluster 120 for utilization information, such as central processing unit (CPU) time, allocated memory, and the like. In addition, DRS module 142 is configured to perform conventional load balancing between hosts 121 within a particular cluster 120 .
- CPU central processing unit
- DRS module 142 triggers migration module 144 to perform migration of executing VMs 122 from more highly loaded hosts 121 to first host, which is lightly loaded. In this way, DRS module 142 maintains utilization of the computing resources of hosts 121 in a particular cluster 120 within an optimal range.
- Host provisioning module 143 is configured to logically add one or more available hosts 131 to a cluster 120 or remove hosts 121 from the cluster 120 in response to a triggering event, so that utilization of computing resources of hosts 121 and availability of VMs 122 executing on hosts 121 are maintained within an optimal range.
- a triggering event may be detection of an average utilization of hosts 121 in a cluster 120 exceeding a maximum or minimum threshold value, or the detection of a host 121 in the cluster 120 undergoing a partial or total failure.
- host provisioning module 143 is configured to logically add one or more available hosts 131 to a particular cluster 120 when a host 121 in the particular cluster 120 suffers a partial or total failure. Afterwards, HA module 141 is able to migrate VMs executing in the failed host 121 to other hosts 121 within that cluster 120 and/or to the newly added available host 131 . For example, when a large number of VMs 122 are executing in a particular cluster 120 and a host failure occurs, the physical computing resources per VM 122 are reduced, i.e., CPU time, allocated memory, etc., and availability for any particular VM 122 in the cluster 120 is also reduced.
- host provisioning module 143 is configured to rectify this issue by logically adding one or more available hosts 131 to the cluster 120 based not only on the detection of a host failure, but also on an availability of the VMs 122 executing in the cluster 120 .
- host provisioning module 143 may logically add one or more available hosts 131 to the cluster 120 experiencing the host failure when availability for one or more VMs 122 in the cluster 120 cannot be increased above the minimum threshold value via migration of VMs 122 within the cluster 120 .
- host provisioning module 143 is configured to logically add available hosts 131 to a particular cluster 120 in which the utilization of computing resources of the hosts 121 in the particular cluster 120 is greater than a maximum utilization threshold for the cluster 120 . Thus, host provisioning module 143 increases the total number hosts within the cluster when utilization is too high. Further, in some embodiments, host provisioning module 143 is configured to logically remove hosts 121 from a particular cluster 120 in which the utilization of hosts 121 in the particular cluster 120 is less than a minimum utilization threshold value for the cluster 120 . In such embodiments, host provisioning module 143 decreases the total number of hosts within the cluster when utilization is too low.
- Metrics for quantifying availability of a particular VM 122 may include, for example, effective CPU resources available for the VM 122 of interest (in MHz or GHz), effective memory resources available for the VM 122 of interest (in kB, MB, or GB), a CPU fairness value representing the fraction of host CPU resources allocated to the VM 122 of interest (in percent), and/or a memory fairness value representing the fraction of memory resources allocated to the VM 122 of interest (in percent).
- host provisioning module 143 logically adds available hosts 131 to and/or removes hosts 121 from a particular cluster 120 in response to host failures, low VM availability, and/or low or high host utilization is described below in conjunction with FIGS. 4-7 .
- computing environment 100 may include a failed host pool 150 .
- Failed host pool 150 is a logical construct with which hosts 120 that have undergone a partial or total failure can be associated. Hosts 120 that have undergone either partial or total failure are logically associated with failed host pool 150 as failed hosts 151 , so that these host devices cannot be subsequently added to a cluster 120 of computing environment 100 . Thus, failed host pool 150 facilitates the identification of host devices in computing environment 100 that require maintenance, diagnostic analysis, and/or replacement.
- FIG. 2 schematically illustrates a cluster 120 of computing environment 100 in FIG. 1 , according to one or more embodiments.
- cluster 120 includes up to N hosts 121 , i.e., hosts 121 A, 121 B, . . . , 121 N, each of which is communicatively coupled to VM management server 140 .
- cluster 120 is a software-based “virtual storage area network” (VSAN) environment that leverages the commodity local storage resources housed in or directly attached to hosts 121 in cluster 120 to provide an aggregate object store 230 to virtual machines (VMs) 122 executing on hosts 121 .
- VSAN virtual storage area network
- VMs virtual machines
- use of the term “housed” or “housed in” may be used to encompass both housed in or otherwise directly attached to.
- Each host 121 is a computing device in which virtualization software and a plurality of VMs 122 are executing.
- Each host 121 is typically a server-class computer, although, in some embodiments, hosts 121 may include a variety of classes of computers, such as mainframe computers, desktop computers, and laptop computers.
- Each host 121 includes a virtualization layer or hypervisor 213 , a storage management module (referred to herein as a “VSAN module”) 214 , and a hardware platform 225 that typically includes central processing units (CPUs), random access memory (RAM), local storage resources, such as solid state drives (SSDs) 226 and/or magnetic disks 227 , a host bus adapter (HBA) that enable external storage devices to be connected to the host 120 , and one or more physical NICs that enable host 121 to communicate over a network.
- These physical computing resources of each host 121 are managed by hypervisor 213 , and through hypervisor 213 , a host 121 is able to launch and run multiple VMs 122 .
- the local commodity storage resources housed in or otherwise directly attached to hosts 121 may include combinations of SSDs 226 and/or magnetic or spinning disks 227 .
- SSDs 226 serve as a read cache and/or write buffer in front of magnetic disks 227 to increase I/O performance.
- VSAN module 214 in each host 121 is configured to automate storage management workflows (e.g., creating objects in object store 230 ) and provide access to objects in object store 230 (e.g., handling I/O operations to objects in object store 230 ).
- VSAN module 214 may then create an “object” for the specified virtual disk associated with the VM 122 by backing the virtual disk with physical storage resources of object store 230 such as SSDs 226 and/or magnetic disks 227 .
- object store 230 such as SSDs 226 and/or magnetic disks 227 .
- SSD 226 A and magnetic disk 227 A of host 121 A, SSD 226 B and magnetic disk 227 B of host 121 B, and so on may in combination be used to back object store 230 .
- Each VM 122 is a software abstraction of a physical computing system that is capable of running one or more applications under the control of a guest operating system (not shown), where the guest operating system provides various operating system services (such as central processing unit (or CPU) allocation and memory management).
- the guest operating system interacts with a virtual hardware platform, which is an emulated hardware platform for the corresponding VM.
- Virtual hardware platforms (not depicted in the figure) are implemented on a particular host 121 by hypervisor 213 , and typically comprise virtual CPUs, virtual RAM, virtual disks, and, for network communication, a virtual network interface controller (or NIC). That applications executing within VMs 121 are executing in a virtual machine is transparent to operation of these applications. Thus, such applications may be installed in a VM 122 unchanged from how such applications would be installed on a physical computer.
- the guest operating system installed in each VM 122 is executing on top of a virtualized hardware platform, rather than on a physical hardware platform is transparent to the guest operating system.
- the guest operating system may be installed in a VM 122 in the same manner that the guest operating system is installed on a physical computer. Examples of guest operating systems include the various versions of Microsoft's Windows® operating system, the Linux operating system, and Apple's Mac OS X.
- Each VM 122 executing in a particular cluster 120 accesses computing services by interacting with the virtual hardware platform associated with the particular cluster 120 and implemented by hypervisor 213 .
- each host 121 has one hypervisor 213 executing therein.
- hypervisor 213 provides a virtualization platform on which VMs 122 execute.
- Hypervisor 213 enables each VM 122 that executes under its control to access physical computing resources in host 120 .
- hypervisor 213 schedules the virtual CPU of that VM 122 to run on a physical CPU of host 120 .
- an application executing in a VM 122 may require additional virtual RAM in order to execute a particular function.
- the application would issue a memory allocation request to the guest operating system, which would allocate an amount of virtual RAM to satisfy the request.
- hypervisor 213 would allocate a certain amount of physical RAM that corresponds to the allocated virtual RAM.
- Hypervisor 213 also manages the physical computing resources of the corresponding host 120 , and allocates those resources among the executing VMs 122 , as well as other (i.e., non-VM) processes. Hypervisor 213 allocates physical CPU time to virtual CPUs in the VMs 122 , as well as physical disk space to virtual disks for each of the VMs 122 . Hypervisor 213 also enables the transmission and receipt of network traffic (through the virtual NICs) for each VM 122 .
- VSAN module 214 can, in some embodiments, be implemented as a VSAN device driver within hypervisor 213 .
- VSAN module 214 provides access to a conceptual VSAN 215 through which an administrator can create a number of top-level “device” or namespace objects that are backed by object store 230 .
- the administrator may specify a particular file system for the device object (referred to as “file system objects”).
- file system objects For example, each hypervisor 213 in each host 120 may, during a boot process, discover a/vsan/root node for a conceptual global namespace that is exposed by VSAN module 214 .
- hypervisor 213 can then determine the top-level file system objects (or other types of top-level device objects) currently residing in VSAN 215 .
- hypervisor 213 may dynamically “auto-mount” the file system object at that time.
- Each VSAN module 214 communicates with other VSAN modules 214 of other hosts 121 in cluster 120 to create and maintain an in-memory metadata database that contains metadata describing the locations, configurations, policies, and relationships among the various objects stored in object store 230 .
- an in-memory metadata database is maintained separately but in synchronized fashion in the memory of each host 121 .
- hosts 121 in cluster 120 are connected to the same shared storage formed from aggregated local storage resources, i.e., object store 230 , via a VSAN, i.e., VSAN 215 .
- cluster 120 may be configured to connect hosts 121 to shared storage using other approaches.
- the shared storage may be provided through a storage area network (SAN), and access to the SAN is provided by the HBA in each host 121 .
- the shared storage may be provided by a network attached storage (NAS) which is accessed through the NIC in each host 121 .
- SAN storage area network
- NAS network attached storage
- FIGS. 3A-3C are conceptual diagrams illustrating the logical removal of an available host 131 from spare host pool 130 and the logical addition of the available host 131 to one of the clusters 120 , according to one or more embodiments.
- host provisioning module 143 maintains a table for each cluster to track which hosts belong to which cluster. When a host is logically added to a cluster, the host identifier (ID) is added to the table for that cluster, and when a host is logically removed from a cluster, the host ID is deleted from the table.
- ID host identifier
- FIG. 3A illustrates spare host pool 130 , failed host pool 150 , and one of clusters 120 of computing environment 100 prior to a triggering event that results in a change in the number of hosts 121 logically included in cluster 120 .
- spare host pool 131 includes a plurality of available hosts 131 , for example, five
- cluster 120 includes multiple hosts 121 , for example three
- failed host pool 150 includes one failed host 151 .
- FIG. 3B illustrates spare host pool 130 , failed host pool 150 , and one of clusters 120 of computing environment 100 after a triggering event is detected.
- a triggering event may be the detection of an average utilization of hosts 121 in cluster 120 exceeding a maximum threshold value or an availability of a particular VM 122 (or group of VMs 122 ) falling below a minimum allowable threshold value.
- such a triggering event may be the detection of a host 121 undergoing a partial or total failure.
- Partial failures of a host 121 that may be considered a triggering event include the failure of a component of the host that allows continued operation of the host 121 , such as a partial memory failure, a fan failure, the failure of a single disk of a multi-disk drive, the failure of a single magnetic disk drive in a host 121 that includes multiple magnetic disk drives, etc.
- Total failures of a host 121 that may be considered a triggering event include a software-related host freeze, complete failure of the only magnetic disk drive in the host, and any other faults or failures that prevent the continued operation of the host 121 .
- host provisioning module 143 adds an available host 131 to cluster 120 as a new host 321 , as shown in FIG. 3B .
- HA module 141 may not be able to maintain an availability of a particular VM 122 or group of VMs 122 if the VMs were to be migrated from failed host 321 to different hosts 121 within cluster 120 .
- host provisioning module 143 increases the number of hosts 121 in the cluster 120 by logically adding an available host 131 from spare host pool 130 , as shown.
- host provisioning module 143 logically removes failed host 321 from cluster 120 and logically adds failed host 321 to failed host pool 150 , as shown.
- host provisioning module 143 increases the number of hosts 121 in the cluster 120 by adding an available host 131 from spare host pool 130 .
- FIG. 3C illustrates spare host pool 130 , failed host pool 150 , and one of clusters 120 of computing environment 100 after the number of hosts 120 in cluster 120 has been increased, for example in response to a failure of a host 121 in cluster 120 .
- spare host pool 131 includes one fewer available hosts 131 , for example, four
- cluster 120 includes the same number of hosts 121 , for example three
- failed host pool 150 includes an additional failed host 151 , for example two.
- FIG. 4 sets forth a flowchart of method steps carried out by VM management server 140 to address low utilization in a cluster, according to one or more embodiments.
- the method steps in FIG. 4 are described in conjunction with computing environment 100 of FIGS. 1-3 , persons skilled in the art will understand that the method in FIG. 4 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs.
- a method 400 begins at step 401 , in which resource utilization monitor 142 A of VM management server 140 determines that a utilization in a particular cluster 120 is less than a minimum threshold value.
- the utilization may be a utilization of a specific host 121 in the particular cluster 120 .
- the utilization may be a utilization associated with a group of hosts 121 in the particular cluster 120 , or of all hosts 121 in the particular cluster 120 , such as an average utilization thereof.
- the utilization is typically measured or quantified via performance monitoring functions included in VM management server 140 , and may be quantified in terms of computing resources in use by a host or hosts in the particular cluster, such as percentage utilization of CPU, RAM, and the like.
- VM management server 140 reduces the current number of hosts 121 that are logically included in the particular cluster 120 via the subsequent steps of method 400 .
- step 402 in response to the determination of step 401 , host provisioning module 143 VM management server 140 selects a host 121 in the particular cluster 120 to be logically removed therefrom.
- the selected host is the host 121 in the particular cluster 120 with the highest utilization, thereby maximizing the impact on utilization in the particular cluster 120 .
- additional write I/O's from VMs executing on hosts 121 to the local storage resources included in the selected host are not allowed, so that data stored locally on the selected host can be moved to the remaining hosts 121 .
- read I/O's are still permitted to the selected host, so that VMs 122 executing on hosts 121 can access object store 230 as needed.
- DRS module 142 triggers migration of VMs (performed by migration module 144 ) from the selected host to other hosts 121 in the particular cluster.
- Techniques for load-balancing between hosts 121 that are well-known in the art may be employed to complete migration of VMs from the selected host in step 403 .
- step 404 DRS module 142 copies data that are stored, as part of object store 230 , in local storage resources housed in the selected host. The data are copied to other local storage resources housed in one or more other hosts 121 in the particular cluster 120 .
- VMs executing in the particular cluster 120 no longer access the local storage resources housed in the selected host, since all file system objects associated with VSAN 215 are stored elsewhere within the particular cluster 120 .
- step 405 host provisioning module 143 logically removes the selected host from the particular cluster 120 .
- step 406 host provisioning module 143 logically adds the selected host to available host pool 130 as an additional available host 131 .
- FIG. 5 sets forth a flowchart of method steps carried out by VM management server 140 to address high utilization in a cluster, according to one or more embodiments.
- the method steps in FIG. 5 are described in conjunction with computing environment 100 of FIGS. 1-3 , persons skilled in the art will understand that the method in FIG. 5 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs.
- method 500 begins at step 501 , in which resource utilization monitor 142 A determines that a utilization in a particular cluster 120 is higher than a maximum threshold value, where the utilization is substantially similar to that described above in step 401 of method 400 , and measured as described. It is noted that because a utilization associated with the particular cluster 120 is greater than the maximum threshold value, insufficient computing resources are employed in the particular cluster 120 than are required to provide failover capacity and/or to efficiently execute the VMs 122 currently running on the particular cluster 120 . Consequently, VM management server 140 increases the number of hosts 121 that are currently logically included in the particular cluster 120 via the subsequent steps of method 500 .
- step 502 in response to the determination of step 501 , host provisioning module 143 selects a host from available host pool 130 .
- step 503 host provisioning module 143 prepares the host selected in step 502 for use in the particular cluster 120 .
- step 503 includes imaging a hypervisor 213 on the selected host and configuring networking connections between the selected host and components of cluster 120 , for example, via the HBA and/or NICs of the selected available host 131 .
- the selected host is provided permissions for accessing object store 230 and the various local storage resources of the particular cluster 120 , such as SSDs 226 and/or magnetic disks 227 .
- host provisioning module 143 logically adds the selected host to the particular cluster 120 .
- a cluster membership data structure associated with the particular cluster 120 is modified with a unique host identifier for the selected host being added.
- VSAN 215 is notified of the new physical storage addresses associated with the newly added host, since these new physical storage addresses are used to back a portion of object store 230 . Thus, VSAN 215 is informed of the addition of the selected host.
- FIG. 6 sets forth a flowchart of method steps carried out by VM management server 140 in response to a failure of a host in a particular cluster 120 , according to one or more embodiments.
- the method steps in FIG. 6 are described in conjunction with computing environment 100 of FIGS. 1-3 , persons skilled in the art will understand that the method in FIG. 6 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs.
- method 600 begins at step 601 , in which failure detection module 141 A determines that a host 121 included in a particular cluster 120 has experienced a failure.
- host 121 may experience a software crash or hardware fault that prevents continued operation of host 121 .
- step 602 in response to determining that host 121 in the particular cluster 120 has experienced the crash or failure, VM management server 140 determines whether the particular cluster 120 currently includes sufficient operable hosts 121 for proper operation of cluster 120 . If yes, method 600 proceeds to step 610 and terminates; if no, method 600 proceeds to step 603 . Alternatively, optional step 602 is skipped and method 600 proceeds directly from step 601 to step 603 .
- step 602 host provisioning module 143 determines whether the particular cluster 120 currently includes sufficient operable hosts 121 based on whether the detected failure of the host 121 results in the total number of operable hosts 121 in the particular cluster 120 to be less than a predetermined minimum threshold number of hosts 121 . For example, to provide sufficient failover capacity and/or redundancy in the particular cluster 120 , a minimum of four operable hosts 120 may be in effect. Alternatively or additionally, in some embodiments, in step 602 VM management server 140 determines whether the particular cluster 120 currently includes sufficient operable hosts 121 based on the availability of the VMs 122 currently executing on the hosts 121 of the particular cluster 120 .
- VM management server 140 may determine whether the availability of the VMs 122 currently executing on the hosts 121 of the particular cluster 120 is greater than a minimum requirement or target. In such embodiments, VM management server 140 may determine availability based on whether there are sufficient computing resources for these VMs to execute with an acceptable latency, such as CPU processing time, available memory, and the like.
- step 603 in response to the determination of step 601 , host provisioning module 143 selects a host from available host pool 130 .
- step 604 host provisioning module 143 prepares the host selected in step 603 for use in the particular cluster 120 .
- step 604 in method 600 may be substantially similar to step 503 in method 500 .
- step 605 host provisioning module 143 logically adds the selected host 131 to the particular cluster 120 .
- step 605 in method 600 may be substantially similar to step 504 in method 500 .
- implementation of method 600 enables the number of operable hosts 121 that are currently logically included in a particular cluster 120 to be increased when insufficient computing resources are employed in the particular cluster 120 as a result of a host failure.
- FIG. 7 sets forth a flowchart of method steps carried out by VM management server 140 in response to a partial failure of a host in a particular cluster 120 , according to one or more embodiments.
- the method steps in FIG. 7 are described in conjunction with computing environment 100 of FIGS. 1-3 , persons skilled in the art will understand that the method in FIG. 7 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs.
- method 700 begins at step 701 , in which failure detection module 141 A determines that a host 121 included in a particular cluster 120 has experienced a partial failure, and is compromised.
- the host 121 may experience the failure of a component that allows continued operation of the host 121 , such as a partial memory failure, a fan failure, the failure of a single disk of a multi-disk drive, the failure of a single magnetic disk drive in a host 121 that includes multiple magnetic disk drives, and the like.
- Such a host 121 is referred to hereinafter as a “compromised host.” According to some embodiments, because operation of the compromised host is at least partially compromised, the compromised host is subsequently replaced in the particular cluster 120 with an available host 131 via the subsequent steps of method 700 .
- step 702 in response to determining that a host 121 in the particular cluster 120 has experienced the partial failure, host provisioning module 143 determines whether the particular cluster 120 currently includes sufficient operable hosts 121 for proper operation of cluster 120 . If yes, method 700 proceeds directly to step 706 ; if no, method 700 proceeds to step 703 . Alternatively, optional step 702 is skipped and method 700 proceeds directly from step 701 to step 703 .
- step 703 host provisioning module 143 selects a host from available host pool 130 .
- step 704 host provisioning module 143 prepares the host selected in step 703 for use in the particular cluster 120 .
- step 704 in method 700 may be substantially similar to step 503 in method 500 .
- step 705 host provisioning module 143 logically adds the selected host 131 to the particular cluster 120 .
- step 705 in method 700 may be substantially similar to step 504 in method 500 .
- step 706 host provisioning module 143 copies data that are stored, as part of object store 230 , in local storage resources housed in the compromised host. The data are copied to other local storage resources housed in one or more other hosts 121 in the particular cluster 120 , and/or to the newly added available host 131 .
- VMs executing in the particular cluster 120 no longer access the local storage resources housed in the compromised host, since all file system objects associated with VSAN 215 are stored elsewhere within the particular cluster 120 .
- HA module 141 triggers migration of VMs (performed by migration module 144 ) from the compromised host to other hosts 121 in the particular cluster 120 .
- some or all VMs executing on the compromised host are migrated to the newly added available host 131 .
- the VMs executing on the compromised host are instead distributed among the other hosts 121 in the particular cluster 120 .
- Techniques for load-balancing between hosts 121 may be employed to complete migration of VMs from the compromised host in step 707 .
- implementation of method 700 enables the number of operable hosts 121 that are currently logically included in a particular cluster 120 to be increased when insufficient computing resources are employed in the particular cluster 120 as a result of a partial host failure.
- Certain embodiments as described above involve a hardware abstraction layer on top of a host computer.
- the hardware abstraction layer allows multiple contexts or virtual computing instances to share the hardware resource.
- these virtual computing instances are isolated from each other, each having at least a user application running therein.
- the hardware abstraction layer thus provides benefits of resource isolation and allocation among the virtual computing instances.
- virtual machines are used as an example for the virtual computing instances and hypervisors as an example for the hardware abstraction layer.
- each virtual machine includes a guest operating system in which at least one application runs. It should be noted that these embodiments may also apply to other examples of virtual computing instances, such as containers not including a guest operating system, referred to herein as “OS-less containers” (see, e.g., docker.com).
- OS-less containers implement operating system-level virtualization, wherein an abstraction layer is provided on top of the kernel of an operating system on a host computer.
- the abstraction layer supports multiple OS-less containers each including an application and its dependencies.
- Each OS-less container runs as an isolated process in user space on the host operating system and shares the kernel with other containers.
- the OS-less container relies on the kernel's functionality to make use of resource isolation (CPU, memory, block I/O, network, etc.) and separate namespaces and to completely isolate the application's view of the operating environments.
- resource isolation CPU, memory, block I/O, network, etc.
- By using OS-less containers resources can be isolated, services restricted, and processes provisioned to have a private view of the operating system with their own process ID space, file system structure, and network interfaces.
- Multiple containers can share the same kernel, but each container can be constrained to only use a defined amount of resources such as CPU, memory and I/O.
- the various embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities—usually, though not necessarily, these quantities may take the form of electrical or magnetic signals, where they or representations of them are capable of being stored, transferred, combined, compared, or otherwise manipulated. Further, such manipulations are often referred to in terms, such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments of the invention may be useful machine operations.
- one or more embodiments of the invention also relate to a device or an apparatus for performing these operations.
- the apparatus may be specially constructed for specific required purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer.
- various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
- One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer readable media.
- the term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system—computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer.
- Examples of a computer readable medium include a hard drive, network attached storage (NAS), read-only memory, random-access memory (e.g., a flash memory device), a CD (Compact Discs)-CD-ROM, a CD-R, or a CD-RW, a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices.
- the computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
- Virtualization systems in accordance with the various embodiments may be implemented as hosted embodiments, non-hosted embodiments or as embodiments that tend to blur distinctions between the two, are all envisioned.
- various virtualization operations may be wholly or partially implemented in hardware.
- a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.
- the virtualization software can therefore include components of a host, console, or guest operating system that performs virtualization functions.
- Plural instances may be provided for components, operations or structures described herein as a single instance.
- boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s).
- structures and functionality presented as separate components in exemplary configurations may be implemented as a combined structure or component.
- structures and functionality presented as a single component may be implemented as separate components.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Hardware Redundancy (AREA)
Abstract
A number of hosts in a logical cluster is adjusted up or down in an elastic manner by tracking membership of hosts in the cluster using a first data structure and tracking membership of hosts in a spare pool using a second data structure, and upon determining that a triggering condition for adding another host is met and that all hosts in the cluster are being used, selecting a host from the spare pool, and programmatically adding an identifier of the selected host to the first data structure and programmatically deleting the identifier of the selected host from the second data structure.
Description
- This application is a division of and claims the benefit of U.S. patent application Ser. No. 15/368,381, entitled “System and Method for Managing Size of Clusters in a Computing Environment,” and filed Dec. 2, 2016, which is hereby incorporated by reference in its entirety.
- A virtual machine (VM) is a software abstraction of a physical computing system capable of running one or more applications under the control of a guest operating system, where the guest operating system interacts with an emulated hardware platform, also referred to as a virtual hardware platform. One or multiple VMs and a virtual hardware platform are executed on a physical host device, such as a server-class computer. VMs are frequently employed in data centers, cloud computing platforms, and other distributed computing systems, and are executed on the physical host devices of such systems. Typically, these host devices are logically grouped or “clustered” together as a single logical construct. Thus, the aggregated computing and memory resources of the cluster that are available for running VMs can be provisioned flexibly and dynamically to the various VMs being executed.
- However, there are also drawbacks to organizing host devices in clusters when executing VMs. For example, when cluster utilization is nearly full, i.e., when computing, memory, and/or networking resources of a cluster are fully utilized, VM availability can be compromised and VM latency increased significantly. While the performance of VMs in a cluster with high utilization can be improved by a system administrator manually adding host devices to the cluster and/or migrating VMs across clusters (e.g., to a less utilized cluster), such customizations are generally not scalable across the plurality of clusters included in a typical distributed computing environment and require VMs to be powered down. Further, performing such manual customizations in real time in response to dynamic workloads in a cluster is generally impracticable. Instead, manual customization of clusters is typically performed on a periodic basis, e.g., daily or weekly.
- In addition, to maximize VM availability, clusters of host devices often include reserved failover capacity, i.e., host devices in the cluster that remain idle during normal operation and are therefore available for executing VMs whenever a host device in the cluster fails. Such reserved failover capacity can make up a significant portion of the resources of a cluster, but are infrequently utilized. For example, for a distributed computing system that includes 50 clusters, where each cluster includes 10 host devices and has a failover capacity of 20%, then the capacity equivalent to 100 host devices are unused in the system until a failure occurs. Because failures are relatively infrequent, the majority of this reserved failover capacity is infrequently utilized, thereby incurring both capital and operational costs for little benefit.
- According to embodiments, a number of hosts in a logical cluster is adjusted up or down in an elastic manner. A method of adjusting the number of hosts in the cluster, according to an embodiment, includes the steps of tracking membership of hosts in the cluster using a first data structure and tracking membership of hosts in a spare pool using a second data structure, and upon determining that a triggering condition for adding another host is met and that all hosts in the cluster are being used, selecting a host from the spare pool, and programmatically adding an identifier of the selected host to the first data structure and programmatically deleting the identifier of the selected host from the second data structure.
- Further embodiments provide a non-transitory computer-readable medium that includes instructions that, when executed, enable a computer to implement one or more aspects of the above method, and a system of computers including a management server that is programmed to implement one or more aspects of the above method.
-
FIG. 1 illustrates a computing environment, according to one embodiment. -
FIG. 2 is a conceptual diagram that depicts an expanded view of the hardware platform of a computer host ofFIG. 1 , according to one or more embodiments. -
FIGS. 3A-3C are conceptual diagrams illustrating the logical removal of an available host from a spare host pool and the logical addition of the available host to one of the clusters, according to one or more embodiments. -
FIG. 4 sets forth a flowchart of method steps carried out by the VM management server ofFIG. 1 to address low utilization in a cluster, according to one or more embodiments. -
FIG. 5 sets forth a flowchart of method steps carried out by the VM management server ofFIG. 1 to address high utilization in a cluster, according to one or more embodiments. -
FIG. 6 sets forth a flowchart of method steps carried out by the VM management server ofFIG. 1 in response to a failure of a host in a particular cluster, according to one or more embodiments. -
FIG. 7 sets forth a flowchart of method steps carried out by the VM management server ofFIG. 1 in response to a partial failure of a host in a particular cluster, according to one or more embodiments. -
FIG. 1 illustrates acomputing environment 100, according to one embodiment.Computing environment 100 is a virtual data center, also referred to as a software-defined data center, and includes multiple clusters of host devices, or “clusters” 120, aspare host pool 130 of available physical host devices, or “hosts,” a virtual machine (VM)management server 140, and, in some embodiments, a failedhost pool 150.Computing environment 100 may include multiple virtual data centers residing within a single physical data center. Alternatively or additionally, the virtual components ofcomputing environment 100 may be distributed across multiple physical data centers or locations. - Each
cluster 120 ofcomputing environment 100 includes a plurality ofhosts 121A-121N (referred to collectively as hosts 121), each configured to execute onemore VMs 122. According to various embodiments, the number N ofhosts 121 logically included in aparticular cluster 120 can be varied from a predetermined minimum value, for example four, to a maximum implementable value, for example 64 or 128. More specifically, one ormore hosts 121 are logically added to and/or removed from aparticular cluster 120 in response to utilization of the cluster exceeding a maximum utilization threshold value, falling below a minimum utilization threshold value, and/or detection of a partial or total host failure. One embodiment ofhosts 121 is described in greater detail below in conjunction withFIG. 2 . -
Spare host pool 130 includes a plurality ofavailable hosts 131A-131M (referred to collectively as hosts 131). According to various embodiments, the number M ofhosts 131 logically included inspare host pool 130 can be varied dynamically. Specifically, asavailable hosts 131 are logically removed fromspare host pool 130 and logically added to any ofclusters 120 asadditional hosts 121, the number M ofavailable hosts 131 inspare host pool 130 decreases. Likewise, ashosts 121 are logically removed from any ofcluster 120 and logically added to sparehost pool 130 asavailable hosts 131, the number M ofavailable hosts 131 inspare host pool 130 increases. Each ofavailable hosts 131 may be substantially similar in physical configuration to hosts 121, which are described in greater detail below in conjunction withFIG. 2 . - In some embodiments, spare
host pool 130 may includeavailable hosts 131 that are provisioned to computingenvironment 100 as cloud-based resources, and consequently are located at one more physical locations and/or data centers that are remote from thehosts 121 included in one or more ofclusters 120. In other embodiments,available hosts 131 are located in the same physical location and/or data center as thehosts 121 included in one or more ofclusters 120. -
VM management server 140 implements a management plane ofcomputing environment 100 and is configured to managehosts 121 andavailable hosts 131 associated withcomputing environment 100. More specifically,VM management server 140 is configured to manage interactions betweenhosts 121, to determine whenavailable hosts 131 are added to aparticular cluster 120, and to determine whenhosts 121 are removed from aparticular cluster 120 and added to sparehost pool 130. To that end,VM management server 140 includes a high availability (HA)module 141, a distributed resource scheduler (DRS)module 142, and ahost provisioning module 143.HA module 141 andDRS module 142 are each configured for a computing environment in whichclusters 120 are elastic, i.e., eachcluster 120 can be dynamically increased and decreased in size in response to host failure and/or cluster utilization levels. -
VM management server 140 typically runs on a computer host that is separate fromhosts 121 and is accessible by a system administrator via a graphical user interface (GUI) or via a command line interface (CLI). In some embodiments,VM management server 140 may be configured to enable a system administrator to perform various management functions forcomputing system 100 in addition to the automated functionality ofHA module 141 andDRS module 142. For example,VM management server 140 may be configured to enable a system administrator to deployVMs 122 to thedifferent hosts 121 in a particular cluster, monitor the performance ofsuch VMs 122, and/or manually power up and power downVMs 122. -
HA module 141 is configured to ensure availability ofVMs 121 executing in eachcluster 120 in response to a partial or total host failure. For example, when one ofhosts 121 in aparticular cluster 120 experiences a software crash or hardware fault and is no longer operable,HA module 141 is configured to trigger migration module 144 to migrate VMs executing in the failedhost 121 toother hosts 121 within thatcluster 120. In addition, according various embodiments,HA module 141 is configured with afailure detection module 141A to monitorclusters 120 for host faults. For example,failure detection module 141A periodically polls eachhost 121 incomputing environment 100 for software crashes and/or hardware faults or evidence thereof. Alternatively or additionally,failure detection module 141A may be configured to receive fault warnings fromhosts 121 in eachcluster 120. -
DRS module 142 is configured to maintain the utilization ofhosts 121 in eachcluster 120 between a minimum utilization threshold value and a maximum utilization threshold. To that end,DRS module 142 is configured to determine whether utilization ofhosts 121 in eachcluster 120 is above the minimum utilization threshold value and less than the maximum utilization threshold. For example,resource utilization monitor 142A inDRS module 142 polls thehosts 121 in eachcluster 120 for utilization information, such as central processing unit (CPU) time, allocated memory, and the like. In addition,DRS module 142 is configured to perform conventional load balancing betweenhosts 121 within aparticular cluster 120. For example, whenresource utilization monitor 142A determines that the utilization of computing resources of afirst host 121 in aparticular cluster 120 is less than a minimum utilization threshold value,DRS module 142 triggers migration module 144 to perform migration of executingVMs 122 from more highly loadedhosts 121 to first host, which is lightly loaded. In this way,DRS module 142 maintains utilization of the computing resources ofhosts 121 in aparticular cluster 120 within an optimal range. -
Host provisioning module 143 is configured to logically add one or moreavailable hosts 131 to acluster 120 or removehosts 121 from thecluster 120 in response to a triggering event, so that utilization of computing resources ofhosts 121 and availability ofVMs 122 executing onhosts 121 are maintained within an optimal range. A triggering event may be detection of an average utilization ofhosts 121 in acluster 120 exceeding a maximum or minimum threshold value, or the detection of ahost 121 in thecluster 120 undergoing a partial or total failure. - In one embodiment,
host provisioning module 143 is configured to logically add one or moreavailable hosts 131 to aparticular cluster 120 when ahost 121 in theparticular cluster 120 suffers a partial or total failure. Afterwards,HA module 141 is able to migrate VMs executing in the failedhost 121 toother hosts 121 within thatcluster 120 and/or to the newly addedavailable host 131. For example, when a large number ofVMs 122 are executing in aparticular cluster 120 and a host failure occurs, the physical computing resources perVM 122 are reduced, i.e., CPU time, allocated memory, etc., and availability for anyparticular VM 122 in thecluster 120 is also reduced. Thus,host provisioning module 143 is configured to rectify this issue by logically adding one or moreavailable hosts 131 to thecluster 120 based not only on the detection of a host failure, but also on an availability of theVMs 122 executing in thecluster 120. Thus, in such embodiments,host provisioning module 143 may logically add one or moreavailable hosts 131 to thecluster 120 experiencing the host failure when availability for one ormore VMs 122 in thecluster 120 cannot be increased above the minimum threshold value via migration ofVMs 122 within thecluster 120. - In some embodiments,
host provisioning module 143 is configured to logically addavailable hosts 131 to aparticular cluster 120 in which the utilization of computing resources of thehosts 121 in theparticular cluster 120 is greater than a maximum utilization threshold for thecluster 120. Thus,host provisioning module 143 increases the total number hosts within the cluster when utilization is too high. Further, in some embodiments,host provisioning module 143 is configured to logically removehosts 121 from aparticular cluster 120 in which the utilization ofhosts 121 in theparticular cluster 120 is less than a minimum utilization threshold value for thecluster 120. In such embodiments,host provisioning module 143 decreases the total number of hosts within the cluster when utilization is too low. - Metrics for quantifying availability of a
particular VM 122 may include, for example, effective CPU resources available for theVM 122 of interest (in MHz or GHz), effective memory resources available for theVM 122 of interest (in kB, MB, or GB), a CPU fairness value representing the fraction of host CPU resources allocated to theVM 122 of interest (in percent), and/or a memory fairness value representing the fraction of memory resources allocated to theVM 122 of interest (in percent). How and in what situations hostprovisioning module 143 logically addsavailable hosts 131 to and/or removeshosts 121 from aparticular cluster 120 in response to host failures, low VM availability, and/or low or high host utilization is described below in conjunction withFIGS. 4-7 . - As noted above, in some embodiments,
computing environment 100 may include a failedhost pool 150.Failed host pool 150 is a logical construct with which hosts 120 that have undergone a partial or total failure can be associated.Hosts 120 that have undergone either partial or total failure are logically associated with failedhost pool 150 as failed hosts 151, so that these host devices cannot be subsequently added to acluster 120 ofcomputing environment 100. Thus, failedhost pool 150 facilitates the identification of host devices incomputing environment 100 that require maintenance, diagnostic analysis, and/or replacement. -
FIG. 2 schematically illustrates acluster 120 ofcomputing environment 100 inFIG. 1 , according to one or more embodiments. As shown,cluster 120 includes up to N hosts 121, i.e., hosts 121A, 121B, . . . , 121N, each of which is communicatively coupled toVM management server 140. In the embodiment shown inFIG. 1 ,cluster 120 is a software-based “virtual storage area network” (VSAN) environment that leverages the commodity local storage resources housed in or directly attached tohosts 121 incluster 120 to provide anaggregate object store 230 to virtual machines (VMs) 122 executing onhosts 121. Hereinafter, use of the term “housed” or “housed in” may be used to encompass both housed in or otherwise directly attached to. - Each
host 121 is a computing device in which virtualization software and a plurality of VMs122 are executing. Eachhost 121 is typically a server-class computer, although, in some embodiments, hosts 121 may include a variety of classes of computers, such as mainframe computers, desktop computers, and laptop computers. Eachhost 121 includes a virtualization layer orhypervisor 213, a storage management module (referred to herein as a “VSAN module”) 214, and ahardware platform 225 that typically includes central processing units (CPUs), random access memory (RAM), local storage resources, such as solid state drives (SSDs) 226 and/or magnetic disks 227, a host bus adapter (HBA) that enable external storage devices to be connected to thehost 120, and one or more physical NICs that enablehost 121 to communicate over a network. These physical computing resources of eachhost 121 are managed byhypervisor 213, and throughhypervisor 213, ahost 121 is able to launch and runmultiple VMs 122. - The local commodity storage resources housed in or otherwise directly attached to
hosts 121 may include combinations of SSDs 226 and/or magnetic or spinning disks 227. In certain embodiments, SSDs 226 serve as a read cache and/or write buffer in front of magnetic disks 227 to increase I/O performance.VSAN module 214 in eachhost 121 is configured to automate storage management workflows (e.g., creating objects in object store 230) and provide access to objects in object store 230 (e.g., handling I/O operations to objects in object store 230). For eachVM 122,VSAN module 214 may then create an “object” for the specified virtual disk associated with theVM 122 by backing the virtual disk with physical storage resources ofobject store 230 such as SSDs 226 and/or magnetic disks 227. For example,SSD 226A andmagnetic disk 227A ofhost 121A,SSD 226B andmagnetic disk 227B ofhost 121B, and so on may in combination be used to backobject store 230. - Each
VM 122 is a software abstraction of a physical computing system that is capable of running one or more applications under the control of a guest operating system (not shown), where the guest operating system provides various operating system services (such as central processing unit (or CPU) allocation and memory management). The guest operating system interacts with a virtual hardware platform, which is an emulated hardware platform for the corresponding VM. Virtual hardware platforms (not depicted in the figure) are implemented on aparticular host 121 byhypervisor 213, and typically comprise virtual CPUs, virtual RAM, virtual disks, and, for network communication, a virtual network interface controller (or NIC). That applications executing withinVMs 121 are executing in a virtual machine is transparent to operation of these applications. Thus, such applications may be installed in aVM 122 unchanged from how such applications would be installed on a physical computer. - Similarly, the fact that the guest operating system installed in each
VM 122 is executing on top of a virtualized hardware platform, rather than on a physical hardware platform is transparent to the guest operating system. Thus, the guest operating system may be installed in aVM 122 in the same manner that the guest operating system is installed on a physical computer. Examples of guest operating systems include the various versions of Microsoft's Windows® operating system, the Linux operating system, and Apple's Mac OS X. - Each
VM 122 executing in aparticular cluster 120 accesses computing services by interacting with the virtual hardware platform associated with theparticular cluster 120 and implemented byhypervisor 213. As shown, eachhost 121 has onehypervisor 213 executing therein. As noted above,hypervisor 213 provides a virtualization platform on whichVMs 122 execute.Hypervisor 213 enables eachVM 122 that executes under its control to access physical computing resources inhost 120. Thus, when the guest operating system of aVM 122 schedules a process to be run on a virtual CPU of the virtual hardware platform, hypervisor 213 schedules the virtual CPU of thatVM 122 to run on a physical CPU ofhost 120. In another example, an application executing in aVM 122 may require additional virtual RAM in order to execute a particular function. In this case, the application would issue a memory allocation request to the guest operating system, which would allocate an amount of virtual RAM to satisfy the request. In turn,hypervisor 213 would allocate a certain amount of physical RAM that corresponds to the allocated virtual RAM. -
Hypervisor 213 also manages the physical computing resources of thecorresponding host 120, and allocates those resources among the executingVMs 122, as well as other (i.e., non-VM) processes.Hypervisor 213 allocates physical CPU time to virtual CPUs in theVMs 122, as well as physical disk space to virtual disks for each of theVMs 122.Hypervisor 213 also enables the transmission and receipt of network traffic (through the virtual NICs) for eachVM 122. -
VSAN module 214 can, in some embodiments, be implemented as a VSAN device driver withinhypervisor 213. In such embodiments,VSAN module 214 provides access to aconceptual VSAN 215 through which an administrator can create a number of top-level “device” or namespace objects that are backed byobject store 230. In one common scenario, during creation of a device object, the administrator may specify a particular file system for the device object (referred to as “file system objects”). For example, each hypervisor 213 in eachhost 120 may, during a boot process, discover a/vsan/root node for a conceptual global namespace that is exposed byVSAN module 214. By, for example, accessing APIs exposed byVSAN module 214,hypervisor 213 can then determine the top-level file system objects (or other types of top-level device objects) currently residing inVSAN 215. When a VM 122 (or other client) attempts to access one of the file system objects inVSAN 215,hypervisor 213 may dynamically “auto-mount” the file system object at that time. EachVSAN module 214 communicates withother VSAN modules 214 ofother hosts 121 incluster 120 to create and maintain an in-memory metadata database that contains metadata describing the locations, configurations, policies, and relationships among the various objects stored inobject store 230. Thus, in eachhost 121, such an in-memory metadata database is maintained separately but in synchronized fashion in the memory of eachhost 121. - In the embodiment illustrated in
FIG. 2 , hosts 121 incluster 120 are connected to the same shared storage formed from aggregated local storage resources, i.e.,object store 230, via a VSAN, i.e.,VSAN 215. Alternatively,cluster 120 may be configured to connecthosts 121 to shared storage using other approaches. For example, in one embodiment, the shared storage may be provided through a storage area network (SAN), and access to the SAN is provided by the HBA in eachhost 121. In another embodiment, the shared storage may be provided by a network attached storage (NAS) which is accessed through the NIC in eachhost 121. -
FIGS. 3A-3C are conceptual diagrams illustrating the logical removal of anavailable host 131 fromspare host pool 130 and the logical addition of theavailable host 131 to one of theclusters 120, according to one or more embodiments. In one embodiment,host provisioning module 143 maintains a table for each cluster to track which hosts belong to which cluster. When a host is logically added to a cluster, the host identifier (ID) is added to the table for that cluster, and when a host is logically removed from a cluster, the host ID is deleted from the table. -
FIG. 3A illustratesspare host pool 130, failedhost pool 150, and one ofclusters 120 ofcomputing environment 100 prior to a triggering event that results in a change in the number ofhosts 121 logically included incluster 120. As shown,spare host pool 131 includes a plurality ofavailable hosts 131, for example, five,cluster 120 includesmultiple hosts 121, for example three, and failedhost pool 150 includes one failedhost 151. -
FIG. 3B illustratesspare host pool 130, failedhost pool 150, and one ofclusters 120 ofcomputing environment 100 after a triggering event is detected. In some embodiments, such a triggering event may be the detection of an average utilization ofhosts 121 incluster 120 exceeding a maximum threshold value or an availability of a particular VM 122 (or group of VMs 122) falling below a minimum allowable threshold value. In another embodiment, such a triggering event may be the detection of ahost 121 undergoing a partial or total failure. Partial failures of ahost 121 that may be considered a triggering event include the failure of a component of the host that allows continued operation of thehost 121, such as a partial memory failure, a fan failure, the failure of a single disk of a multi-disk drive, the failure of a single magnetic disk drive in ahost 121 that includes multiple magnetic disk drives, etc. Total failures of ahost 121 that may be considered a triggering event include a software-related host freeze, complete failure of the only magnetic disk drive in the host, and any other faults or failures that prevent the continued operation of thehost 121. - In response to the above-described triggering event,
host provisioning module 143 adds anavailable host 131 to cluster 120 as anew host 321, as shown inFIG. 3B . For example, in some situations, when a failed host 322 is detected,HA module 141 may not be able to maintain an availability of aparticular VM 122 or group ofVMs 122 if the VMs were to be migrated from failedhost 321 todifferent hosts 121 withincluster 120. In such a scenario,host provisioning module 143 increases the number ofhosts 121 in thecluster 120 by logically adding anavailable host 131 fromspare host pool 130, as shown. In addition,host provisioning module 143 logically removes failedhost 321 fromcluster 120 and logically adds failedhost 321 to failedhost pool 150, as shown. In another example, whenDRS module 142 cannot maintain a utilization of allhosts 121 in aparticular cluster 120 below a maximum allowable utilization by performing load balancing ofhosts 121 within thecluster 120,host provisioning module 143 increases the number ofhosts 121 in thecluster 120 by adding anavailable host 131 fromspare host pool 130. -
FIG. 3C illustratesspare host pool 130, failedhost pool 150, and one ofclusters 120 ofcomputing environment 100 after the number ofhosts 120 incluster 120 has been increased, for example in response to a failure of ahost 121 incluster 120. As shown,spare host pool 131 includes one feweravailable hosts 131, for example, four,cluster 120 includes the same number ofhosts 121, for example three, and failedhost pool 150 includes an additional failedhost 151, for example two. -
FIG. 4 sets forth a flowchart of method steps carried out byVM management server 140 to address low utilization in a cluster, according to one or more embodiments. Although the method steps inFIG. 4 are described in conjunction withcomputing environment 100 ofFIGS. 1-3 , persons skilled in the art will understand that the method inFIG. 4 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs. - As shown, a
method 400 begins atstep 401, in whichresource utilization monitor 142A ofVM management server 140 determines that a utilization in aparticular cluster 120 is less than a minimum threshold value. For example, in some embodiments, the utilization may be a utilization of aspecific host 121 in theparticular cluster 120. Alternatively, the utilization may be a utilization associated with a group ofhosts 121 in theparticular cluster 120, or of allhosts 121 in theparticular cluster 120, such as an average utilization thereof. Instep 401, the utilization is typically measured or quantified via performance monitoring functions included inVM management server 140, and may be quantified in terms of computing resources in use by a host or hosts in the particular cluster, such as percentage utilization of CPU, RAM, and the like. - It is noted that because a utilization associated with the
particular cluster 120 is less than the minimum threshold value, more computing resources are employed in theparticular cluster 120 than are required to efficiently execute theVMs 122 currently running on theparticular cluster 120. Consequently,VM management server 140 reduces the current number ofhosts 121 that are logically included in theparticular cluster 120 via the subsequent steps ofmethod 400. - In
step 402, in response to the determination ofstep 401,host provisioning module 143VM management server 140 selects ahost 121 in theparticular cluster 120 to be logically removed therefrom. In some embodiments, the selected host is thehost 121 in theparticular cluster 120 with the highest utilization, thereby maximizing the impact on utilization in theparticular cluster 120. In some embodiments, once the selected host instep 402 is selected, additional write I/O's from VMs executing onhosts 121 to the local storage resources included in the selected host are not allowed, so that data stored locally on the selected host can be moved to the remaininghosts 121. By contrast, in such embodiments, read I/O's are still permitted to the selected host, so thatVMs 122 executing onhosts 121 can accessobject store 230 as needed. - In
step 403,DRS module 142 triggers migration of VMs (performed by migration module 144) from the selected host toother hosts 121 in the particular cluster. Techniques for load-balancing betweenhosts 121 that are well-known in the art may be employed to complete migration of VMs from the selected host instep 403. - In
step 404,DRS module 142 copies data that are stored, as part ofobject store 230, in local storage resources housed in the selected host. The data are copied to other local storage resources housed in one or moreother hosts 121 in theparticular cluster 120. Upon completion ofstep 404, VMs executing in theparticular cluster 120 no longer access the local storage resources housed in the selected host, since all file system objects associated withVSAN 215 are stored elsewhere within theparticular cluster 120. - In
step 405,host provisioning module 143 logically removes the selected host from theparticular cluster 120. In step 406,host provisioning module 143 logically adds the selected host toavailable host pool 130 as an additionalavailable host 131. -
FIG. 5 sets forth a flowchart of method steps carried out byVM management server 140 to address high utilization in a cluster, according to one or more embodiments. Although the method steps inFIG. 5 are described in conjunction withcomputing environment 100 ofFIGS. 1-3 , persons skilled in the art will understand that the method inFIG. 5 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs. - As shown,
method 500 begins atstep 501, in which resource utilization monitor 142A determines that a utilization in aparticular cluster 120 is higher than a maximum threshold value, where the utilization is substantially similar to that described above instep 401 ofmethod 400, and measured as described. It is noted that because a utilization associated with theparticular cluster 120 is greater than the maximum threshold value, insufficient computing resources are employed in theparticular cluster 120 than are required to provide failover capacity and/or to efficiently execute theVMs 122 currently running on theparticular cluster 120. Consequently,VM management server 140 increases the number ofhosts 121 that are currently logically included in theparticular cluster 120 via the subsequent steps ofmethod 500. - In
step 502, in response to the determination ofstep 501,host provisioning module 143 selects a host fromavailable host pool 130. - In
step 503,host provisioning module 143 prepares the host selected instep 502 for use in theparticular cluster 120. In some embodiments,step 503 includes imaging ahypervisor 213 on the selected host and configuring networking connections between the selected host and components ofcluster 120, for example, via the HBA and/or NICs of the selectedavailable host 131. In such embodiments, the selected host is provided permissions for accessingobject store 230 and the various local storage resources of theparticular cluster 120, such as SSDs 226 and/or magnetic disks 227. - In
step 504,host provisioning module 143 logically adds the selected host to theparticular cluster 120. For example, in some embodiments, a cluster membership data structure associated with theparticular cluster 120 is modified with a unique host identifier for the selected host being added. In addition,VSAN 215 is notified of the new physical storage addresses associated with the newly added host, since these new physical storage addresses are used to back a portion ofobject store 230. Thus,VSAN 215 is informed of the addition of the selected host. -
FIG. 6 sets forth a flowchart of method steps carried out byVM management server 140 in response to a failure of a host in aparticular cluster 120, according to one or more embodiments. Although the method steps inFIG. 6 are described in conjunction withcomputing environment 100 ofFIGS. 1-3 , persons skilled in the art will understand that the method inFIG. 6 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs. - As shown,
method 600 begins atstep 601, in whichfailure detection module 141A determines that ahost 121 included in aparticular cluster 120 has experienced a failure. For example, host 121 may experience a software crash or hardware fault that prevents continued operation ofhost 121. - In
optional step 602, in response to determining thathost 121 in theparticular cluster 120 has experienced the crash or failure,VM management server 140 determines whether theparticular cluster 120 currently includes sufficientoperable hosts 121 for proper operation ofcluster 120. If yes,method 600 proceeds to step 610 and terminates; if no,method 600 proceeds to step 603. Alternatively,optional step 602 is skipped andmethod 600 proceeds directly fromstep 601 to step 603. - In some embodiments, in
step 602host provisioning module 143 determines whether theparticular cluster 120 currently includes sufficientoperable hosts 121 based on whether the detected failure of thehost 121 results in the total number ofoperable hosts 121 in theparticular cluster 120 to be less than a predetermined minimum threshold number ofhosts 121. For example, to provide sufficient failover capacity and/or redundancy in theparticular cluster 120, a minimum of fouroperable hosts 120 may be in effect. Alternatively or additionally, in some embodiments, instep 602VM management server 140 determines whether theparticular cluster 120 currently includes sufficientoperable hosts 121 based on the availability of theVMs 122 currently executing on thehosts 121 of theparticular cluster 120. That is,VM management server 140 may determine whether the availability of theVMs 122 currently executing on thehosts 121 of theparticular cluster 120 is greater than a minimum requirement or target. In such embodiments,VM management server 140 may determine availability based on whether there are sufficient computing resources for these VMs to execute with an acceptable latency, such as CPU processing time, available memory, and the like. - In
step 603, in response to the determination ofstep 601,host provisioning module 143 selects a host fromavailable host pool 130. - In
step 604,host provisioning module 143 prepares the host selected instep 603 for use in theparticular cluster 120. Generally,step 604 inmethod 600 may be substantially similar to step 503 inmethod 500. - In
step 605,host provisioning module 143 logically adds the selectedhost 131 to theparticular cluster 120. Generally,step 605 inmethod 600 may be substantially similar to step 504 inmethod 500. - Thus, implementation of
method 600 enables the number ofoperable hosts 121 that are currently logically included in aparticular cluster 120 to be increased when insufficient computing resources are employed in theparticular cluster 120 as a result of a host failure. -
FIG. 7 sets forth a flowchart of method steps carried out byVM management server 140 in response to a partial failure of a host in aparticular cluster 120, according to one or more embodiments. Although the method steps inFIG. 7 are described in conjunction withcomputing environment 100 ofFIGS. 1-3 , persons skilled in the art will understand that the method inFIG. 7 may also be performed with other types of computing systems, for example, any distributed computing system that includes a cluster of host devices executing VMs. - As shown,
method 700 begins atstep 701, in whichfailure detection module 141A determines that ahost 121 included in aparticular cluster 120 has experienced a partial failure, and is compromised. For example, thehost 121 may experience the failure of a component that allows continued operation of thehost 121, such as a partial memory failure, a fan failure, the failure of a single disk of a multi-disk drive, the failure of a single magnetic disk drive in ahost 121 that includes multiple magnetic disk drives, and the like. Such ahost 121 is referred to hereinafter as a “compromised host.” According to some embodiments, because operation of the compromised host is at least partially compromised, the compromised host is subsequently replaced in theparticular cluster 120 with anavailable host 131 via the subsequent steps ofmethod 700. - In
optional step 702, in response to determining that ahost 121 in theparticular cluster 120 has experienced the partial failure,host provisioning module 143 determines whether theparticular cluster 120 currently includes sufficientoperable hosts 121 for proper operation ofcluster 120. If yes,method 700 proceeds directly to step 706; if no,method 700 proceeds to step 703. Alternatively,optional step 702 is skipped andmethod 700 proceeds directly fromstep 701 to step 703. - In
step 703,host provisioning module 143 selects a host fromavailable host pool 130. - In
step 704,host provisioning module 143 prepares the host selected instep 703 for use in theparticular cluster 120. Generally,step 704 inmethod 700 may be substantially similar to step 503 inmethod 500. - In
step 705,host provisioning module 143 logically adds the selectedhost 131 to theparticular cluster 120. Generally,step 705 inmethod 700 may be substantially similar to step 504 inmethod 500. - In
step 706,host provisioning module 143 copies data that are stored, as part ofobject store 230, in local storage resources housed in the compromised host. The data are copied to other local storage resources housed in one or moreother hosts 121 in theparticular cluster 120, and/or to the newly addedavailable host 131. Upon completion ofstep 705, VMs executing in theparticular cluster 120 no longer access the local storage resources housed in the compromised host, since all file system objects associated withVSAN 215 are stored elsewhere within theparticular cluster 120. - In
step 707,HA module 141 triggers migration of VMs (performed by migration module 144) from the compromised host toother hosts 121 in theparticular cluster 120. In embodiments in which anavailable host 131 is selected instep 703, some or all VMs executing on the compromised host are migrated to the newly addedavailable host 131. Alternatively, the VMs executing on the compromised host are instead distributed among theother hosts 121 in theparticular cluster 120. Techniques for load-balancing betweenhosts 121 that are well-known in the art may be employed to complete migration of VMs from the compromised host instep 707. - Thus, implementation of
method 700 enables the number ofoperable hosts 121 that are currently logically included in aparticular cluster 120 to be increased when insufficient computing resources are employed in theparticular cluster 120 as a result of a partial host failure. - Certain embodiments as described above involve a hardware abstraction layer on top of a host computer. The hardware abstraction layer allows multiple contexts or virtual computing instances to share the hardware resource. In one embodiment, these virtual computing instances are isolated from each other, each having at least a user application running therein. The hardware abstraction layer thus provides benefits of resource isolation and allocation among the virtual computing instances. In the foregoing embodiments, virtual machines are used as an example for the virtual computing instances and hypervisors as an example for the hardware abstraction layer. As described above, each virtual machine includes a guest operating system in which at least one application runs. It should be noted that these embodiments may also apply to other examples of virtual computing instances, such as containers not including a guest operating system, referred to herein as “OS-less containers” (see, e.g., docker.com). OS-less containers implement operating system-level virtualization, wherein an abstraction layer is provided on top of the kernel of an operating system on a host computer. The abstraction layer supports multiple OS-less containers each including an application and its dependencies. Each OS-less container runs as an isolated process in user space on the host operating system and shares the kernel with other containers. The OS-less container relies on the kernel's functionality to make use of resource isolation (CPU, memory, block I/O, network, etc.) and separate namespaces and to completely isolate the application's view of the operating environments. By using OS-less containers, resources can be isolated, services restricted, and processes provisioned to have a private view of the operating system with their own process ID space, file system structure, and network interfaces. Multiple containers can share the same kernel, but each container can be constrained to only use a defined amount of resources such as CPU, memory and I/O.
- The various embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities—usually, though not necessarily, these quantities may take the form of electrical or magnetic signals, where they or representations of them are capable of being stored, transferred, combined, compared, or otherwise manipulated. Further, such manipulations are often referred to in terms, such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments of the invention may be useful machine operations. In addition, one or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for specific required purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
- The various embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like.
- One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system—computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer. Examples of a computer readable medium include a hard drive, network attached storage (NAS), read-only memory, random-access memory (e.g., a flash memory device), a CD (Compact Discs)-CD-ROM, a CD-R, or a CD-RW, a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices. The computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
- Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein, but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation, unless explicitly stated in the claims.
- Virtualization systems in accordance with the various embodiments may be implemented as hosted embodiments, non-hosted embodiments or as embodiments that tend to blur distinctions between the two, are all envisioned. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.
- Many variations, modifications, additions, and improvements are possible, regardless the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest operating system that performs virtualization functions. Plural instances may be provided for components, operations or structures described herein as a single instance. Finally, boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s). In general, structures and functionality presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements may fall within the scope of the appended claim(s).
Claims (20)
1. In a data center comprising a cluster of hosts, a spare host pool, and a failed host pool, a method of adjusting the number of hosts in the cluster, comprising:
tracking membership of hosts in the cluster using a first data structure;
tracking membership of hosts in the spare host pool using a second data structure;
tracking membership of hosts in the failed host pool using a third data structure;
determining that the cluster does not include a sufficient number of operable hosts based on determining that an availability of one or more virtual machines (VMs) on a first host in the cluster;
selecting a second host from the spare host pool;
adding an identifier of the selected second host to the first data structure;
migrating the VM from the first host to the second host;
adding an identifier of the first host to the third data structure; and
deleting the identifier of the first host from the first data structure.
2. The method of claim 1 , wherein determining that the cluster does not include a sufficient number of operable hosts is based on determining the existence of a partial or total failure of the first host in the cluster.
3. The method of claim 1 , wherein the VM is migrated from the first host to the second host upon a failure of the first host.
4. The method of claim 1 , further comprising:
copying data locally stored in the first host to a storage device accessible by the cluster.
5. The method of claim 1 , wherein the VM is migrated from the first host to the second host when an average resource utilization in the first host is greater than an upper threshold utilization.
6. The method of claim 5 , wherein the resource is CPU or memory.
7. The method of claim 6 , further comprising:
copying data locally stored in a host with the lowest resource utilization to a storage device accessible by the cluster.
8. A non-transitory computer-readable medium comprising instructions that are executable in a computing device to cause the computing device to at least:
track membership of hosts in the cluster using a first data structure;
track membership of hosts in the spare host pool using a second data structure;
track membership of hosts in the failed host pool using a third data structure;
determine that the cluster does not include a sufficient number of operable hosts based on determining that an availability of one or more virtual machines (VMs) on a first host in the cluster;
select a second host from the spare host pool;
add an identifier of the selected second host to the first data structure;
migrate the VM from the first host to the second host;
add an identifier of the first host to the third data structure; and
delete the identifier of the first host from the first data structure.
9. The non-transitory computer-readable medium of claim 8 , wherein determining that the cluster does not include a sufficient number of operable hosts is based on determining the existence of a partial or total failure of the first host in the cluster.
10. The non-transitory computer-readable medium of claim 8 , wherein the VM is migrated from the first host to the second host upon a failure of the first host.
11. The non-transitory computer-readable medium of claim 8 , wherein the instructions further cause the computing device to copy data locally stored in the first host to a storage device accessible by the cluster.
12. The non-transitory computer-readable medium of claim 8 , wherein the VM is migrated from the first host to the second host when an average resource utilization in the first host is greater than an upper threshold utilization.
13. The non-transitory computer-readable medium of claim 12 , wherein the resource is CPU or memory.
14. The non-transitory computer-readable medium of claim 13 , wherein the instructions cause the computing device to copy data locally stored in a host with the lowest resource utilization to a storage device accessible by the cluster.
15. A system of computers, comprising:
a cluster of hosts;
a spare pool of hosts; and
a management server configured to:
track membership of hosts in the cluster using a first data structure;
track membership of hosts in the spare host pool using a second data structure;
track membership of hosts in the failed host pool using a third data structure;
determine that the cluster does not include a sufficient number of operable hosts based on determining that an availability of one or more virtual machines (VMs) on a first host in the cluster;
select a second host from the spare host pool;
add an identifier of the selected second host to the first data structure;
migrate the VM from the first host to the second host;
add an identifier of the first host to the third data structure; and
delete the identifier of the first host from the first data structure.
16. The system of claim 15 , wherein determining that the cluster does not include a sufficient number of operable hosts is based on determining the existence of a partial or total failure of the first host in the cluster.
17. The system of claim 15 , wherein the VM is migrated from the first host to the second host upon a failure of the first host.
18. The system of claim 15 , further comprising copying data locally stored in the first host to a storage device accessible by the cluster.
19. The system of claim 15 , wherein the VM is migrated from the first host to the second host when an average resource utilization in the first host is greater than an upper threshold utilization.
20. The system of claim 19 , wherein the resource is CPU or memory.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/504,416 US20190332415A1 (en) | 2016-12-02 | 2019-07-08 | System and Method for Managing Size of Clusters in a Computing Environment |
US17/647,689 US20220129299A1 (en) | 2016-12-02 | 2022-01-11 | System and Method for Managing Size of Clusters in a Computing Environment |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/368,381 US10346191B2 (en) | 2016-12-02 | 2016-12-02 | System and method for managing size of clusters in a computing environment |
US16/504,416 US20190332415A1 (en) | 2016-12-02 | 2019-07-08 | System and Method for Managing Size of Clusters in a Computing Environment |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/368,381 Continuation US10346191B2 (en) | 2016-12-02 | 2016-12-02 | System and method for managing size of clusters in a computing environment |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/647,689 Continuation US20220129299A1 (en) | 2016-12-02 | 2022-01-11 | System and Method for Managing Size of Clusters in a Computing Environment |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190332415A1 true US20190332415A1 (en) | 2019-10-31 |
Family
ID=62240010
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/368,381 Active 2037-08-18 US10346191B2 (en) | 2016-12-02 | 2016-12-02 | System and method for managing size of clusters in a computing environment |
US16/504,416 Abandoned US20190332415A1 (en) | 2016-12-02 | 2019-07-08 | System and Method for Managing Size of Clusters in a Computing Environment |
US17/647,689 Pending US20220129299A1 (en) | 2016-12-02 | 2022-01-11 | System and Method for Managing Size of Clusters in a Computing Environment |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/368,381 Active 2037-08-18 US10346191B2 (en) | 2016-12-02 | 2016-12-02 | System and method for managing size of clusters in a computing environment |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/647,689 Pending US20220129299A1 (en) | 2016-12-02 | 2022-01-11 | System and Method for Managing Size of Clusters in a Computing Environment |
Country Status (1)
Country | Link |
---|---|
US (3) | US10346191B2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023177703A1 (en) * | 2022-03-15 | 2023-09-21 | Liveperson, Inc. | Methods and systems for dynamic load balancing of processing resources in distributed environments |
US11875190B2 (en) | 2022-03-15 | 2024-01-16 | Liveperson, Inc. | Methods and systems for AI-based load balancing of processing resources in distributed environments |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10581687B2 (en) | 2013-09-26 | 2020-03-03 | Appformix Inc. | Real-time cloud-infrastructure policy implementation and management |
US10291472B2 (en) | 2015-07-29 | 2019-05-14 | AppFormix, Inc. | Assessment of operational states of a computing environment |
US11487585B1 (en) * | 2016-12-14 | 2022-11-01 | Xilinx, Inc. | Dynamic load balancing and configuration management for heterogeneous compute accelerators in a data center |
US20180260262A1 (en) * | 2017-03-07 | 2018-09-13 | Microsoft Technology Licensing, Llc | Availability management interfaces in a distributed computing system |
US10868742B2 (en) | 2017-03-29 | 2020-12-15 | Juniper Networks, Inc. | Multi-cluster dashboard for distributed virtualization infrastructure element monitoring and policy control |
US11068314B2 (en) * | 2017-03-29 | 2021-07-20 | Juniper Networks, Inc. | Micro-level monitoring, visibility and control of shared resources internal to a processor of a host machine for a virtual environment |
US11323327B1 (en) | 2017-04-19 | 2022-05-03 | Juniper Networks, Inc. | Virtualization infrastructure element monitoring and policy control in a cloud environment using profiles |
US10353803B2 (en) * | 2017-08-21 | 2019-07-16 | Facebook, Inc. | Dynamic device clustering |
CN111104250B (en) * | 2018-10-26 | 2023-10-10 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer readable medium for data processing |
US11593234B2 (en) * | 2020-01-16 | 2023-02-28 | Vmware, Inc. | Cloud restart for VM failover and capacity management |
US11347521B2 (en) * | 2020-01-16 | 2022-05-31 | Vmware, Inc. | Cloud restart for non-critical performance virtual machines |
US20230025538A1 (en) * | 2021-07-26 | 2023-01-26 | Dell Products L.P. | Secure vsan cluster using device authentication and integrity measurments |
Family Cites Families (110)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020042830A1 (en) * | 2000-03-31 | 2002-04-11 | Subhra Bose | System, method and applications real-time messaging over HTTP-based protocols |
US7039694B2 (en) * | 2000-05-02 | 2006-05-02 | Sun Microsystems, Inc. | Cluster membership monitor |
US7296268B2 (en) * | 2000-12-18 | 2007-11-13 | Microsoft Corporation | Dynamic monitor and controller of availability of a load-balancing cluster |
US7644144B1 (en) * | 2001-12-21 | 2010-01-05 | Microsoft Corporation | Methods, tools, and interfaces for the dynamic assignment of people to groups to enable enhanced communication and collaboration |
US20040015581A1 (en) * | 2002-07-22 | 2004-01-22 | Forbes Bryn B. | Dynamic deployment mechanism |
US20040039815A1 (en) * | 2002-08-20 | 2004-02-26 | Compaq Information Technologies Group, L.P. | Dynamic provisioning system for a network of computers |
US7596625B2 (en) * | 2003-01-27 | 2009-09-29 | Microsoft Corporation | Peer-to-peer grouping interfaces and methods |
US20050193113A1 (en) * | 2003-04-14 | 2005-09-01 | Fujitsu Limited | Server allocation control method |
US7197632B2 (en) * | 2003-04-29 | 2007-03-27 | International Business Machines Corporation | Storage system and cluster maintenance |
US7739541B1 (en) * | 2003-07-25 | 2010-06-15 | Symantec Operating Corporation | System and method for resolving cluster partitions in out-of-band storage virtualization environments |
US7516221B2 (en) * | 2003-08-14 | 2009-04-07 | Oracle International Corporation | Hierarchical management of the dynamic allocation of resources in a multi-node system |
US7320085B2 (en) * | 2004-03-09 | 2008-01-15 | Scaleout Software, Inc | Scalable, software-based quorum architecture |
EP1762055A4 (en) * | 2004-03-10 | 2010-03-17 | Scaleout Software Inc | Scalable, highly available cluster membership architecture |
US8156490B2 (en) * | 2004-05-08 | 2012-04-10 | International Business Machines Corporation | Dynamic migration of virtual machine computer programs upon satisfaction of conditions |
US7383405B2 (en) * | 2004-06-30 | 2008-06-03 | Microsoft Corporation | Systems and methods for voluntary migration of a virtual machine between hosts with common storage connectivity |
JP4339763B2 (en) * | 2004-09-07 | 2009-10-07 | 株式会社日立製作所 | Failover method and computer system |
US7506264B2 (en) * | 2005-04-28 | 2009-03-17 | International Business Machines Corporation | Method and apparatus for presenting navigable data center information in virtual reality using leading edge rendering engines |
US7739677B1 (en) * | 2005-05-27 | 2010-06-15 | Symantec Operating Corporation | System and method to prevent data corruption due to split brain in shared data clusters |
US7774010B2 (en) * | 2005-07-06 | 2010-08-10 | Nokia Corporation | Peer-to-peer group management framework and methodology |
US7587492B2 (en) * | 2005-07-29 | 2009-09-08 | Hewlett-Packard Development Company, L.P. | Dynamic performance management for virtual servers |
JP4823717B2 (en) * | 2006-02-28 | 2011-11-24 | 株式会社日立製作所 | Encryption communication system, terminal state management server, encryption communication method, and terminal state management method |
US8572138B2 (en) * | 2006-03-30 | 2013-10-29 | Ca, Inc. | Distributed computing system having autonomic deployment of virtual machine disk images |
US20070271560A1 (en) * | 2006-05-18 | 2007-11-22 | Microsoft Corporation | Deploying virtual machine to host based on workload characterizations |
CN101098260A (en) * | 2006-06-29 | 2008-01-02 | 国际商业机器公司 | Distributed equipment monitor management method, equipment and system |
US8055735B2 (en) * | 2007-10-30 | 2011-11-08 | Hewlett-Packard Development Company, L.P. | Method and system for forming a cluster of networked nodes |
US8117495B2 (en) * | 2007-11-26 | 2012-02-14 | Stratus Technologies Bermuda Ltd | Systems and methods of high availability cluster environment failover protection |
US8825834B2 (en) * | 2007-12-03 | 2014-09-02 | International Business Machines Corporation | Automated cluster member management based on node capabilities |
US7962587B2 (en) * | 2007-12-10 | 2011-06-14 | Oracle America, Inc. | Method and system for enforcing resource constraints for virtual machines across migration |
US8069362B2 (en) * | 2008-02-26 | 2011-11-29 | Sap Ag | Automated electrical power savings in virtualization environments |
US10108460B2 (en) * | 2008-02-28 | 2018-10-23 | International Business Machines Corporation | Method and system for integrated deployment planning for virtual appliances |
US8099615B2 (en) * | 2008-06-30 | 2012-01-17 | Oracle America, Inc. | Method and system for power management in a virtual machine environment without disrupting network connectivity |
US20100017801A1 (en) * | 2008-07-18 | 2010-01-21 | Vmware, Inc. | Profile based creation of virtual machines in a virtualization environment |
US8656018B1 (en) * | 2008-09-23 | 2014-02-18 | Gogrid, LLC | System and method for automated allocation of hosting resources controlled by different hypervisors |
US8448004B2 (en) * | 2008-10-27 | 2013-05-21 | Netapp, Inc. | Power savings using dynamic storage cluster membership |
US8850571B2 (en) * | 2008-11-03 | 2014-09-30 | Fireeye, Inc. | Systems and methods for detecting malicious network content |
US8997219B2 (en) * | 2008-11-03 | 2015-03-31 | Fireeye, Inc. | Systems and methods for detecting malicious PDF network content |
US20100161922A1 (en) * | 2008-12-19 | 2010-06-24 | Richard William Sharp | Systems and methods for facilitating migration of virtual machines among a plurality of physical machines |
US8549364B2 (en) * | 2009-02-18 | 2013-10-01 | Vmware, Inc. | Failure detection and recovery of host computers in a cluster |
US8301859B2 (en) * | 2009-06-22 | 2012-10-30 | Red Hat Israel, Ltd. | Automatically adjusting memory of a VM on a power client |
US8489744B2 (en) * | 2009-06-29 | 2013-07-16 | Red Hat Israel, Ltd. | Selecting a host from a host cluster for live migration of a virtual machine |
US8930731B2 (en) * | 2009-07-21 | 2015-01-06 | Oracle International Corporation | Reducing power consumption in data centers having nodes for hosting virtual machines |
US8359374B2 (en) * | 2009-09-09 | 2013-01-22 | Vmware, Inc. | Fast determination of compatibility of virtual machines and hosts |
US8429449B2 (en) * | 2010-03-01 | 2013-04-23 | International Business Machines Corporation | Optimized placement of virtual machines in a network environment |
CN101778005B (en) * | 2010-03-05 | 2014-03-12 | 中兴通讯股份有限公司 | Method and system for configuring complex network |
US8510590B2 (en) * | 2010-03-17 | 2013-08-13 | Vmware, Inc. | Method and system for cluster resource management in a virtualized computing environment |
US8271814B2 (en) * | 2010-03-22 | 2012-09-18 | Microsoft Corporation | Migrating a client computer to a virtual machine server when the client computer is deemed to be idle |
US9317314B2 (en) * | 2010-06-29 | 2016-04-19 | Microsoft Techology Licensing, Llc | Techniques for migrating a virtual machine using shared storage |
US8635493B2 (en) * | 2011-05-17 | 2014-01-21 | Vmware, Inc. | High availability system allowing conditionally reserved computing resource use and reclamation upon a failover |
US8661182B2 (en) * | 2011-05-26 | 2014-02-25 | Vmware, Inc. | Capacity and load analysis using storage attributes |
US8881142B1 (en) * | 2011-06-21 | 2014-11-04 | Amazon Technologies, Inc. | Determining and using probable instance lifetimes |
US9185152B2 (en) * | 2011-08-25 | 2015-11-10 | Ustream, Inc. | Bidirectional communication on live multimedia broadcasts |
US8639815B2 (en) * | 2011-08-31 | 2014-01-28 | International Business Machines Corporation | Selecting a primary-secondary host pair for mirroring virtual machines |
US9203900B2 (en) * | 2011-09-23 | 2015-12-01 | Netapp, Inc. | Storage area network attached clustered storage system |
US20130083690A1 (en) * | 2011-10-04 | 2013-04-04 | International Business Machines Corporation | Network Adapter Hardware State Migration Discovery in a Stateful Environment |
US20130086298A1 (en) * | 2011-10-04 | 2013-04-04 | International Business Machines Corporation | Live Logical Partition Migration with Stateful Offload Connections Using Context Extraction and Insertion |
GB2496840A (en) * | 2011-11-15 | 2013-05-29 | Ibm | Controlling access to a shared storage system |
US8949415B2 (en) * | 2012-01-04 | 2015-02-03 | International Business Machines Corporation | Activity-based virtual machine availability in a networked computing environment |
JP5982842B2 (en) * | 2012-02-03 | 2016-08-31 | 富士通株式会社 | Computer fault monitoring program, method, and apparatus |
JP5740652B2 (en) * | 2012-03-28 | 2015-06-24 | 株式会社日立製作所 | Computer system and subsystem management method |
US8843935B2 (en) * | 2012-05-03 | 2014-09-23 | Vmware, Inc. | Automatically changing a pre-selected datastore associated with a requested host for a virtual machine deployment based on resource availability during deployment of the virtual machine |
US10664296B2 (en) * | 2012-06-27 | 2020-05-26 | Qatar Foundation | Allocating network interface resources to virtual machines based on total cost |
US8825550B2 (en) * | 2012-08-23 | 2014-09-02 | Amazon Technologies, Inc. | Scaling a virtual machine instance |
US10454997B2 (en) * | 2012-09-07 | 2019-10-22 | Avigilon Corporation | Distributed physical security system |
US9135048B2 (en) * | 2012-09-20 | 2015-09-15 | Amazon Technologies, Inc. | Automated profiling of resource usage |
FR3002807A1 (en) * | 2013-03-01 | 2014-09-05 | France Telecom | METHOD OF DETECTING ATTACKS |
US9069608B2 (en) * | 2013-03-06 | 2015-06-30 | Vmware, Inc. | Method and system for providing a roaming remote desktop |
US20140282520A1 (en) * | 2013-03-15 | 2014-09-18 | Navin Sabharwal | Provisioning virtual machines on a physical infrastructure |
US9454326B1 (en) * | 2013-03-31 | 2016-09-27 | Emc Corporation | File metro cluster for site failover of data storage system |
US9870268B2 (en) * | 2013-08-05 | 2018-01-16 | Amazon Technologies, Inc. | Virtual computing instance migration |
US10348628B2 (en) * | 2013-09-12 | 2019-07-09 | Vmware, Inc. | Placement of virtual machines in a virtualized computing environment |
WO2015080752A1 (en) * | 2013-11-29 | 2015-06-04 | New Jersey Institute Of Technology | Allocation of virtual machines to physical machines through dominant resource assisted heuristics |
US9529609B2 (en) * | 2013-12-10 | 2016-12-27 | Vmware, Inc. | Tracking guest memory characteristics for memory scheduling |
US9430480B1 (en) * | 2013-12-31 | 2016-08-30 | Emc Corporation | Active-active metro-cluster scale-out for unified data path architecture |
US9817683B2 (en) * | 2014-01-10 | 2017-11-14 | Dell Products, Lp | Optimized remediation policy in a virtualized environment |
US9396009B2 (en) * | 2014-01-30 | 2016-07-19 | International Business Machines Corporation | Optimized global capacity management in a virtualized computing environment |
US10481932B2 (en) * | 2014-03-31 | 2019-11-19 | Vmware, Inc. | Auto-scaling virtual switches |
US9405572B2 (en) * | 2014-04-07 | 2016-08-02 | International Business Machines Corporation | Optimized resource allocation and management in a virtualized computing environment |
US9660834B2 (en) * | 2014-05-13 | 2017-05-23 | International Business Machines Corporation | Bursting cloud resources to affect state change performance |
US9417896B2 (en) * | 2014-05-19 | 2016-08-16 | International Business Machines Corporation | Allocating hypervisor resources |
US9678838B2 (en) * | 2014-06-20 | 2017-06-13 | Vmware, Inc. | Protecting virtual machines from network failures |
US11182185B2 (en) * | 2014-06-27 | 2021-11-23 | Vmware, Inc. | Network-based signaling to control virtual machine placement |
US9367407B2 (en) * | 2014-08-05 | 2016-06-14 | International Business Machines Corporation | Optimizing placement plans for host failure in cloud computing environments |
US10481933B2 (en) * | 2014-08-22 | 2019-11-19 | Nicira, Inc. | Enabling virtual machines access to switches configured by different management entities |
US9389901B2 (en) * | 2014-09-09 | 2016-07-12 | Vmware, Inc. | Load balancing of cloned virtual machines |
US9774537B2 (en) * | 2014-09-30 | 2017-09-26 | Nicira, Inc. | Dynamically adjusting load balancing |
US9619266B2 (en) * | 2014-10-10 | 2017-04-11 | International Business Machines Corporation | Tearing down virtual machines implementing parallel operators in a streaming application based on performance |
US10282222B2 (en) * | 2014-10-13 | 2019-05-07 | Vmware, Inc. | Cloud virtual machine defragmentation for hybrid cloud infrastructure |
US9891954B2 (en) * | 2014-11-04 | 2018-02-13 | Vmware, Inc. | Cluster resource management in a virtualized computing environment |
US9891699B2 (en) * | 2014-12-18 | 2018-02-13 | Vmware, Inc. | System and method for performing distributed power management without power cycling hosts |
US10218585B2 (en) * | 2015-02-19 | 2019-02-26 | Red Hat, Inc. | Container host discovery |
US10411966B2 (en) * | 2015-02-26 | 2019-09-10 | Red Hat, Inc. | Host network analyzer |
US10693806B2 (en) * | 2015-03-11 | 2020-06-23 | Vmware, Inc. | Network bandwidth reservations for system traffic and virtual computing instances |
JP6394455B2 (en) * | 2015-03-24 | 2018-09-26 | 富士通株式会社 | Information processing system, management apparatus, and program |
US10430248B2 (en) * | 2015-04-17 | 2019-10-01 | Vmware, Inc. | Proactive high availability in a virtualized computer system |
US10025609B2 (en) * | 2015-04-23 | 2018-07-17 | International Business Machines Corporation | Virtual machine (VM)-to-VM flow control for overlay networks |
US9658875B2 (en) * | 2015-05-01 | 2017-05-23 | International Business Machines Corporation | Anti-collocating multiple virtual entities using prioritized graph coloring and iterative placement |
US9524389B1 (en) * | 2015-06-08 | 2016-12-20 | Amazon Technologies, Inc. | Forensic instance snapshotting |
US9766945B2 (en) * | 2015-06-25 | 2017-09-19 | Wmware, Inc. | Virtual resource scheduling for containers with migration |
US9817734B2 (en) * | 2015-06-29 | 2017-11-14 | Vmware, Inc. | Virtual machine recovery on non-shared storage in a single virtual infrastructure management instance |
US10135692B2 (en) * | 2015-06-29 | 2018-11-20 | Vmware, Inc. | Host management across virtualization management servers |
CN106452818B (en) * | 2015-08-13 | 2020-01-21 | 阿里巴巴集团控股有限公司 | Resource scheduling method and system |
US20170060609A1 (en) * | 2015-08-28 | 2017-03-02 | International Business Machines Corporation | Managing a shared pool of configurable computing resources which has a set of containers |
US9696985B1 (en) * | 2016-01-06 | 2017-07-04 | International Business Machines Corporation | Patching of virtual machines within sequential time windows |
US9569277B1 (en) * | 2016-01-29 | 2017-02-14 | International Business Machines Corporation | Rebalancing virtual resources for virtual machines based on multiple resource capacities |
US10482108B2 (en) * | 2016-02-26 | 2019-11-19 | Red Hat, Inc. | Container clustering in a container-based architecture |
US10262228B2 (en) * | 2016-04-01 | 2019-04-16 | Nicira, Inc. | Automated realization of hand-drawn topologies |
US10135712B2 (en) * | 2016-04-07 | 2018-11-20 | At&T Intellectual Property I, L.P. | Auto-scaling software-defined monitoring platform for software-defined networking service assurance |
US10506022B2 (en) * | 2016-04-20 | 2019-12-10 | Nicira, Inc. | Configuration change realization assessment and timeline builder |
US10310884B2 (en) * | 2016-05-31 | 2019-06-04 | Huawei Technologies Co., Ltd. | Virtual machine placement in a heterogeneous data center |
US10305747B2 (en) * | 2016-06-23 | 2019-05-28 | Sap Se | Container-based multi-tenant computing infrastructure |
-
2016
- 2016-12-02 US US15/368,381 patent/US10346191B2/en active Active
-
2019
- 2019-07-08 US US16/504,416 patent/US20190332415A1/en not_active Abandoned
-
2022
- 2022-01-11 US US17/647,689 patent/US20220129299A1/en active Pending
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023177703A1 (en) * | 2022-03-15 | 2023-09-21 | Liveperson, Inc. | Methods and systems for dynamic load balancing of processing resources in distributed environments |
US11875190B2 (en) | 2022-03-15 | 2024-01-16 | Liveperson, Inc. | Methods and systems for AI-based load balancing of processing resources in distributed environments |
US11875195B2 (en) | 2022-03-15 | 2024-01-16 | Liveperson, Inc. | Methods and systems for dynamic load balancing of processing resources in distributed environments |
Also Published As
Publication number | Publication date |
---|---|
US20180157511A1 (en) | 2018-06-07 |
US20220129299A1 (en) | 2022-04-28 |
US10346191B2 (en) | 2019-07-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220129299A1 (en) | System and Method for Managing Size of Clusters in a Computing Environment | |
US9448728B2 (en) | Consistent unmapping of application data in presence of concurrent, unquiesced writers and readers | |
US9823881B2 (en) | Ensuring storage availability for virtual machines | |
US10387042B2 (en) | System software interfaces for space-optimized block devices | |
US9052825B2 (en) | System software interfaces for space-optimized block devices | |
US8510590B2 (en) | Method and system for cluster resource management in a virtualized computing environment | |
US9569242B2 (en) | Implementing dynamic adjustment of I/O bandwidth for virtual machines using a single root I/O virtualization (SRIOV) adapter | |
US11693789B2 (en) | System and method for mapping objects to regions | |
AU2016213817B2 (en) | Techniques for surfacing host-side flash storage capacity to virtual machines | |
US11461191B2 (en) | Orchestrating and prioritizing the rebuild of storage object components in a hyper-converged infrastructure | |
US9128746B2 (en) | Asynchronous unmap of thinly provisioned storage for virtual machines | |
CN115280285A (en) | Scheduling workloads on a common set of resources by multiple schedulers operating independently | |
US10592297B2 (en) | Use minimal variance to distribute disk slices to avoid over-commitment | |
US20230031741A1 (en) | Quality of service for cloud based storage system | |
US20230176889A1 (en) | Update of virtual machines using clones |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |