US8341626B1 - Migration of a virtual machine in response to regional environment effects - Google Patents

Migration of a virtual machine in response to regional environment effects Download PDF

Info

Publication number
US8341626B1
US8341626B1 US12/240,611 US24061108A US8341626B1 US 8341626 B1 US8341626 B1 US 8341626B1 US 24061108 A US24061108 A US 24061108A US 8341626 B1 US8341626 B1 US 8341626B1
Authority
US
United States
Prior art keywords
machines
power
physical
physical machines
virtual machines
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/240,611
Inventor
Robert D. Gardner
Bret A. McKee
Brian J. Watson
Chris D. Hyser
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hewlett Packard Enterprise Development LP
Original Assignee
Hewlett Packard Development Co LP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US486907P priority Critical
Application filed by Hewlett Packard Development Co LP filed Critical Hewlett Packard Development Co LP
Priority to US12/240,611 priority patent/US8341626B1/en
Assigned to HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P. reassignment HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GARDNER, ROBERT D, HYSER, CHRIS D, MCKEE, BRET A, WATSON, BRIAN J
Application granted granted Critical
Publication of US8341626B1 publication Critical patent/US8341626B1/en
Assigned to HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP reassignment HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485Task life-cycle, e.g. stopping, restarting, resuming execution
    • G06F9/4856Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration

Abstract

To place virtual machines on physical machines, at least one controller receives indicators related to regional environment effects. The at least one controller selects at least one of the virtual machines to migrate from one or more first physical machines to one or more second physical machines in response to the received indicators.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This Application claims the benefit of U.S. Provisional Application Ser. No. 61/004,869, filed Nov. 30, 2007, titled “Migration Of A Virtual Machine In Response To Regional Environment Effects”

BACKGROUND

Virtual machines can be provided in a computer to enhance flexibility and performance. A virtual machine typically refers to some arrangement of components (software and/or hardware) for virtualizing or emulating an actual computer, where the virtual machine can include an operating system and software applications. Virtual machines can allow different operating systems to be deployed on the same computer, such that applications written for different operating systems can be executed in different virtual machines (that contain corresponding operating systems) in the same computer. Moreover, the operating system of a virtual machine can be different from the host operating system that may be running on the computer on which the virtual machine is deployed.

In addition, a greater level of isolation is provided between or among applications running in different virtual machines. In some cases, virtual machines also allow multiple applications to more efficiently share common resources (processing resources, input/output or I/O resources, and storage resources) of the computer

For enhanced performance, virtual machines can be provided on multiple computers that are interconnected by a network. When deploying virtual machines on multiple computers, a human administrator usually has to decide at virtual machine creation time on which physical machine the virtual machine should be deployed. Often, placement of virtual machines is typically performed manually by an administrator. Although some systems are able to provide some indication to the administrator that migrating a virtual machine from one computer to another computer would be desirable, the actual selection of which computer a virtual machine should be migrated to is performed manually by the human administrator. Such manual placement of virtual machines by a human administrator is typically a time-consuming process and often does not lead to optimal or even better placement of virtual machines.

Although some techniques have been proposed to automate the migration of virtual machines across physical machines in response to various conditions, it is noted that such automated migration techniques do not consider some factors that may be relevant to effective or efficient execution of virtual machines.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the invention are described, by way of example, with respect to the following figures:

FIG. 1 is a block diagram of data centers including corresponding physical machines that contain virtual machines, and a global placement controller coupled to the data centers for controlling placement of virtual machines in accordance with an embodiment;

FIG. 2 is a flow diagram of a process of controlling migration of virtual machines across physical machines in response to regional environment effects, in accordance with an embodiment;

FIG. 3 is a block diagram of the global placement controller of FIG. 1, in accordance with an embodiment;

FIG. 4 is a block diagram of physical machines in a data center that also includes a local placement controller according to an embodiment; and

FIG. 5 is a flow diagram of a virtual machine layout selection process performed by the global or local placement controller, in accordance with an embodiment.

DETAILED DESCRIPTION

FIG. 1 illustrates an example arrangement that includes multiple data centers (data center 1 and data center 2 depicted in the example of FIG. 1), where each data center includes a corresponding arrangement of physical machines. For example, data center 1 includes physical machines 100, 102, and data center 2 includes physical machines 104, 106. Although just two physical machines are depicted in FIG. 1 for each data center, it is noted that each data center can include fewer physical machines or more physical machines.

A data center refers to some collection of computing resources, whether geographically concentrated or dispersed, that are able to cooperate to provide computing tasks for one or more clients. In one example, data centers 1 and 2 depicted in FIG. 1 can be located in different geographic regions, such as in different cities, states, countries, etc. Examples of data centers include application server farms, storage server farms, web server farms, switch or router farms, or any other collection of computing resources.

Each physical machine includes one or more virtual machines. In the example of FIG. 1, physical machine 100 includes virtual machines 108, 110; physical machine 102 includes virtual machines 112, 114, 116; physical machine 104 includes virtual machines 118, 120; and physical machine 106 includes virtual machines 122, 124, 126, and 128.

A physical machine can be located within one cabinet (or rack), or alternatively, the physical machine can be located in multiple cabinets (or racks). The physical machines in each data center are interconnected by respective local networks (network 130 in data center 1 and network 132 in data center 2). In the example of FIG. 1, the networks 130 and 132 are depicted as being local area networks, although other types of networks can be used in other implementations. Each network 130, 132 can also include a system bus or any other fast interconnect.

The data centers are interconnected by a global data network 134, which can be the Internet, a wide area network, or any other type of network. The term “network” is intended to refer to either a single network or some collection of multiple networks.

Examples of physical machines include computers, communications modules (e.g., switches, routers, etc.), and other types of machines. A physical machine indicates that the machine is an actual machine made up of software and hardware. Although each of the physical machines is depicted as being contained within a box, it is noted that a physical machine can be a distributed machine having multiple nodes that provide a distributed and parallel processing system.

A virtual machine refers to some partition or segment (made up of software and/or hardware) of the physical machine that is provided to virtualize or emulate a physical machine. From the perspective of a user, a virtual machine looks just like a physical machine. According to some embodiments, a virtual machine includes one or more software applications, an operating system, and one or more device drivers (which are typically part of the operating system).

It is noted that the operating systems that are part of corresponding virtual machines within a physical machine can be different types of operating systems or different versions of an operating system. This allows software applications designed for different operating systems to execute on the same physical machine.

The virtual machines within a physical machine are designed to share the physical resources of the physical machine. In the physical machine, these physical resources include the hardware of the physical machine, where the hardware can include one or more central processing units (CPUs), memory (volatile memory and/or persistent storage, such as disk-based storage), a network interface, and other resources (such as a storage area network interface, not shown).

As further depicted in the example of FIG. 1, each data center includes a corresponding local placement controller 136, 138. Each local placement controller 136, 138 provides automated management of the placement of virtual machines across physical machines of the respective data center. Management of placement of the virtual machines includes migrating virtual machines from source physical machines to destination physical machines.

Each local placement controller 136, 138 is able to determine an efficient placement of the virtual machines across physical machines based on predefined local criteria (according to a target policy set by an enterprise, a user, an administrator, etc.). The predefined local criteria according to the policy can include a target quality-of-service level to be provided, loading criteria (which specify some target loading level on a physical resource or machine), balancing criteria (which specify that load on physical machines should be balanced when possible), cooling criteria (which specify temperatures in physical machines that should not be exceeded), power criteria (which specify that power consumption should be reduced where possible), and other criteria. The loading criteria and balancing criteria are examples of distribution criteria in that such criteria tend to encourage distribution of virtual machines across a larger number of physical machines. On the other hand, the cooling criteria and power criteria are examples of consolidation criteria that tend to consolidate virtual machines onto fewer physical machines.

Each local placement controller 136, 138 is able to determine whether the current state of the overall corresponding data center (which includes the multiple physical machines) conforms to the policy. If the local placement controller determines that the state of the data center does not conform to the policy, then the local placement controller 136 can perform a re-evaluation to determine a different placement of virtual machines on physical machines to increase conformance to the policy.

If a different placement is determined by the local placement controller to be desirable, the local placement controller is able to perform migration of virtual machines from one physical machine to another physical machine within the data center. After placement of the virtual machines has been changed (by migration), the local placement controller can subsequently determine whether further migration should be performed. Note that workloads at the physical machines are continually changing. The local placement controller repeatedly checks (in a loop) whether the state of the data center conforms to policy. In effect, the local placement controller provides a feedback-driven mechanism to control the mapping of virtual machines onto physical machines. This feedback-driven mechanism allows automated adjustments of placement of the virtual machines on physical machines. By automating the virtual machine placement control, relatively quick response times can be achieved in changing placements of virtual machines in response to observed or predicted events that affect conformance with policy.

Migrating a virtual machine refers to moving the state of the virtual machine from one physical machine to another physical machine. The state of the virtual machine includes content of registers of various hardware devices (e.g., CPUs, I/O devices, and so forth). Data in memory associated with the migrated virtual machine can also be transferred gradually (e.g., lazily) to the destination physical machine to which the virtual machine is migrated. Control of the migration can be accomplished by using migration control software in the local placement controllers 136, 138, and/or by using migration control software in the physical machines.

The automated placement (initial and dynamic) of virtual machines on physical machines provided by the local placement controller 136, 138 according to some embodiments allows each data center to meet customer quality-of-service (QoS) goals and the data center's cost-of-service (CoS) goals on a given resource (made up of the physical machines in the system).

In addition to performing local placement of virtual machines on physical machines of a data center, the system of FIG. 1 also provides a global placement controller 140, which is coupled to the global data network 134, to allow for migration of virtual machines from one data center to another data center (or from first geographic region to a second geographic region remotely located from the first geographic region). A first geographic region is remotely located from, or separated from, a second geographic region if the first and second geographic regions are separated by some substantial distance (e.g., a city block, greater than one mile, a city limit, a state line, a country's border, etc.).

In performing placement of virtual machines, the global placement controller 140 takes into account regional environment effects, which include at least some of the following: availability of alternative power sources such as sunlight, hydroelectric power, wind power, etc.; availability of cool temperature; presence of inclement weather; presence of a natural disaster; presence of civil or military unrest; presence of a regional power outage; and so forth. Generally, the term “regional environment effect” refers to some condition that affects a relatively large geographic region or a relatively large group of physical machines. Such regional environment effects may indicate that executing virtual machines in some geographic regions or on some group of physical machines may not be efficient or effective. For example, availability of sunlight in some geographic regions may mean that an alternative, cheaper source of power is available in such geographic regions. Therefore, it may make sense to migrate virtual machines to physical machines in geographic regions where such alternative source of power is available. Cooling costs are also a concern when executing virtual machines in physical machines. Generally, sophisticated cooling equipment may be used to cool a large number of physical machines in a data center. Such sophisticated cooling equipment may consume relatively large amounts of power and thus, may be expensive to run. Therefore, migrating virtual machines to geographic regions that have cooler weather may reduce the load on cooling equipment, and thus reduce cooling costs. Also, physical machines in geographic regions where inclement weather, a natural disaster, civil or military unrest, and/or power outage have occurred may not be available. Therefore, it may be desirable to migrate virtual machines from such geographic regions to other geographic regions.

FIG. 1 thus depicts a multi-tiered virtual machine placement control system, where local placement controllers perform placement of virtual machines within a data center according to policies of the data center, and where the global placement controller 140 performs control of migration of virtual machines across different data centers (or different geographic regions) in response to regional environment effects. The global and local placement controllers can interact with each other to control placement of virtual machines in the overall system. More generally, the global placement controller is able to perform control of migration of virtual machines between or among different groups of physical machines in response to regional environment effects, where a “group” refers to either a data center or a geographic region. By using the multi-tiered placement control mechanism according to some embodiments, more effective automated placement of virtual machines can be accomplished to meet local policies of data centers, as well as to address issues associated with regional environment effects.

In a different embodiment, instead of using separate global and local placement controllers, it is noted that a single-tiered placement controller (or a single-tiered group of placement controllers) can be used to perform control of placement of virtual machines within the data center and across data centers (or geographic regions).

As further depicted in FIG. 1, multiple regional environment effect information sources 142 are coupled to the global data network 134. The regional environment effect information sources 142 are used to communicate indicators of corresponding regional environment effects to the global placement controller 140 across the global data network 134. Examples of regional environment effect information sources 142 include: news media sources; temperature sensors in various geographic regions; power systems indicating availability of alternative sources of power; systems to detect a relatively large region of power outages; weather forecast systems; and so forth. The regional environment effect information sources can also include computers or data entry terminals through which users or operators can enter information relating to regional environment effects.

FIG. 2 illustrates a procedure performed by the global placement controller 140 according to an embodiment. The global placement controller 140 receives (at 202) indicators related to regional environment effects, where the indicators are received from the regional environment effect information sources 142 depicted in FIG. 1. The indicators can be in the form of reports of predefined one or more formats, a stream of data, and so forth. Next, the global placement controller 140 selects (at 204) one or more virtual machines to migrate across data centers (or across geographic regions) in response to the indicators. The selection of the virtual machines to migrate to different data centers takes into consideration the combination of all of the regional environment effects that have been received by the global placement controller 140. Note that in some cases, some regional environment effects may indicate that virtual machines should be migrated to a first geographic region (or first data center), while a different regional environment effect would indicate that virtual machines should be migrated to a second geographic region (or second data center). For example, a geographic region having long, sunny days may also be relatively hot, particularly during summer months. Thus, even though there is ample availability of an alternative power source, in this example solar power, in such a geographic region, the load placed on cooling equipment if a large number of virtual machines were migrated to a data center located in such a hot geographic region may cause the cooling costs to exceed any benefit that would be derived from using solar power. Thus, the global placement controller 140 has to compare the benefits and costs associated with migrating virtual machines according to the different regional environment effects prior to selecting which virtual machines of which data centers (or geographic regions) to migrate to other data centers (or other geographic regions). For example, it may be the case that the global placement controller 140 can decide to leave virtual machines in a physical machine located in a cool geographic region (even though such a geographic region may not have good sunlight availability) due to the expected increase in costs associated with moving the virtual machines to another geographic region having a lot of sunlight, but in which the temperature is relatively elevated.

Next, the global placement controller 140 sends (at 206) command(s) to the appropriate entities to control migration of the selected virtual machine(s) across data centers. The appropriate entities can include local placement controllers 136, 138 and/or the physical machines that are involved.

The tasks of FIG. 2 may be provided in the context of information technology (IT) services offered by one organization to another organization. For example, the infrastructure (including the placement controllers, physical machines, and virtual machines of FIG. 1) may be owned by a first organization. The IT services may be offered as part of an IT services contract, for example.

FIG. 3 shows an example global placement controller 140 (which can be implemented as a computer), which includes a virtual machine (VM) layout selector 302 that receives as input various regional environment effect indicators (indicators 1 to N depicted, where N refers to the number of regional environment effects that are considered by the VM layout selector 302). The VM layout selector 302 selects virtual machines that are executing on physical machine(s) of a first group (e.g., first data center or first geographic region) for migration to physical machine(s) of a second group (e.g., second data center or second geographic region).

The global placement controller 140 also includes a migration control module 304 to control the migration of selected virtual machines (selected by the VM layout selector 302) between or among physical machines in different groups. The VM layout selector 302 and migration control module 304 are software modules executable on one or more central processing unit (CPUs) 306 in the global placement controller 140. The CPU(s) is (are) connected to memory 308 and to a network interface 310 to allow the global placement controller 140 to communicate over the global data network 134.

FIG. 4 illustrates the physical machines 100 and 102 of data center 1 in FIG. 1, as well as the components of the local placement controller 136. Data center 2 has a similar arrangement of physical machines and placement controller.

As depicted in the example of FIG. 4, the virtual machine 108 in the physical machine 100 includes one or more software applications 402, an operating system 404, and device drivers 406 (that are usually part of the operating system 404). The virtual machine 110 similarly includes corresponding one or more software applications, operating system, and device drivers.

The virtual machines 108 and 110 in the physical machine 100 are designed to share physical resources, including hardware 408, which includes one or more CPUs 410, memory 412, a network interface 414, and other components. The network interface 414 allows each of the virtual machines 108 and 110 to communicate over the local network 130.

The physical machine 100 also includes a virtual machine monitor (VMM) 416, also called a hypervisor, which manages the sharing (by virtual machines 108, 110) of the physical resources, including the hardware 408, of the physical machine 100. The VMM 416 virtualizes the physical resources, including the hardware 408, of the physical machine 100. Also, the VMM 416 intercepts requests for resources from operating systems in the respective virtual machines 108, 110 so that proper allocation of the physical resources of the physical machine 100 can be performed. For example, the VMM 416 manages memory access, input/output (I/O) device access, and CPU scheduling for the virtual machines. Effectively, the VMM 416 provides an interface between the operating system of each virtual machine and the underlying hardware 408 of the physical machine 100. The interface provided by the VMM 416 to an operating system of a virtual machine is designed to emulate the interface provided by the actual hardware of the physical machine 100.

The other physical machine 102 similarly includes a VMM 418 that provides the interface between the virtual machines 112, 114, 116 and the physical resources, including hardware 420, of the physical machine 102.

As further depicted in FIG. 4, the local placement controller 136 includes one or more CPUs 422 and memory 424. Also, a network interface 426 is provided in the placement controller 136 to allow communication between the placement controller 136 and the local network 130.

The local placement controller 136 also includes a VM layout selector 428 (to perform selection of a layout of the virtual machines mapped to physical machines within data center 1). Additionally, the local placement controller 136 includes a migration control module 430 to control migration of virtual machines from one physical machine to another physical machine (in accordance with the layout selected by the VM layout selector 428). In some implementations, the VM layout selector 428 and migration control module 430 are software modules executable on the CPU 422 of the local placement controller 136.

As noted above, in determining placement of virtual machines onto corresponding physical machines, the local placement controller 136 (more specifically the VM layout selector 428) takes into account various predefined criteria. One type of predefined criteria is related to loading of physical resources or machines. Since there are various different physical resources (e.g., CPU, memory, network interface, storage network, etc.), there are different loading criteria to consider. The different loading criteria thus include a loading criterion relating to the CPU usage, a loading criterion relating to memory usage, a loading criterion relating to network interface usage, a loading criterion relating to storage network usage, and any other loading criteria relating to usage of other resources.

Also, another loading criterion is an overloading criterion, which specifies that any particular physical machine should not be loaded by greater than some predefined threshold (e.g., 80%). If loading of a physical machine exceeds this predefined threshold, then migration of virtual machine(s) from the overloaded physical machine would be desirable.

As noted above, placement of virtual machines should satisfy a QoS goal of the system. This QoS may have been set by a user or by the system. One example QoS goal is the allocation of scheduling shares to each virtual machine within a physical machine. The VMM 416 or 418 according to some implementations includes a scheduler that schedules access to physical resources of the physical machine 100 or 102, where the scheduler in the VMM specifies the number of shares to allocate to each virtual machine.

Other example criteria that are considered by the VM layout selector 428 include cooling and power criteria. During operation, temperatures in physical machines can exceed a temperature threshold. If this occurs, then migration of virtual machines from the over-heated physical machines may be desirable. Cooling criteria can also specify cooling costs. In large data centers, for example, sophisticated cooling equipment is usually provided to cool physical machines. To reduce cooling costs, migration of virtual machines can be performed in a manner that allows some part of the cooling equipment to be turned off (or run in a lower power mode). In one example, virtual machines can be placed such that one or more physical machines can be turned off (or placed in a lower power state), such that corresponding cooling equipment can be shut off (or run in a lower power mode). The lower power state of a physical machine refers to any one of several possible power savings states of such machines, including sleep states, standby states, hibernation states, and so forth.

Power criteria are also considered by the VM layout selector 428. In addition to turning off (or placing into a lower power state) physical machines for the purpose of reducing cooling costs, such action can also be performed for the purpose of reducing power consumption of physical machines (for more efficient usage).

Both the cooling criteria and power criteria are considered consolidation criteria that favor migrating virtual machines onto fewer physical machines so that at least some of the physical machines can be turned off or otherwise placed into an idle state, for cost and power conservation. However, the cooling and power criteria are counter-balanced by some of the other factors, such as loading criteria that tend to favor distributing or spreading virtual machines across a larger number of physical machines.

Another criterion that can be considered by the VM layout selector 428 is the cost of performing migration of a virtual machine. Migrating a virtual machine from one physical machine to another physical machine is associated with a cost (in terms of the virtual machine being unavailable during migration and the bandwidth cost associated with moving data associated with the migrated virtual machine from one storage location to another storage location). If the VM layout selector 428 determines that the benefit of migrating a virtual machine is less than the cost of performing the migration, then the migration should not be performed.

The placement problem for identifying an optimal solution for placement of virtual machines on physical machines is a relatively complex problem. For a large system having many physical machines and virtual machines, the computation time for identifying the optimal placement of virtual machines can be very large. In accordance with an embodiment, a simulated annealing technique can be used to find an approximation of a globally optimal placement of virtual machines on physical machines. Generally, simulated annealing considers a current set (that represents some random placement of virtual machines on physical machines), and iteratively adjusts the current set until a better set can be identified. The simulated annealing algorithm attempts to replace a current solution with a random “nearby” solution. A new set that is more optimal than the current set is used as the new current set, and this process repeats until an approximated optimal solution is identified.

The simulated annealing algorithm performed by the VM layout selector 302 in the global placement controller 140 (FIG. 3) or the VM layout selector 428 in the local placement controller 136 (FIG. 4) can be based on a cost function (alternatively referred to as a “goodness function”) that takes into account various constraints that represent the predefined criteria, e.g., criteria considered by the local placement controller or the regional environment effects considered by the global placement controller 140. The cost function is represented by some aggregation of terms, where each term is multiplied by a weight factor. Generally, the cost function has the form: f=w1×t1+w2×t2+ . . . , where the ti terms (i=1 to N) represent corresponding criteria (N total criteria, where N is an integer) to be considered, and the wi parameters represent weights to be applied to the terms. As examples, the ti terms can represent resource loading criteria (n terms to correspond to the n dimensions of resources), balancing criteria, cooling criteria, power criteria, regional environment effect factors, and so forth. The weight to be multiplied to each term generally represents the importance of the corresponding term in computing the goodness rating for a particular layout of virtual machines.

A flow diagram of a process performed by the VM layout selector 302 (FIG. 3) or the VM layout selector 428 (FIG. 4) is depicted in FIG. 5. The flow of FIG. 5 is performed repeatedly, such as at regular intervals or in response to predefined events. The VM layout selector receives (at 502) various parameters (e.g., parameters relating to regional environment effects and/or other criteria discussed above) relating to the terms to be considered for the cost function discussed above. The received parameters are converted (at 504) into the terms for use in the cost function.

The VM layout selector then evaluates (at 506) the cost function based on the terms derived from the measured parameters. The cost function produces an output that provides some indication of the performance of the particular placement of virtual machines on physical machines (such as in the form of a goodness rating or some other indicator). Next, the VM layout selector searches (at 508) for a better layout that represents another placement of virtual machines on physical machines.

If a better virtual machine layout is found in the search (at 508) for the better layout, the VM layout selector causes the migration control module (304 in FIG. 3 or 430 in FIG. 4) to perform (at 510) migration of one or more virtual machines according to the better virtual machine layout. As noted above, migrating a virtual machine refers to moving the state of the virtual machine from one physical machine to another physical machine. The state of the virtual machine includes content of registers of various hardware devices (e.g., CPUs, I/O devices, and so forth). Data in memory associated with the migrated virtual machine can also be transferred gradually (e.g., lazily) to the destination physical machine to which the virtual machine is migrated.

The VM layout selector determines (at 512) if the process at 504-510 is to be repeated. Repetition of the process at 504-510 can be performed periodically or in response to predefined events (e.g., detection of overheating, alarm indicating some fault has occurred, indication of overloading, and/or change in regional environment effects, etc.).

Instructions of software described above (including the VM layout selector 302 or 428, migration control module 304 or 430) are loaded for execution on a processor (such as one or more CPUs). The processor includes microprocessors, microcontrollers, processor modules or subsystems (including one or more microprocessors or microcontrollers), or other control or computing devices. As used here, a “processor” can refer to a single component or to plural components.

Data and instructions (of the software) are stored in respective storage devices, which are implemented as one or more machine-readable storage media. The storage media include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; and optical media such as compact disks (CDs) or digital video disks (DVDs). Note that the instructions of the software discussed above can be provided on one computer-readable or computer-usable storage medium, or alternatively, can be provided on multiple computer-readable or computer-usable storage media distributed in a large system having possibly plural nodes. Such computer-readable or computer-usable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components.

In the foregoing description, numerous details are set forth to provide an understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these details. While the invention has been disclosed with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover such modifications and variations as fall within the true spirit and scope of the invention.

Claims (19)

1. A method of placing virtual machines on physical machines, comprising:
receiving, by at least one controller, indicators related to regional environment effects, wherein the indicators are related to an availability of an alternative source of power, where the alternative source of power is at least one selected from among sunlight, a hydroelectric power source, and a wind power source; and
selecting, by the at least one controller, at least one of the virtual machines to migrate from one or more first physical machines to one or more second physical machines in response to the availability of the alternative source of power as indicated in the received indicators.
2. The method of claim 1, wherein receiving the indicators comprises receiving the indicators from sources in different geographic regions that are coupled to a network.
3. The method of claim 2, wherein receiving the indicators from the sources comprises receiving the indicators from at least one of the following: news media sources; and power systems indicating availability of alternative sources of power.
4. The method of claim 1, further comprising migrating the at least one of the virtual machines from the one or more first physical machines to the one or more second physical machines, wherein the one or more first physical machines are part of a first geographic region, and the one or more second physical machines are in a second geographic region remotely located from the first geographic region, wherein the received indicators indicate that the alternative source of power is available in the second geographic region and the alternative source of power is unavailable in the first geographic region.
5. The method of claim 1, further comprising:
migrating the at least one virtual machine from the one or more first physical machines to the one or more second physical machines, wherein the one or more first physical machines are part of a first data center, and wherein the one or more second physical machines are part of a second, distinct data center, wherein the received indicators indicate that the alternative source of power is available in the second data center and the alternative source of power is unavailable in the first data center.
6. The method of claim 1, wherein the at least one controller comprises a global placement controller, the one or more first physical machines are part of a first group, and the one or more second physical machines are part of a second, distinct group, the method further comprising:
selecting, by local placement controllers in the corresponding first and second groups, virtual machines to migrate among physical machines in the corresponding first and second groups, wherein the local placement controllers employ respective local criteria to migrate virtual machines among physical machines in the corresponding first and second groups.
7. The method of claim 6, wherein the first and second groups comprise one of separate geographic regions and distinct data centers.
8. The method of claim 6, wherein the local criteria are plural criteria selected from among a loading criterion that specifies a target loading level on a physical machine, and a balancing criterion that specifies balancing of loading across physical machines.
9. The method of claim 1, wherein selecting the at least one of the virtual machines to migrate is performed by a virtual machine layout selector that considers criteria including the regional environment effects.
10. The method of claim 1, further comprising providing information technology services, wherein the receiving and selecting are part of the information technology services.
11. An article comprising at least one computer-readable storage medium containing instructions that when executed cause a computer to:
receive information related to regional environment effects, wherein the received information includes availability of an alternative source of power, and availability of cooler weather in one region as compared to another region, where the alternative source of power is at least one selected from among sunlight, a hydroelectric power source, and a wind power source;
select at least one of the virtual machines to migrate from one or more first physical machines in a first group to one or more second physical machines in a second group, in response to the received information indicating availability of the alternative source of power and the availability of cooler weather.
12. The article of claim 11, wherein the first group comprises a first geographic region, and the second group comprises a second geographic region located remotely from the first geographic region, wherein the received indicators indicate that the alternative source of power is available in the second geographic region and the alternative source of power is unavailable in the first geographic region.
13. The article of claim 11, wherein the first group comprises a first data center, and the second group comprises a second, distinct data center, wherein the received indicators indicate that the alternative source of power is available in the second data center and the alternative source of power is unavailable in the first data center.
14. The article of claim 11, wherein the computer comprises a global placement controller to perform the receiving and selecting tasks, and wherein the global placement controller is part of a multi-tiered placement control system that further includes local placement controllers in the corresponding first and second groups to perform local placement of virtual machines within the corresponding first and second groups according to respective local criteria.
15. The article of claim 14, wherein the local criteria are plural criteria selected from among a loading criterion that specifies a target loading level on a physical machine, and a balancing criterion that specifies balancing of loading across physical machines.
16. A system comprising:
a global placement controller comprising:
a network interface to receive information related to regional environment effects, the received information including availability of an alternative source of power selected from among sunlight, a hydroelectric power source, and a wind power source; and
a processor to:
in response to the received information indicating availability of the alternative source of power, select virtual machines to migrate from physical machines of a first group to physical machines of a second, distinct group; and
control migration of the virtual machines from the physical machines of the first group to the physical machines of the second group.
17. The system of claim 16, further comprising local placement controllers in the corresponding first and second groups to control migration of virtual machines in the respective groups according to respective local criteria.
18. The system of claim 17, wherein the local criteria comprise distribution criteria to distribute virtual machines across physical machines of each group, and consolidation criteria to consolidate virtual machines onto physical machines of each group.
19. The system of claim 16, further comprising sources to communicate the information related to the regional effects to the global placement controller.
US12/240,611 2007-11-30 2008-09-29 Migration of a virtual machine in response to regional environment effects Active 2031-04-07 US8341626B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US486907P true 2007-11-30 2007-11-30
US12/240,611 US8341626B1 (en) 2007-11-30 2008-09-29 Migration of a virtual machine in response to regional environment effects

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12/240,611 US8341626B1 (en) 2007-11-30 2008-09-29 Migration of a virtual machine in response to regional environment effects

Publications (1)

Publication Number Publication Date
US8341626B1 true US8341626B1 (en) 2012-12-25

Family

ID=47359845

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/240,611 Active 2031-04-07 US8341626B1 (en) 2007-11-30 2008-09-29 Migration of a virtual machine in response to regional environment effects

Country Status (1)

Country Link
US (1) US8341626B1 (en)

Cited By (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110004878A1 (en) * 2009-06-30 2011-01-06 Hubert Divoux Methods and systems for selecting a desktop execution location
US20120179817A1 (en) * 2011-01-06 2012-07-12 International Business Machines Corporation Techniques for addressing geographical location issues in computing environments
US20130041989A1 (en) * 2011-08-08 2013-02-14 International Business Machines Corporation Dynamically relocating workloads in a networked computing environment
US20130073730A1 (en) * 2011-09-20 2013-03-21 International Business Machines Corporation Virtual machine placement within a server farm
US20130111033A1 (en) * 2011-10-31 2013-05-02 Yun Mao Systems, methods, and articles of manufacture to provide cloud resource orchestration
US20130138818A1 (en) * 2011-05-26 2013-05-30 Siemens Aktiengesellschaft Method for accessing an automation system and system operating according to the method
US20130247047A1 (en) * 2008-03-28 2013-09-19 Fujitsu Limited Recording medium having virtual machine managing program recorded therein and managing server device
US8560791B2 (en) * 2009-09-14 2013-10-15 Vmware, Inc. Method and system for optimizing live migration of persistent data of virtual machine using disk I/O heuristics
US8620958B1 (en) 2012-09-11 2013-12-31 International Business Machines Corporation Dimensionally constrained synthetic context objects database
US8676857B1 (en) 2012-08-23 2014-03-18 International Business Machines Corporation Context-based search for a data store related to a graph node
US8782777B2 (en) 2012-09-27 2014-07-15 International Business Machines Corporation Use of synthetic context-based objects to secure data stores
US8799269B2 (en) 2012-01-03 2014-08-05 International Business Machines Corporation Optimizing map/reduce searches by using synthetic events
US8856946B2 (en) 2013-01-31 2014-10-07 International Business Machines Corporation Security filter for context-based data gravity wells
US8898165B2 (en) 2012-07-02 2014-11-25 International Business Machines Corporation Identification of null sets in a context-based electronic document search
US8903813B2 (en) 2012-07-02 2014-12-02 International Business Machines Corporation Context-based electronic document search using a synthetic event
US20140359778A1 (en) * 2013-01-15 2014-12-04 Empire Technology Development Llc Function-targeted virtual machine switching
US8914413B2 (en) 2013-01-02 2014-12-16 International Business Machines Corporation Context-based data gravity wells
US8931109B2 (en) 2012-11-19 2015-01-06 International Business Machines Corporation Context-based security screening for accessing data
US8935695B1 (en) * 2012-07-12 2015-01-13 Symantec Corporation Systems and methods for managing multipathing configurations for virtual machines
US8959119B2 (en) 2012-08-27 2015-02-17 International Business Machines Corporation Context-based graph-relational intersect derived database
US8983981B2 (en) 2013-01-02 2015-03-17 International Business Machines Corporation Conformed dimensional and context-based data gravity wells
US9053102B2 (en) 2013-01-31 2015-06-09 International Business Machines Corporation Generation of synthetic context frameworks for dimensionally constrained hierarchical synthetic context-based objects
US20150169349A1 (en) * 2013-12-17 2015-06-18 Vmware, Inc. Location-Based Management of Virtual Machines in Datacenters
US9069752B2 (en) 2013-01-31 2015-06-30 International Business Machines Corporation Measuring and displaying facets in context-based conformed dimensional data gravity wells
US9110722B2 (en) 2013-02-28 2015-08-18 International Business Machines Corporation Data processing work allocation
US20150237066A1 (en) * 2012-06-27 2015-08-20 Qatar Foundation Arrangement configured to migrate a virtual machine in the event of an attack
US20150286493A1 (en) * 2014-04-03 2015-10-08 Strato Scale Ltd. Virtual-machine placement based on information from multiple data centers
US9195608B2 (en) 2013-05-17 2015-11-24 International Business Machines Corporation Stored data analysis
US9207976B2 (en) 2013-08-13 2015-12-08 International Business Machines Corporation Management of prioritizing virtual machines in an operating environment
US9223846B2 (en) 2012-09-18 2015-12-29 International Business Machines Corporation Context-based navigation through a database
US9229932B2 (en) 2013-01-02 2016-01-05 International Business Machines Corporation Conformed dimensional data gravity wells
US9251237B2 (en) 2012-09-11 2016-02-02 International Business Machines Corporation User-specific synthetic context object matching
US9262499B2 (en) 2012-08-08 2016-02-16 International Business Machines Corporation Context-based graphical database
WO2016029440A1 (en) * 2014-08-29 2016-03-03 Hewlett-Packard Development Company, L.P. Virtual machine service availability
US20160070612A1 (en) * 2014-09-04 2016-03-10 Fujitsu Limited Information processing apparatus, information processing method, and information processing system
US9292506B2 (en) 2013-02-28 2016-03-22 International Business Machines Corporation Dynamic generation of demonstrative aids for a meeting
US9348794B2 (en) 2013-05-17 2016-05-24 International Business Machines Corporation Population of context-based data gravity wells
US20160170791A1 (en) * 2014-12-10 2016-06-16 University-Industry Cooperation Group Of Kyung Hee University Device for controlling migration in a distributed cloud environment and method for controlling migration using the same
US20160188218A1 (en) * 2014-12-31 2016-06-30 Cleversafe, Inc. Synchronizing storage of data copies in a dispersed storage network
US9383986B2 (en) 2013-06-18 2016-07-05 Disney Enterprises, Inc. Safe low cost web services software deployments
US9460200B2 (en) 2012-07-02 2016-10-04 International Business Machines Corporation Activity recommendation based on a context-based electronic files search
US9524328B2 (en) 2014-12-28 2016-12-20 Strato Scale Ltd. Recovery synchronization in a distributed storage system
US9594579B2 (en) 2011-07-29 2017-03-14 Hewlett Packard Enterprise Development Lp Migrating virtual machines
US9619580B2 (en) 2012-09-11 2017-04-11 International Business Machines Corporation Generation of synthetic context objects
US20170147811A1 (en) * 2015-05-08 2017-05-25 Telefonaktiebolaget L M Ericssn (Publ) Migration service method and module for software modules
US20170170990A1 (en) * 2015-12-15 2017-06-15 Microsoft Technology Licensing, Llc Scalable Tenant Networks
US9741138B2 (en) 2012-10-10 2017-08-22 International Business Machines Corporation Node cluster relationships in a graph database
US10152526B2 (en) 2013-04-11 2018-12-11 International Business Machines Corporation Generation of synthetic context objects using bounded context objects
US10261874B2 (en) * 2016-12-01 2019-04-16 International Business Machines Corporation Enabling a cloud controller to communicate with power systems

Citations (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5745735A (en) 1995-10-26 1998-04-28 International Business Machines Corporation Localized simulated annealing
US5802062A (en) 1996-06-19 1998-09-01 At&T Corp Preventing conflicts in distributed systems
US20010034752A1 (en) 2000-01-26 2001-10-25 Prompt2U Inc. Method and system for symmetrically distributed adaptive matching of partners of mutual interest in a computer network
US6321337B1 (en) 1997-09-09 2001-11-20 Sanctum Ltd. Method and system for protecting operations of trusted internal networks
US6421739B1 (en) 1999-01-30 2002-07-16 Nortel Networks Limited Fault-tolerant java virtual machine
US20020194389A1 (en) 2001-06-08 2002-12-19 Worley William S. Secure machine platform that interfaces to operating systems and customized control programs
US20040010787A1 (en) 2002-07-11 2004-01-15 Traut Eric P. Method for forking or migrating a virtual machine
US6732139B1 (en) 1999-08-16 2004-05-04 International Business Machines Corporation Method to distribute programs using remote java objects
US20040117539A1 (en) 2002-12-17 2004-06-17 Intel Corporation Methods and systems to control virtual machines
US20040186920A1 (en) 1999-09-28 2004-09-23 Birdwell John D. Parallel data processing architecture
US20040193476A1 (en) 2003-03-31 2004-09-30 Aerdts Reinier J. Data center analysis
US20050039180A1 (en) 2003-08-11 2005-02-17 Scalemp Inc. Cluster-based operating system-agnostic virtual computing system
US20050060590A1 (en) 2003-09-16 2005-03-17 International Business Machines Corporation Power-aware workload balancing usig virtual machines
US6901522B2 (en) 2001-06-07 2005-05-31 Intel Corporation System and method for reducing power consumption in multiprocessor system
US20050149940A1 (en) 2003-12-31 2005-07-07 Sychron Inc. System Providing Methodology for Policy-Based Resource Allocation
US20050166074A1 (en) 2004-01-22 2005-07-28 International Business Machines Corporation Method and apparatus for reducing power consumption in a logically partitioned data processing system
US20050166075A1 (en) 2004-01-22 2005-07-28 International Business Machines Corporation Method and apparatus for managing power and thermal alerts transparently to an operating system in a data processing system with increased granularity in reducing power usage and thermal generation
US6934952B2 (en) 2001-04-09 2005-08-23 International Business Machines Corporation Method and apparatus for managing multiple instances of server code on a machine
US6944699B1 (en) 1998-05-15 2005-09-13 Vmware, Inc. System and method for facilitating context-switching in a multi-context computer system
US20050246505A1 (en) 2004-04-29 2005-11-03 Mckenney Paul E Efficient sharing of memory between applications running under different operating systems on a shared hardware system
US20050251802A1 (en) 2004-05-08 2005-11-10 Bozek James J Dynamic migration of virtual machine computer programs upon satisfaction of conditions
US6970902B1 (en) 2001-05-24 2005-11-29 Cisco Technology, Inc. Method and apparatus for providing a distributed service in a network
US20050278722A1 (en) 2004-05-28 2005-12-15 International Business Machines Corporation System for correct distribution of hypervisor work
US20060005189A1 (en) * 2004-06-30 2006-01-05 Microsoft Corporation Systems and methods for voluntary migration of a virtual machine between hosts with common storage connectivity
US6985951B2 (en) 2001-03-08 2006-01-10 International Business Machines Corporation Inter-partition message passing method, system and program product for managing workload in a partitioned processing environment
US20060023884A1 (en) 2004-07-31 2006-02-02 Mckee Bret Method and system for preventing current-privilege-level-information leaks to non-privileged code
US20060041733A1 (en) 2004-07-31 2006-02-23 Chris Hyser Operating-system-transparent distributed memory
US20060075067A1 (en) 2004-08-30 2006-04-06 International Business Machines Corporation Remote direct memory access with striping over an unreliable datagram transport
US20060136913A1 (en) 2004-12-09 2006-06-22 International Business Machines Corporation Method, system and computer program product for an automatic resource management of a virtual machine
US20060149906A1 (en) 2005-01-04 2006-07-06 International Business Machines Corporation Method and apparatus for inter partition communication within a logical partitioned data processing system
US7089558B2 (en) 2001-03-08 2006-08-08 International Business Machines Corporation Inter-partition message passing method, system and program product for throughput measurement in a partitioned processing environment
US20060195715A1 (en) 2005-02-28 2006-08-31 Herington Daniel E System and method for migrating virtual machines on cluster systems
US20060200820A1 (en) 2005-03-02 2006-09-07 Ludmila Cherkasova System and method for attributing to a corresponding virtual machine CPU usage of a domain in which a shared resource's device driver resides
US20060200819A1 (en) 2005-03-02 2006-09-07 Ludmila Cherkasova System and method for attributing CPU usage of a virtual machine monitor to a corresponding virtual machine
US20060200821A1 (en) 2005-03-02 2006-09-07 Ludmila Cherkasova System and method for attributing to a corresponding virtual machine CPU usage of an isolated driver domain in which a shared resource's device driver resides
US20060230407A1 (en) 2005-04-07 2006-10-12 International Business Machines Corporation Method and apparatus for using virtual machine technology for managing parallel communicating applications
US7127625B2 (en) 2003-09-04 2006-10-24 Hewlett-Packard Development Company, L.P. Application management based on power consumption
US7174381B2 (en) 2001-12-04 2007-02-06 Aspeed Software Corporation Parallel computing system, method and architecture
US20070043860A1 (en) 2005-08-15 2007-02-22 Vipul Pabari Virtual systems management
US20070067435A1 (en) 2003-10-08 2007-03-22 Landis John A Virtual data center that allocates and manages system resources across multiple nodes
US20070079307A1 (en) 2005-09-30 2007-04-05 Puneet Dhawan Virtual machine based network carriers
US7203944B1 (en) 2003-07-09 2007-04-10 Veritas Operating Corporation Migrating virtual machines among computer systems to balance load caused by virtual machines
US20070083642A1 (en) 2005-10-07 2007-04-12 International Business Machines Corporation Fully distributed data collection and consumption to maximize the usage of context, resource, and capacity-based client server interactions
US20070124474A1 (en) 2005-11-30 2007-05-31 Digital Display Innovations, Llc Multi-user display proxy server
US20070171921A1 (en) 2006-01-24 2007-07-26 Citrix Systems, Inc. Methods and systems for interacting, via a hypermedium page, with a virtual machine executing in a terminal services session
US20070180280A1 (en) 2006-02-01 2007-08-02 Bolan Joseph E Controlling the allocation of power to a plurality of computers whose supply of power is managed by a common power manager
US20070180436A1 (en) * 2005-12-07 2007-08-02 Franco Travostino Seamless Live Migration of Virtual Machines across Optical Networks
US20070260417A1 (en) * 2006-03-22 2007-11-08 Cisco Technology, Inc. System and method for selectively affecting a computing environment based on sensed data
US20080104587A1 (en) 2006-10-27 2008-05-01 Magenheimer Daniel J Migrating a virtual machine from a first physical machine in response to receiving a command to lower a power mode of the first physical machine
US20080104608A1 (en) 2006-10-27 2008-05-01 Hyser Chris D Starting up at least one virtual machine in a physical machine by a load balancer
US7480908B1 (en) 2005-06-24 2009-01-20 Azul Systems, Inc. Segmented virtual machine transport mechanism
US20090031307A1 (en) * 2007-07-24 2009-01-29 International Business Machines Corporation Managing a virtual machine
US7519964B1 (en) 2003-12-03 2009-04-14 Sun Microsystems, Inc. System and method for application deployment in a domain for a cluster
US20090228726A1 (en) * 2008-03-07 2009-09-10 Malik Naim R Environmentally Cognizant Power Management
US20100057529A1 (en) * 2008-09-03 2010-03-04 International Business Machines Corporation Provider-requested relocation of computerized workloads
US7856549B2 (en) * 2007-01-24 2010-12-21 Hewlett-Packard Development Company, L.P. Regulating power consumption
US7925923B1 (en) * 2008-01-31 2011-04-12 Hewlett-Packard Development Company, L.P. Migrating a virtual machine in response to failure of an instruction to execute

Patent Citations (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5745735A (en) 1995-10-26 1998-04-28 International Business Machines Corporation Localized simulated annealing
US5802062A (en) 1996-06-19 1998-09-01 At&T Corp Preventing conflicts in distributed systems
US6321337B1 (en) 1997-09-09 2001-11-20 Sanctum Ltd. Method and system for protecting operations of trusted internal networks
US6944699B1 (en) 1998-05-15 2005-09-13 Vmware, Inc. System and method for facilitating context-switching in a multi-context computer system
US6421739B1 (en) 1999-01-30 2002-07-16 Nortel Networks Limited Fault-tolerant java virtual machine
US6732139B1 (en) 1999-08-16 2004-05-04 International Business Machines Corporation Method to distribute programs using remote java objects
US20040186920A1 (en) 1999-09-28 2004-09-23 Birdwell John D. Parallel data processing architecture
US20010034752A1 (en) 2000-01-26 2001-10-25 Prompt2U Inc. Method and system for symmetrically distributed adaptive matching of partners of mutual interest in a computer network
US7089558B2 (en) 2001-03-08 2006-08-08 International Business Machines Corporation Inter-partition message passing method, system and program product for throughput measurement in a partitioned processing environment
US6985951B2 (en) 2001-03-08 2006-01-10 International Business Machines Corporation Inter-partition message passing method, system and program product for managing workload in a partitioned processing environment
US6934952B2 (en) 2001-04-09 2005-08-23 International Business Machines Corporation Method and apparatus for managing multiple instances of server code on a machine
US6970902B1 (en) 2001-05-24 2005-11-29 Cisco Technology, Inc. Method and apparatus for providing a distributed service in a network
US6901522B2 (en) 2001-06-07 2005-05-31 Intel Corporation System and method for reducing power consumption in multiprocessor system
US20020194389A1 (en) 2001-06-08 2002-12-19 Worley William S. Secure machine platform that interfaces to operating systems and customized control programs
US7174381B2 (en) 2001-12-04 2007-02-06 Aspeed Software Corporation Parallel computing system, method and architecture
US20040010787A1 (en) 2002-07-11 2004-01-15 Traut Eric P. Method for forking or migrating a virtual machine
US20040117539A1 (en) 2002-12-17 2004-06-17 Intel Corporation Methods and systems to control virtual machines
US20040193476A1 (en) 2003-03-31 2004-09-30 Aerdts Reinier J. Data center analysis
US7203944B1 (en) 2003-07-09 2007-04-10 Veritas Operating Corporation Migrating virtual machines among computer systems to balance load caused by virtual machines
US20050039180A1 (en) 2003-08-11 2005-02-17 Scalemp Inc. Cluster-based operating system-agnostic virtual computing system
US7127625B2 (en) 2003-09-04 2006-10-24 Hewlett-Packard Development Company, L.P. Application management based on power consumption
US20050060590A1 (en) 2003-09-16 2005-03-17 International Business Machines Corporation Power-aware workload balancing usig virtual machines
US20070067435A1 (en) 2003-10-08 2007-03-22 Landis John A Virtual data center that allocates and manages system resources across multiple nodes
US7519964B1 (en) 2003-12-03 2009-04-14 Sun Microsystems, Inc. System and method for application deployment in a domain for a cluster
US20050149940A1 (en) 2003-12-31 2005-07-07 Sychron Inc. System Providing Methodology for Policy-Based Resource Allocation
US20050166075A1 (en) 2004-01-22 2005-07-28 International Business Machines Corporation Method and apparatus for managing power and thermal alerts transparently to an operating system in a data processing system with increased granularity in reducing power usage and thermal generation
US20050166074A1 (en) 2004-01-22 2005-07-28 International Business Machines Corporation Method and apparatus for reducing power consumption in a logically partitioned data processing system
US20050246505A1 (en) 2004-04-29 2005-11-03 Mckenney Paul E Efficient sharing of memory between applications running under different operating systems on a shared hardware system
US20050251802A1 (en) 2004-05-08 2005-11-10 Bozek James J Dynamic migration of virtual machine computer programs upon satisfaction of conditions
US20050278722A1 (en) 2004-05-28 2005-12-15 International Business Machines Corporation System for correct distribution of hypervisor work
US20060005189A1 (en) * 2004-06-30 2006-01-05 Microsoft Corporation Systems and methods for voluntary migration of a virtual machine between hosts with common storage connectivity
US20060023884A1 (en) 2004-07-31 2006-02-02 Mckee Bret Method and system for preventing current-privilege-level-information leaks to non-privileged code
US20060041733A1 (en) 2004-07-31 2006-02-23 Chris Hyser Operating-system-transparent distributed memory
US20060075067A1 (en) 2004-08-30 2006-04-06 International Business Machines Corporation Remote direct memory access with striping over an unreliable datagram transport
US20060136913A1 (en) 2004-12-09 2006-06-22 International Business Machines Corporation Method, system and computer program product for an automatic resource management of a virtual machine
US20060149906A1 (en) 2005-01-04 2006-07-06 International Business Machines Corporation Method and apparatus for inter partition communication within a logical partitioned data processing system
US20060195715A1 (en) 2005-02-28 2006-08-31 Herington Daniel E System and method for migrating virtual machines on cluster systems
US20060200821A1 (en) 2005-03-02 2006-09-07 Ludmila Cherkasova System and method for attributing to a corresponding virtual machine CPU usage of an isolated driver domain in which a shared resource's device driver resides
US20060200819A1 (en) 2005-03-02 2006-09-07 Ludmila Cherkasova System and method for attributing CPU usage of a virtual machine monitor to a corresponding virtual machine
US20060200820A1 (en) 2005-03-02 2006-09-07 Ludmila Cherkasova System and method for attributing to a corresponding virtual machine CPU usage of a domain in which a shared resource's device driver resides
US20060230407A1 (en) 2005-04-07 2006-10-12 International Business Machines Corporation Method and apparatus for using virtual machine technology for managing parallel communicating applications
US7480908B1 (en) 2005-06-24 2009-01-20 Azul Systems, Inc. Segmented virtual machine transport mechanism
US20070043860A1 (en) 2005-08-15 2007-02-22 Vipul Pabari Virtual systems management
US20070079307A1 (en) 2005-09-30 2007-04-05 Puneet Dhawan Virtual machine based network carriers
US20070083642A1 (en) 2005-10-07 2007-04-12 International Business Machines Corporation Fully distributed data collection and consumption to maximize the usage of context, resource, and capacity-based client server interactions
US20070124474A1 (en) 2005-11-30 2007-05-31 Digital Display Innovations, Llc Multi-user display proxy server
US20070180436A1 (en) * 2005-12-07 2007-08-02 Franco Travostino Seamless Live Migration of Virtual Machines across Optical Networks
US20070180450A1 (en) 2006-01-24 2007-08-02 Citrix Systems, Inc. Methods and systems for selecting a method for execution, by a virtual machine, of an application program
US20070180448A1 (en) 2006-01-24 2007-08-02 Citrix Systems, Inc. Methods and systems for providing access to a computing environment provided by a virtual machine executing in a hypervisor executing in a terminal services session
US20070186212A1 (en) 2006-01-24 2007-08-09 Citrix Systems, Inc. Methods and systems for providing access to a computing environment
US20070171921A1 (en) 2006-01-24 2007-07-26 Citrix Systems, Inc. Methods and systems for interacting, via a hypermedium page, with a virtual machine executing in a terminal services session
US20070180280A1 (en) 2006-02-01 2007-08-02 Bolan Joseph E Controlling the allocation of power to a plurality of computers whose supply of power is managed by a common power manager
US20070260417A1 (en) * 2006-03-22 2007-11-08 Cisco Technology, Inc. System and method for selectively affecting a computing environment based on sensed data
US20080104587A1 (en) 2006-10-27 2008-05-01 Magenheimer Daniel J Migrating a virtual machine from a first physical machine in response to receiving a command to lower a power mode of the first physical machine
US20080104608A1 (en) 2006-10-27 2008-05-01 Hyser Chris D Starting up at least one virtual machine in a physical machine by a load balancer
US7856549B2 (en) * 2007-01-24 2010-12-21 Hewlett-Packard Development Company, L.P. Regulating power consumption
US20090031307A1 (en) * 2007-07-24 2009-01-29 International Business Machines Corporation Managing a virtual machine
US7925923B1 (en) * 2008-01-31 2011-04-12 Hewlett-Packard Development Company, L.P. Migrating a virtual machine in response to failure of an instruction to execute
US20090228726A1 (en) * 2008-03-07 2009-09-10 Malik Naim R Environmentally Cognizant Power Management
US20100057529A1 (en) * 2008-09-03 2010-03-04 International Business Machines Corporation Provider-requested relocation of computerized workloads

Non-Patent Citations (16)

* Cited by examiner, † Cited by third party
Title
Hou et al., "A Genetic Algorithm for Multiprocessor Scheduling," IEEE Transactions on Parallel and Distributed Systems, vol. 5, No. 2, Feb. 1994.
Megow et al., "Stochastic Online Scheduling on Parallel Machines," Springer-Verlag Berlin Heidelberg, pp. 167-180, 2005.
MVware Technology Network, "VMware VirtualCenter 1.2 Support Documentation," http://www.vmware.com/support/vc12/doc/c13migrationover.html, 16 pages (at least as early as Oct. 25, 2006).
TheFreeDictionary definition of "Virtual Machine," http://computing-dictionary.thefreedictionaly.com/virtual%20machine, pp. 1-3 (at least as early as Sep. 6, 2006).
U.S. Appl. No. 11/588,607, Final Rejection dated Mar. 3, 2011, pp. 1-18 and attachments.
U.S. Appl. No. 11/588,607, Non-Final Rejection dated Oct. 15, 2010, pp. 1-10 and attachments.
U.S. Appl. No. 11/588,683, Examiner's Answer dated Aug. 3, 2011, pp. 1-24.
U.S. Appl. No. 11/588,683, Final Rejection dated Nov. 25, 2009, pp. 1-19 and attachments.
U.S. Appl. No. 11/588,683, Non-Final Rejection dated Jul. 13, 2010, pp. 1-20 and attachments.
U.S. Appl. No. 11/588,683, Non-Final Rejection dated Jun. 11, 2009, pp. 1-14 and attachments.
U.S. Appl. No. 11/588,691, Final Rejection dated Mar. 31, 2011, pp. 1-18 and attachments.
U.S. Appl. No. 11/588,691, Non-Final Rejection dated Oct. 14, 2010, pp. 1-15 and attachments.
Wikipedia definition for "Hypervisor," http://en.wikipedia.org/wiki/Hypervisor, pp. 1-2 (at least as early as Sep. 6, 2006).
Wikipedia definition of "Virtual machine," http://en.wikipedia.org/wiki/Virtual-machine, pp. 1-6 (at least as early as Sep. 6, 2006).
Wikipedia definition of "Virtual machine," http://en.wikipedia.org/wiki/Virtual—machine, pp. 1-6 (at least as early as Sep. 6, 2006).
YarKhan et al., "Experiments with Scheduling Using Simulated Annealing in a Grid Environment," Springer-Verlag Berlin Heidelberg, pp. 232-242, 2002.

Cited By (74)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130247047A1 (en) * 2008-03-28 2013-09-19 Fujitsu Limited Recording medium having virtual machine managing program recorded therein and managing server device
US20110004878A1 (en) * 2009-06-30 2011-01-06 Hubert Divoux Methods and systems for selecting a desktop execution location
US8560791B2 (en) * 2009-09-14 2013-10-15 Vmware, Inc. Method and system for optimizing live migration of persistent data of virtual machine using disk I/O heuristics
US8527633B2 (en) * 2011-01-06 2013-09-03 International Business Machines Corporation Techniques for addressing geographical location issues in computing environments
US20120179817A1 (en) * 2011-01-06 2012-07-12 International Business Machines Corporation Techniques for addressing geographical location issues in computing environments
US20130138818A1 (en) * 2011-05-26 2013-05-30 Siemens Aktiengesellschaft Method for accessing an automation system and system operating according to the method
US9594579B2 (en) 2011-07-29 2017-03-14 Hewlett Packard Enterprise Development Lp Migrating virtual machines
US9229777B2 (en) * 2011-08-08 2016-01-05 International Business Machines Corporation Dynamically relocating workloads in a networked computing environment
US20130041989A1 (en) * 2011-08-08 2013-02-14 International Business Machines Corporation Dynamically relocating workloads in a networked computing environment
US20130073730A1 (en) * 2011-09-20 2013-03-21 International Business Machines Corporation Virtual machine placement within a server farm
US8825863B2 (en) * 2011-09-20 2014-09-02 International Business Machines Corporation Virtual machine placement within a server farm
US9311159B2 (en) * 2011-10-31 2016-04-12 At&T Intellectual Property I, L.P. Systems, methods, and articles of manufacture to provide cloud resource orchestration
US20130111033A1 (en) * 2011-10-31 2013-05-02 Yun Mao Systems, methods, and articles of manufacture to provide cloud resource orchestration
US8799269B2 (en) 2012-01-03 2014-08-05 International Business Machines Corporation Optimizing map/reduce searches by using synthetic events
US9819694B2 (en) * 2012-06-27 2017-11-14 Qatar Foundation Arrangement configured to migrate a virtual machine in the event of an attack
US20150237066A1 (en) * 2012-06-27 2015-08-20 Qatar Foundation Arrangement configured to migrate a virtual machine in the event of an attack
US8903813B2 (en) 2012-07-02 2014-12-02 International Business Machines Corporation Context-based electronic document search using a synthetic event
US9460200B2 (en) 2012-07-02 2016-10-04 International Business Machines Corporation Activity recommendation based on a context-based electronic files search
US8898165B2 (en) 2012-07-02 2014-11-25 International Business Machines Corporation Identification of null sets in a context-based electronic document search
US8935695B1 (en) * 2012-07-12 2015-01-13 Symantec Corporation Systems and methods for managing multipathing configurations for virtual machines
US9262499B2 (en) 2012-08-08 2016-02-16 International Business Machines Corporation Context-based graphical database
US8676857B1 (en) 2012-08-23 2014-03-18 International Business Machines Corporation Context-based search for a data store related to a graph node
US8959119B2 (en) 2012-08-27 2015-02-17 International Business Machines Corporation Context-based graph-relational intersect derived database
US8620958B1 (en) 2012-09-11 2013-12-31 International Business Machines Corporation Dimensionally constrained synthetic context objects database
US9619580B2 (en) 2012-09-11 2017-04-11 International Business Machines Corporation Generation of synthetic context objects
US9286358B2 (en) 2012-09-11 2016-03-15 International Business Machines Corporation Dimensionally constrained synthetic context objects database
US9069838B2 (en) 2012-09-11 2015-06-30 International Business Machines Corporation Dimensionally constrained synthetic context objects database
US9251237B2 (en) 2012-09-11 2016-02-02 International Business Machines Corporation User-specific synthetic context object matching
US9223846B2 (en) 2012-09-18 2015-12-29 International Business Machines Corporation Context-based navigation through a database
US8782777B2 (en) 2012-09-27 2014-07-15 International Business Machines Corporation Use of synthetic context-based objects to secure data stores
US9741138B2 (en) 2012-10-10 2017-08-22 International Business Machines Corporation Node cluster relationships in a graph database
US8931109B2 (en) 2012-11-19 2015-01-06 International Business Machines Corporation Context-based security screening for accessing data
US9811683B2 (en) 2012-11-19 2017-11-07 International Business Machines Corporation Context-based security screening for accessing data
US9477844B2 (en) 2012-11-19 2016-10-25 International Business Machines Corporation Context-based security screening for accessing data
US8983981B2 (en) 2013-01-02 2015-03-17 International Business Machines Corporation Conformed dimensional and context-based data gravity wells
US9229932B2 (en) 2013-01-02 2016-01-05 International Business Machines Corporation Conformed dimensional data gravity wells
US8914413B2 (en) 2013-01-02 2014-12-16 International Business Machines Corporation Context-based data gravity wells
US9251246B2 (en) 2013-01-02 2016-02-02 International Business Machines Corporation Conformed dimensional and context-based data gravity wells
US20140359778A1 (en) * 2013-01-15 2014-12-04 Empire Technology Development Llc Function-targeted virtual machine switching
US9304795B2 (en) * 2013-01-15 2016-04-05 Empire Technology Development Llc Function-targeted virtual machine switching
US9619468B2 (en) 2013-01-31 2017-04-11 International Business Machines Coporation Generation of synthetic context frameworks for dimensionally constrained hierarchical synthetic context-based objects
US10127303B2 (en) 2013-01-31 2018-11-13 International Business Machines Corporation Measuring and displaying facets in context-based conformed dimensional data gravity wells
US9449073B2 (en) 2013-01-31 2016-09-20 International Business Machines Corporation Measuring and displaying facets in context-based conformed dimensional data gravity wells
US9053102B2 (en) 2013-01-31 2015-06-09 International Business Machines Corporation Generation of synthetic context frameworks for dimensionally constrained hierarchical synthetic context-based objects
US9069752B2 (en) 2013-01-31 2015-06-30 International Business Machines Corporation Measuring and displaying facets in context-based conformed dimensional data gravity wells
US8856946B2 (en) 2013-01-31 2014-10-07 International Business Machines Corporation Security filter for context-based data gravity wells
US9607048B2 (en) 2013-01-31 2017-03-28 International Business Machines Corporation Generation of synthetic context frameworks for dimensionally constrained hierarchical synthetic context-based objects
US9110722B2 (en) 2013-02-28 2015-08-18 International Business Machines Corporation Data processing work allocation
US9372732B2 (en) 2013-02-28 2016-06-21 International Business Machines Corporation Data processing work allocation
US9292506B2 (en) 2013-02-28 2016-03-22 International Business Machines Corporation Dynamic generation of demonstrative aids for a meeting
US10152526B2 (en) 2013-04-11 2018-12-11 International Business Machines Corporation Generation of synthetic context objects using bounded context objects
US9195608B2 (en) 2013-05-17 2015-11-24 International Business Machines Corporation Stored data analysis
US9348794B2 (en) 2013-05-17 2016-05-24 International Business Machines Corporation Population of context-based data gravity wells
US9383986B2 (en) 2013-06-18 2016-07-05 Disney Enterprises, Inc. Safe low cost web services software deployments
US9207976B2 (en) 2013-08-13 2015-12-08 International Business Machines Corporation Management of prioritizing virtual machines in an operating environment
US9213573B2 (en) 2013-08-13 2015-12-15 International Business Machines Corporation Management of prioritizing virtual machines in an operating environment
US20150169349A1 (en) * 2013-12-17 2015-06-18 Vmware, Inc. Location-Based Management of Virtual Machines in Datacenters
CN106133715A (en) * 2014-04-03 2016-11-16 斯特拉托斯卡莱有限公司 Virtual-machine placement based on information from multiple data centers
US20150286493A1 (en) * 2014-04-03 2015-10-08 Strato Scale Ltd. Virtual-machine placement based on information from multiple data centers
EP3126996A4 (en) * 2014-04-03 2017-12-27 Strato Scale Ltd. Virtual-machine placement based on information from multiple data centers
US10331472B2 (en) 2014-08-29 2019-06-25 Hewlett Packard Enterprise Development Lp Virtual machine service availability
WO2016029440A1 (en) * 2014-08-29 2016-03-03 Hewlett-Packard Development Company, L.P. Virtual machine service availability
US9678823B2 (en) * 2014-09-04 2017-06-13 Fujitsu Limited Information processing apparatus, information processing method, and information processing system
US20160070612A1 (en) * 2014-09-04 2016-03-10 Fujitsu Limited Information processing apparatus, information processing method, and information processing system
JP2016053928A (en) * 2014-09-04 2016-04-14 富士通株式会社 Management device, migration control program, and information processing system
US20160170791A1 (en) * 2014-12-10 2016-06-16 University-Industry Cooperation Group Of Kyung Hee University Device for controlling migration in a distributed cloud environment and method for controlling migration using the same
US9524328B2 (en) 2014-12-28 2016-12-20 Strato Scale Ltd. Recovery synchronization in a distributed storage system
US20160188218A1 (en) * 2014-12-31 2016-06-30 Cleversafe, Inc. Synchronizing storage of data copies in a dispersed storage network
US9727427B2 (en) * 2014-12-31 2017-08-08 International Business Machines Corporation Synchronizing storage of data copies in a dispersed storage network
US20170147811A1 (en) * 2015-05-08 2017-05-25 Telefonaktiebolaget L M Ericssn (Publ) Migration service method and module for software modules
US10169578B2 (en) * 2015-05-08 2019-01-01 Telefonaktiebolaget Lm Ericsson (Publ) Migration service method and module for software modules
US10044525B2 (en) * 2015-12-15 2018-08-07 Microsoft Technology Licensing, Llc Scalable tenant networks
US20170170990A1 (en) * 2015-12-15 2017-06-15 Microsoft Technology Licensing, Llc Scalable Tenant Networks
US10261874B2 (en) * 2016-12-01 2019-04-16 International Business Machines Corporation Enabling a cloud controller to communicate with power systems

Similar Documents

Publication Publication Date Title
Li et al. Enacloud: An energy-saving application live placement approach for cloud computing environments
Mastelic et al. Cloud computing: Survey on energy efficiency
Wood et al. Black-box and Gray-box Strategies for Virtual Machine Migration.
Mazzucco et al. Maximizing cloud providers' revenues via energy aware allocation policies
Buyya et al. Energy-efficient management of data center resources for cloud computing: a vision, architectural elements, and open challenges
Corradi et al. VM consolidation: A real case based on OpenStack Cloud
US9323577B2 (en) Automated profiling of resource usage
US9135048B2 (en) Automated profiling of resource usage
Song et al. Adaptive resource provisioning for the cloud using online bin packing
CA2884796C (en) Automated profiling of resource usage
Mishra et al. Dynamic resource management using virtual machine migrations
US8762538B2 (en) Workload-aware placement in private heterogeneous clouds
US8972983B2 (en) Efficient execution of jobs in a shared pool of resources
Dong et al. Energy-saving virtual machine placement in cloud data centers
Liu et al. GreenCloud: a new architecture for green data center
Verma et al. Power-aware dynamic placement of hpc applications
US20120011254A1 (en) Network-aware virtual machine migration in datacenters
Beloglazov et al. Energy efficient resource management in virtualized cloud data centers
Sakellari et al. A survey of mathematical models, simulation approaches and testbeds used for research in cloud computing
US9632835B2 (en) Deployment of virtual machines to physical host machines based on infrastructure utilization decisions
Garg et al. Green cloud computing and environmental sustainability
US8185893B2 (en) Starting up at least one virtual machine in a physical machine by a load balancer
Maheshwari et al. Dynamic energy efficient data placement and cluster reconfiguration algorithm for MapReduce framework
CN102844724B (en) Management of distributed computing system power supply
US9489222B2 (en) Techniques for workload balancing among a plurality of physical machines

Legal Events

Date Code Title Description
AS Assignment

Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GARDNER, ROBERT D;MCKEE, BRET A;WATSON, BRIAN J;AND OTHERS;SIGNING DATES FROM 20071127 TO 20071128;REEL/FRAME:029298/0674

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP, TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.;REEL/FRAME:037079/0001

Effective date: 20151027

FPAY Fee payment

Year of fee payment: 4