JP5257709B2 - Virtual computer migration method, virtual computer system, and management server - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce power consumption of an entire computer system including a network apparatus by considering overhead associated with migration of a virtual computer and load change in the network apparatus. <P>SOLUTION: A control server controlling a virtual computer to be allocated to a physical computer comprises: a migration destination determination unit for selecting the virtual computer of a migration target and a physical computer as a candidate of a migration destination of the virtual computer; and a load forecasting unit which, for the physical computer selected as the migration destination candidate, calculates first electric power required for migrating the selected virtual computer, calculates second electric power to be reduced after the virtual computer is migrated to the physical computer selected as the migration destination candidate, then calculates first time when the second electric power to be reduced after the virtual computer is migrated becomes equal to the first electric power. The migration destination determination unit calculates second time when the physical computer can start operating after the virtual computer is migrated to the physical computer of the migration destination candidate, then determines the physical computer as the migration destination of the virtual computer if the second time exceeds the first time. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to operation management of a data center composed of servers, storages, and networks and a wide area network connecting the data centers, and is particularly suitable for performing integrated power saving operation management of the data center and the wide area network. Regarding technology.

  In recent years, with the increase in processing amount and speed of ICT (Information and Communication Technology) systems, consumption of network devices constituting a wide area network (such as WAN) connecting servers, storage, network devices and data centers in a data center. Power is increasing rapidly.

  In order to cope with such an increase in power consumption, a technique has been proposed in which a workload is allocated to IT equipment so as to reduce the total power consumption of IT equipment in a data center, power supply equipment, and cooling equipment (for example, Patent Document 1). In the above prior art, the sum of power consumption of the entire data center is used as an objective function, and an optimization problem of a combination of assigning a work load (virtual computer or business) to a device group such as IT equipment is defined, and the objective function is minimized. By solving the stuffing problem (bin packing problem), an optimal solution that minimizes the objective function or an approximate solution that can be executed in the vicinity of the optimal solution is obtained, and the power consumption of the entire data center is reduced.

JP 2009-252056 A

  In the above conventional example, the virtual machine (business) assigned to the server (physical computer) in the data center is optimized, but the assignment of the virtual machine across the data centers is not optimized. Therefore, in the above conventional example, the power consumption of the network connecting the data centers is not reduced, and the bandwidth of the network device is not taken into account when optimizing the virtual computer assigned to the server.

  However, according to recent survey results, it has been pointed out that about half of the power consumption of ICT systems is occupied by the power consumption of network devices such as WAN routers. This is an essential issue. Furthermore, in cloud computing, which is expected to expand in the future, it is expected that operations across data centers will be widely performed. Therefore, by optimizing the wide-area assignment of work between data centers across networks, it is possible to achieve further power consumption reduction that cannot be achieved by just optimizing the assignment of work to virtual machines in individual data centers. At the same time, it will be required to reduce the power consumption of network devices. In order to realize the above, it is necessary to consider the operation status and power consumption of the network device in the task placement optimization for optimizing the placement of virtual machines assigned to servers.

  Furthermore, due to the recent development of virtualization technology, the movement of work between physical servers when performing the work placement optimization is realized by migration of a virtual machine (hereinafter referred to as VM). It is also necessary to consider the overhead associated with this migration (increased power consumption associated with copying of the main memory and disk area).

  The migration process requires copying of VM main storage and disk area. In particular, in an environment where migration destination physical computers are connected by a WAN, extra power is generated by communication via a network device associated with the copy process, and the power reduction effect due to the optimization of the arrangement of the conventional work is reduced. . Furthermore, when migration of a large number of VMs is required due to the above-described conventional task layout optimization, it takes time to complete the VM migration, and the computer system starts operating with an appropriate task layout with low power consumption. The time is delayed and the effect of optimizing work placement is reduced. Furthermore, when migration occurs too frequently, there is a risk that the increase in power accompanying migration exceeds the power reduction effect reduced as a result of migration.

  In the prior art, the overhead associated with the movement of the VM is not considered. That is, there is a problem in that the above-described conventional technique does not examine the overhead that is caused by the increase in power consumption due to the VM migration and the time delay until the proper business arrangement is realized.

  In particular, in the case of obtaining the optimum business arrangement due to the above-described stuffing problem, which is the mainstream in the past, the continuity of the business arrangement up to the present is not considered, and as a result, there is a risk of generating an arrangement in which VM migration occurs frequently. There is sex.

  When considering the VM migration destination, the total of the future resource usage prediction value of the corresponding VM and the future resource usage prediction value of the migration destination must not exceed the physical resource limit value. . For example, when considering network traffic as the resource usage, not only the NIC at the exit of each physical server but also the virtual NIC assigned to each virtual server, the network line in the data center, Since traffic on the data center exit line may increase, it is necessary to make sure that neither part becomes a bottleneck. Since the predicted value of network usage changes from time to time, the network usage will remain below the bandwidth (maximum effective communication speed) for a while after the VM is migrated. May be expected to exceed. Even in such a case, if the time during which the network usage does not exceed the network bandwidth is long and the power consumption can be sufficiently saved, the power consumption can be saved by temporarily moving the VM to the corresponding physical server. For example, this corresponds to a case where traffic decreases at night due to a change in the load on the VM.

  On the other hand, if the network usage exceeds the bandwidth due to load fluctuations immediately after VM migration, and the work allocation allocated to the virtual server must be restored, the power savings due to migration are small. This is not a good idea because the power required for migration exceeds the power consumption and the power consumption increases by moving the corresponding VM. As described above, in determining the migration destination of the VM, it is necessary to consider how much power can be reduced in the data center in total by migration of the corresponding VM in consideration of a change in the load of the VM. In a computer system that applies the optimal allocation of business in the past, the load status was judged within a fixed time range, so it was not possible to efficiently cope with the physical server load that fluctuated as described above. .

  The present invention considers the overhead of moving a virtual machine when reducing the power consumption by moving a virtual machine that provides a business between physical servers, and considers the fluctuation of the load of the network equipment. An object of the present invention is to provide a control method capable of reducing the power consumption of the entire computer system including the above.

The present invention includes a plurality of physical computers each having a processor and a memory, a network device that connects the plurality of physical computers, and a management server that manages the plurality of physical computers. A virtual computer migration method for executing the virtualization unit that provides the above virtual computer and controlling the virtual computer that the management server allocates to the physical computer so as to reduce power consumption of the plurality of physical computers. the management server includes a first step of selecting the virtual machine to be moved that run on the first physical computer, the management server, a second physics to move the candidate of the selected virtual machine a second step of selecting a computer, the management server, the second physical computer is selected as a candidate of the destination and the source first physical computer For the network devices to be connected, a third step of calculating a first amount of power required to move the virtual machine to which the selected, the management server, a second selected as candidates for the destination For the physical computer, the second power is calculated from the difference between the power that increases after moving the selected virtual computer and the power that is reduced by the first physical computer after moving the virtual computer to be moved. A fourth step of calculating, and a second power amount obtained by multiplying the second power after the management server moves the selected virtual machine by a time is equal to the first power amount. resources and fifth step of calculating the first time, the management server, which after moving the selected virtual machine, the second physical computer is selected as a candidate of the move destination is used There a sixth step of calculating a second time to reach a predetermined limit value, the management server, the second physical computer the second time than the first time, the selection And a seventh step of determining as the second physical computer that becomes the destination of the virtual computer.

  Therefore, the present invention obtains not only the power consumption reduced as a result of the movement of the virtual machine, but also the power consumption required for the movement of the virtual machine, and the total of the computer system including the network device by the movement of the virtual machine. Only when the amount of power consumption is reduced, it is determined to move the process. As a result, it is possible to prevent useless transfer of processing.

1 is a block diagram of a data center computer system according to an embodiment of this invention. It is a block diagram of a management server, showing an embodiment of the present invention. FIG. 5 is a block diagram illustrating a change in a communication path when a virtual computer is moved in a data center computer system according to the embodiment of this invention. It is a flowchart which shows embodiment of this invention and shows an example of the load collection process performed by the management server. It is a flowchart which shows embodiment of this invention and shows an example of VM movement destination determination processing performed with a management server. It is a graph which shows embodiment of this invention and shows an example which determines the balance of the power consumption accompanying the movement of a virtual server. FIG. 10 is an explanatory diagram illustrating an example of determining an operable time after a virtual machine is moved in consideration of a load variation of a network device according to the embodiment of this invention. It is a flowchart which shows embodiment of this invention and shows the detail of the movement destination determination process of FIG. It is explanatory drawing which shows embodiment of this invention and shows an example of monitoring information. An embodiment of the present invention is shown, and an explanatory diagram showing an example of a network management table shows a state before migration. An embodiment of the present invention is shown, and an explanatory diagram showing an example of a network management table shows a state after migration. 1 is a block diagram illustrating an exemplary configuration of a server 1 according to an embodiment of the present invention. It is explanatory drawing which shows embodiment of this invention and shows an example of a load estimated value. It is explanatory drawing which shows embodiment of this invention and shows an example of a virtual server allocation table. FIG. 10 is a block diagram of a computer system of a plurality of data centers according to the second embodiment of this invention.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing an embodiment of the present invention and an example of the configuration of a computer system in a data center to which the present invention is applied.

  Although the present invention can be applied to the determination of the destination of a virtual machine (business) between data centers, the determination of the destination of a virtual machine (hereinafter referred to as a virtual server) can also be applied to the movement of a virtual server within the data center. is there. In the first embodiment, the movement of the virtual server in the data center 100 will be described in order to simplify the description. In the present embodiment, an example is shown in which an application, a service, or a daemon is executed in order for one virtual server to provide one business or service. In the following description, the movement of a virtual server is referred to as business movement.

  In FIG. 1, 100 is a data center, 50 is a WAN, and 10 to 11 are clients. In the data center 100, 140 is a router at the entrance of the data center 100, 150 to 154 are switches in the data center, and 110 to 118 are servers. Servers (physical computers) 110 to 118 are connected to the WAN 50 and the clients 10 and 11 via a router 140 and switches 150 to 154.

  In the data center 100, a management server 120 that manages the servers 110 to 118 and an administrator terminal 190 that performs input and output between the management servers 120 are further installed. The management server 120 controls the servers 110 to 118 and the virtual machine according to the command received from the administrator terminal 190.

  Within each of the servers 110 to 118, virtual servers (hereinafter referred to as VM (Virtual Server)) 110a, 110b, 111a, 111b, 112a, 112b, 113a, 116b, 117a, and 117b that perform business provision are executed. The servers 110 to 118, the router 140, the switches 150 to 154, and the VMs 110 a to 117 b can have any number and configuration in addition to the configuration in FIG. 1.

  In the management server 120, configuration information 121 of the computer system in the data center 100, and operation information collected from the network equipment (router 140, switches 150 to 154), servers 110 to 118, and VMs 110a to 117b in the data center 100. The monitoring information 122 which is the repository is stored in the memory (main memory). The management server 120 is connected to the servers 110 to 118, the switches 150 to 154, and the router 140 via the management network 120a, and acquires operation information. The servers 110 to 118 are connected to the clients (client computers) 10 and 11 via the switches 150 to 154, the router 140, and the WAN 50, and the servers 110 to 118 perform services and operations in response to requests from the clients 10 and 11. provide.

  In FIG. 1, p0 to p3 of the router 140 and the switches 150 to 154 indicate port numbers 0 to 3, respectively.

  FIG. 2 is a block diagram showing an example of the configuration of the management server 120. The management server 120 includes a storage 126 that stores the configuration information 121 and the monitoring information 122, a CPU 130 that performs arithmetic processing, a chip set 123 that connects the CPU 130 to the main memory and an interface, and a NIC that is connected to the management network 120a. (Network interface card) 125 and a computer composed mainly of a main memory 124 for storing data and programs.

  The configuration information 121 includes a configuration information table (not shown) that stores basic information such as the physical configuration of the server (CPU performance, memory, and storage capacity), identifiers of the servers (physical computers) 110 to 118, A virtual server allocation table 1210 for managing the correspondence between the identifiers of the virtual servers 110a to 117b is included. The configuration information 121 is managed by the VM management program (VM management unit) 129 of the management server 120.

  In the main memory 124, a load value collection program 128 that collects load values of the servers 110 to 118 and the router 140, a movement destination determination program (movement destination determination unit) 127 of the present invention, and a movement destination determination program 127. Based on the determination result, each of the future devices (servers) based on the VM management program (VM management unit) 129 that moves the VMs 110a to 117b assigned to the servers 110 to 118 to the servers 110 to 118 and the collected monitoring information 122 110 to 118 and switches 150 to 154 or the router 140), a load prediction program (load prediction unit) 131 for predicting the load is stored and executed by the CPU 130.

  Here, the load value collection program 128 (load monitor unit) for monitoring the loads of the servers 110 to 118 and the network equipment can be applied with a known or well-known technique, and will not be described in detail here. In the load value collection process, for example, monitoring data representing the operating status of each of the switches 150 to 154 and the servers 110 to 118 can be collected by means such as an SNMP manager.

  The VM management program 129 (virtualization management unit) that allocates the physical resources of the servers 110 to 118 to the virtual servers 110a to 117b can use a known or well-known technique. As the virtualization management unit, management software such as a hypervisor or a VMM (Virtual Machine Monitor) may be used, and will not be described in detail here. Further, the VM migration processing for moving the virtual servers 110a to 117b to another server (physical computer) may be a well-known or well-known technique and will not be described in detail here. Each company provides programs for executing VM migration. For example, data used by the VMs 110a to 117b and the VM can be moved between servers by a program such as VCenter provided by VMware.

  A known or well-known technique can also be used for the load prediction program 131 (load prediction unit). For example, each company provides a program for predicting future values from past load value histories using the ARIMA model (Autoregressive Integrated Moving Average: Autoregressive Integrated Moving Average). SPSS etc. is an example). In the present embodiment, a known or well-known technique is employed for the load prediction process, and thus detailed description thereof is omitted.

  The main memory 124 further shows the configuration information of the communication paths of the predicted servers 110 to 118 and the predicted predicted value repository 131a representing the operating status of the future servers 110 and network devices and the virtual servers 110a to 117b predicted by the load prediction program 131. A network management table 127a is stored. The network management table 127a is used by the management server 120 to determine the operating status of each device when the VM is moved.

  Each program executed in the management server 120 is held in the storage 126 and loaded into the main memory 124 as necessary. The storage 126 functions as a storage medium for these programs.

  FIG. 10 is a block diagram illustrating an example of the configuration of the server 110. The server 110 stores a CPU 200 that performs arithmetic processing, a chipset 201 that connects the main memory 202 and interface 204 to the CPU 200, a NIC (network interface card) 203 that is connected to the WAN 50 and the management network 120a, and data and programs. The physical computer is composed mainly of the main memory 202. A storage 210 that holds data 211 and a program 212 is connected to the interface 204. The storage 210 can function as a shared storage that can be accessed by the servers 110 to 118.

  The main memory 202 stores a virtualization unit 220 that allocates physical resources of the server 110 to a plurality of virtual servers (virtual computers), and is executed by the CPU 200. In the example of FIG. 10, an example is shown in which the virtualization unit 220 executes two virtual servers 110 a and 110 b. In the virtual server 110a, the OS 160a is executed, and the application 170a is executed on the OS 160a to provide a business (or service) to the clients 10 and 11. In the virtual server 110b, the OS 160b is executed, and the application 170b is executed on the OS 160b to provide a business (or service) to the clients 10 and 11.

  Here, as the virtualization unit 220, for example, a hypervisor or a VMM (Virtual Machine Monitor) can be employed. The virtualization unit 220 generates, moves, and deletes the virtual servers 110a to 117b in response to a command from the management server 120. Further, when requested by the load value collection program 128 of the management server 120 for operation information, the virtualization unit 220 determines the traffic volume for each flow of the virtual servers 110a and 110b shown in FIG. The CPU usage rate detected by the OS 160a of 110b and the usage rate of the CPU 200 can be returned as operation information.

  FIG. 12 is an explanatory diagram illustrating an example of the virtual server allocation table 1210 managed by the VM management program 129 of the management server 120. The VM management program 129 manages the allocation amount and the like for the virtual servers 110a to 117b to which the physical resources of the servers (physical computers) 110 to 118 are allocated.

  The virtual server allocation table 1210 includes a physical computer NO 1211 that stores the identifier of the physical computer, a virtual computer NO 1212 that stores the identifier of the virtual server, an allocated processor amount 1213 that stores the amount of processors allocated to the virtual server, and a virtual server. One entry from the allocated memory amount 1214 for storing the amount of memory allocated to, the allocated storage amount 1215 for storing the amount of storage allocated to the virtual server, and the application NO 1216 for storing the identifier of the application executed on the virtual server Is configured.

  The VM management program 129 of the management server 120 updates the virtual server allocation table 1210 when the virtual server is generated, moved, or deleted.

  FIG. 3 is a diagram illustrating a method for reducing the power consumption of the entire computer system by migrating VMs that provide business among the servers 110 to 118 by the VM migration destination determination of the present invention in the computer system of FIG. FIG.

  P0 to p3 in FIG. 3 indicate port numbers for input / output by the switches 150 to 154 and the router 140, as in FIG. Here, as an example, a case in which the VM 113a running on the server 4 (113) is migrated to the VM 116a of the server 7 (116) is shown.

  As a result of the migration, there is no VM running on the server 4 (113), and the server is in an idle state. Therefore, it is possible to shut off the power or shift to the power saving mode, and the power consumption of the computer system Can be limited. Further, since the VMs are not operating in the servers 1103 to 115 connected to the switch 4 (152), no communication occurs. For this reason, it is possible to further reduce power consumption by shutting off the power supply or shifting to the power saving mode for the port connected to the server that has been switched to the power shutdown or power saving mode. Note that. For power saving control of a network device such as a switch, for example, a known technique such as JP 2009-33691 A may be used.

  Here, the VM 112a communicates with the VM 110a operating on the server 1 (110) and the client 10 via the network. Regarding the communication path before VM migration, the communication path (flow) between the VM 110a and the VM 113a is indicated by a thick solid line 90 and a two-dot chain line 91, and the communication path between the VM 113a and the client 10 is indicated by a thick solid line 95 and a one-dot chain line 96.

  On the other hand, after migration of the VM 116a, a part of the communication path passes through the thick dotted lines 90 and 95. Specifically, regarding the flow of the VM 110a and the VM 116a, a thick dotted line 92 is used instead of the two-dot chain line 91, and regarding the flow of the client 10 and the VM 116a, a broken line 97 is used instead of the one-dot chain line 96. Accordingly, the traffic volume of the router 140 and the switches 153 and 154 through which the new flow passes increases, while the traffic volume of the switch 4 (152) and the switch 1 (150) decreases.

  FIG. 8 is an explanatory diagram illustrating an example of monitoring information. FIG. 8 shows an example of the result of the management server 120 monitoring the traffic volume for each network flow in the monitoring information 122 according to the present invention.

  The load value collection program 128 of the management server 120 measures the traffic at regular intervals (every 10 seconds in the example in the figure) for each flow (combination of source IP address and destination IP address), and Accumulated in the monitoring information 122. Based on the data of the monitoring information 122, the load prediction program 131 predicts the future operation status.

  The monitoring information 122 includes SourceIP 1221 for storing the source IP address, destIP1222 for storing the destination IP address, and monitoring results 1223-1 to 1223-n for storing the traffic amount between the SourceIP 1221 and the destIP 1222 for each time. One entry is configured. In the illustrated example, in order to simplify the description, an example in which the identifier of the computer is stored instead of the IP address is shown. The monitoring results 1223-1 to 1223-n store the traffic amount for each predetermined time. The traffic volume is expressed by, for example, a data volume per unit time (for example, Mbps). In addition to this, the traffic volume may be expressed by an average value or a maximum value of the data volume at fixed time intervals. In this embodiment, the management server 120 assigns an IP address to each of the virtual servers 110a to 117b. However, the IP address may be manually set from the administrator terminal 190. Since the source IP 1221 and the destination IP 1222 only need to be able to specify the transmission source and the destination, a unique identifier such as a MAC address (or virtual MAC address) or WWN may be used in addition to the IP address.

  The monitoring information 122 is measured by the management server 120 and the network device using a known or well-known monitoring technique. Alternatively, the traffic value collected by the virtualization unit 220 of each of the servers 110 to 118 may be acquired by the load value collection program 128 of the management server 120. When the monitoring information 122 is operation information of each of the virtual servers 110a to 117b, the destIP 1222 is blank, and the usage rate of the CPU detected by the OS 160a on the virtual servers 110a to 117b is monitored every hour. 1 to 1223-n may be stored. When the monitoring information 122 is operation information of each of the servers 110 to 118, destIP 1222 is blank, and the usage rate of the CPU 200 detected by each virtualization unit 220 is the monitoring results 1223-1 to 1223- for each hour. What is necessary is just to store in n.

  9A and 9B are explanatory diagrams illustrating an example of the network management table 127a. FIG. 9A shows a state before migration, and FIG. 9B shows a state after migration.

  The network management table 127a represents, for each flow (combination of source IP address and destination IP address), a communication device through which the flow passes and a port of the communication device. The network management table 127a is managed by the management server 120. The network management table 127a includes a SourceIP 1271 that stores the IP address of the transmission source, a destIP 1272 that stores the IP address of the destination, and ports 1273-1 to 1273 that store the identifiers of the network devices on the communication path between the SourceIP 1271 and the destIP1272. One entry is configured from -n. In the illustrated example, a computer identifier is stored instead of the IP address.

  The network management table 127a can indicate how the flow before migration and the flow after migration have changed.

  The network management table 127a can be created by a known or well-known system configuration management technique.

  By referring to the network management table 127a by an administrator using the administrator terminal 190, each network flow passes through any network device in the data center 100, and the VMs 110a to 117b are migrated, whereby the flow It is possible to know how the network device that passes through changes.

  In the migration shown in FIG. 3, the communication flow between the VM 113a (the VM 116a after migration) and the VM 110a will be described. After the migration, the flow between the VM 116a and the VM 110a goes through the switch 3, the switch 1, the router, the switch 2 and the switch 5 (the port numbers shown in the table are the switches in FIG. (The port number shown on the router). In FIG. 9B, shaded portions in the table after migration are different from those before migration. Therefore, since there is a possibility that traffic will newly increase for the shaded portion, according to the method of the present invention, the period during which the traffic volume after migration does not exceed the hardware limit is determined, and the migration of the VM It is necessary to determine how long it can be operated later.

<Management server processing>
In the following, the VM migration destination determination process according to the present invention will be described with reference to the flowcharts of FIGS.

  4A and 4B are flowcharts illustrating the entire processing performed by the management server 120. FIG. The management server 120 executes the flowchart shown in FIG. 4A at a predetermined short cycle (for example, every 10 seconds), periodically (or as necessary) executes the flowchart shown in FIG. 4B, and the computers in the data center 100 The arrangement of VMs in the computer system is optimized so that the power consumption of the entire system can be reduced. Note that the period for executing the flowchart of FIG. 4B is, for example, 10 minutes.

  First, in step 1001, the management server 120 collects operation information of each server, switch, and router installed in the data center 100 via the management network 120 a by the load value collection program 128 and stores it on the storage 126. Stored as monitoring information 122. As shown in FIG. 8, the collected data is the CPU usage rate of each VM 110a to 117b and the traffic amount of each network flow (a combination of source IP and destination IP). An example of the collected data is shown in FIG. FIG. 8 shows an example in which a time-series change in traffic volume is recorded for each flow on the network.

  It should be noted that the execution cycle of the flowchart of FIG. 4A for collecting load values and the flowchart of FIG. 4B are different, and there are a plurality of processes in the period in which steps 1002 to 1004 of FIG. 4B are performed. The data of the time is collected. For example, even when the VM placement determination process of FIG. 4B is performed once every 10 minutes, as shown in FIG. 8, the load value collection process of FIG. 4A is executed every 10 seconds, for example. The reason is that since VM migration generally takes several tens of seconds to several minutes, it is difficult to change the system configuration following the short cycle even if the placement determination is performed in a short cycle. On the other hand, in collecting load values, it is necessary to monitor load values at a certain time interval in order to determine the placement of VMs in consideration of the influence of jitter and the like accompanying fine load fluctuations. .

  After collecting the load values in step 1001, the management server 120 starts the process of FIG. 4B at a predetermined cycle (every 10 minutes). In step 1002 of FIG. 4B, the management server 120 starts the load prediction program 131 based on the monitoring information 122 accumulated in the process of FIG. 4A, and uses the above-described ARIMA model or the like to load the load that is time-series data. A value pattern is specified, the specified pattern is modeled, a future load value is predicted, and stored in the load predicted value repository 131a on the main memory.

  The data format of the load predicted value repository 131a is the same as the monitoring information 122 of FIG. 8, and the future time-series change of the CPU usage rate of each VM and the traffic amount of each network flow is determined at predetermined time intervals (for example, 10 seconds). This is a table obtained every time.

  FIG. 11 is an explanatory diagram of an example of the load predicted value repository 131a. The predicted load value repository 131a includes a SourceIP 1311 that stores the IP address of the transmission source, a destIP 1312 that stores the IP address of the destination, and a predicted value 1313-1 that stores a predicted value of the traffic amount between the SourceIP 1311 and the destIP1312 every time. ˜1313-n constitute one entry. In the illustrated example, in order to simplify the description, an example in which the identifier of the computer is stored instead of the IP address is shown. Further, when the load predicted value repository 131a is operation information of each of the virtual servers 110a to 117b, the destIP 1312 is blank and the usage rate of the CPU used by the OS 160a on the virtual servers 110a to 117b is the predicted value for each hour. It may be stored in 1313-1 to 1313-n.

  Using the predicted load value repository 131a, the management server 120 selects VMs 110a to 117b to be moved in step 2003. The VM to be moved is selected from the following viewpoints.

(Case 1) When the resource usage amount is predicted to exceed the limit For example, the traffic amount flowing through the link of the network to which a certain VM is connected has reached a predetermined limit value, or the CPU usage rate of the servers 110 to 118 is When the load value predicted by the load prediction value repository 131a reaches the capacity of the physical resource, such as reaching 100%, the VM that uses the physical resource is moved to a server (physical computer) that has a free physical resource. . Also, when the network resource reaches the limit, the VM is moved to a server with available physical resources under a switch with available physical resources. This movement is essential to avoid overloading the system. The network traffic limit is set in advance by the administrator of the data center 100, and is set to an upper limit value of the effective communication speed. Further, the limit of the traffic amount of the network can be set for each network device when the manufacturers and specifications of the network devices such as the router 140 and the switches 150 to 154 are different.

  The VM to be moved in this case is a VM that uses a physical resource that reaches capacity. When a plurality of VMs use the same physical resource, a VM with a small resource usage is selected.

(Case 2) When the resource usage is small For example, when the resource usage rate of the servers 110 to 118 (for example, the usage rate of the CPU 200) is smaller than a preset threshold value, all VMs running on the corresponding server Is moved to a server with sufficient resource usage, and the power of the corresponding server is cut off or shifted to the power saving mode to reduce the power consumption of the entire computer system (in the example of FIG. 3, This applies to the case where the VM 133a placed on the server 4 (113) is migrated to the VM 116a of the server 7 (116).

  In this case, the VMs to be moved are all VMs operating on the corresponding server with a low resource usage rate.

  The target of the VM migration destination determination of the present embodiment is a process of determining to which server 110 to 118 the selected VM is to be moved in the case 2 described above. The VM movement destination determination in case 1 will be described in a later-described modification.

  In Step 1003 of FIG. 4, when the VM to be moved corresponds to the above case 2, the management server 120 determines the movement destination of the VM in Step 1004. FIG. 7 shows detailed processing for determining the migration destination of the VM.

  Prior to the description of the VM movement destination determination process shown in FIG. 7, the power balance due to the movement of the VM will be described with reference to FIG.

  FIG. 5 is a schematic diagram of a future power change when VMs operating on a plurality of servers are aggregated into one server to reduce power consumption. The horizontal axis is time, and time 0 is the current time (the management server determines the migration destination of the VM). The vertical axis is the power consumption of the entire computer system, which is the sum of the power consumption of all servers and the power consumption of all network devices (when optimizing the arrangement between data centers, the power consumption of each data center, It may be the sum of power consumption of network devices on the WAN).

  In FIG. 5, the current power is W0. When VM migration is not performed, the power consumption of the computer system changes at W0.

  Define the following variables:

Power reduced by relocation due to migration: Wb
Time required for migration: t ₀
A period in which power consumption can be reduced in the configuration after VM migration after the placement change trigger (hereinafter referred to as a power reduction realization period): t _f
Electricity required for migration: A
Electricity that can be reduced after migration: B
Power amount that can be reduced from the start of migration to the power reduction realization period t _f : Pm
Next, to reduce the power consumption of the computer system by moving VM,
Pm> 0 (1)
Therefore, the condition of the power reduction realization period t _f that satisfies the conditional expression (1) is obtained. Details of the processing will be described below.

  First, basic migration parameters are calculated. The following basic parameters are calculated based on the contents of migration (the size of the main memory used by the migration target VM, the size of the storage, and the line throughput allocated to the traffic of the VM migration) (the values obtained below are approximate values) ).

Time required for migration: t ₀
t ₀ = (C1 × VM main storage size + C2 × storage size + C3) ÷ line throughput
(2)
Electricity required for migration: A
A = C4 × VM main storage size + C5 × storage size + C6)
× (C8 + C7 × network communication distance) (3)
Here, C1 to C7 are constants determined by the system. t ₀ and A include not only the main storage size of the VM but also the cost of moving data (storage size). For the line throughput, the average value or effective value of the communication speed of the network in the data center 100 can be used. Alternatively, the line throughput may be preset for each network device, and the slowest line throughput may be selected from a plurality of line throughputs.

Power that can be reduced (power saving) as a result of migration: Wb
Wb = Shutdown / Power consumption reduction amount of the server that has shifted to the power saving mode
− Increase in power consumption of servers with increased load
+ C9 × Σ (Traffic reduction amount of each network device) ...... (4)
Here, C9 is a constant determined by the system. Regarding the power of the network device in the last term, it is necessary to calculate the power balance for all the related network devices (through which the migration target VM flow passes) (it becomes a negative value when the traffic increases). ).

Using the above basic parameters, the power reduction realization period t _f that must be operated in the configuration after migration is calculated. First, when the amount of power Pm that can be reduced by migration is calculated, it can be expressed as follows (provided that t _f > t ₀ ):

Pm = BA−Wb × (t _f −t ₀ ) −A (5)
Here, in order to realize power reduction by migration,
Pm> 0
Because
Pm = BA−Wb × (t _f −t ₀ ) −A> 0 (5 ′)
Need to be. Can be obtained power reduction achieved period t _f by solving the above inequality. The load fluctuation, if the period of the power reduction achieved period t _f configuration after movement of VM is not expected to be operational until after, since even by moving the VM can not be reduced in power consumption, the movement of the VM Not performed.

On the other hand, the load change, if the structure after movement of the VM is expected to be operational until the period of the power reduction achieved period t _f, since power consumption can be reduced if moving VM, the movement of the VM Run. That is,
FIG. 6 shows an example of determining how long the operation can be performed in the configuration after the migration of the VM in consideration of the load fluctuation. FIG. 6 shows an example of determining whether the network traffic of the physical network link reaches the limit value of the physical resource, but the determination of whether the CPU usage rate reaches the limit value of the physical resource is the same. It can be implemented by the process.

  FIG. 6 shows the predicted value of the resource usage of the network device through which the flow flows to the moving VM and each destination candidate server (for example, the bandwidth usage of the switch link).

Here, as described in FIG. 9, there are generally a plurality of devices (shaded portions in FIG. 9B) through which a new flow passes after VM migration. Therefore, there are generally a plurality of network usage graphs (for each device through which a flow passes) for one destination candidate. When the network flow goes through multiple devices, if any one device becomes a bottleneck, the system performance will drop, so the time that can be operated in the configuration after migration of VM is the operation obtained by each network device The minimum possible time. Further, the same evaluation is performed with respect to the CPU usage rate, and when the CPU usage rate becomes a bottleneck earlier than the traffic volume, the period during which the CPU usage rate does not become the bottleneck must be set as the operable period _tf1. Don't be. In the following description, the period in which the servers 110 to 118 can be operated without the resource usage reaching the limit value in the configuration after the migration of the VM is set as the operable period t _f1 . In other words, the period during which the migration target VM can be smoothly operated on the server (physical computer) as the migration destination candidate is the operational period t _f1 . That is, when the traffic amount reaches the limit value in the network device connected to the server that is the destination candidate or the usage rate of the CPU 200 of the server reaches the limit value (100%), the response of the application 170a executed by the VM Decreases. For this reason, after the VM is migrated, the period during which smooth operation can be guaranteed is set as the operable period t _f1 . Then, the migrated VM is migrated to the next operable server by the management server 120 when the operable period t _f1 has elapsed.
In FIG. 6, the horizontal axis represents the time after the present, and the vertical axis represents the network usage (for example, Mbps). (A) shown in FIG. 6 shows the predicted value 2001 of the network traffic amount used by the migration target VMx. (B) to (D) shown in FIG. 6 show a case where there are three servers that are destination candidates, and show predicted values of the traffic amount from each destination server.

  (B) shown in FIG. 6 shows the predicted value 2012 of the traffic amount of the VM operating on the server of the movement destination candidate A. A value obtained by adding a predicted value of the traffic amount used by the movement target VMx on the server of the movement destination candidate A is indicated by a one-dot chain line 2011.

In the movement destination determination process of step 1004, the movement destination determination program 127 of the management server 120 uses the predicted traffic volume value used by the VMx to be moved as the predicted value 2012 of the traffic volume of the VM on the server of the movement destination candidate A. The operation possible time t _f1 until the value (total of predicted values) 2011 added with the threshold value (limit value) set in advance is obtained. The management server 120 reads the load prediction value repository 131a, acquires the prediction values 1313-1 to 1313 -n at each time for the corresponding flow, and predicts the traffic amount of the VM operating on the server of the migration destination candidate A. The predicted value 2001 of the traffic amount used by the VMx to be moved is added to the value 2012 every time, and the operable period until reaching a predetermined limit value is obtained as _tf1 . In the case of (B) in FIG. 6, when the migration target VMx is migrated on the server of the migration destination candidate A, the sum of the predicted values of the traffic amount reaches the limit value after 3 hours from now. Therefore, the management server 120 sets the operational period until reaching a predetermined limit value as t _f1 being 3 hours.

The case of (C) in FIG. 6 is the same as (B) above, and the predicted value of the traffic amount used by the migration target VMx is added to the predicted traffic amount 2022 of the VM on the server of the destination candidate B. An operable period t _f1 until the value (total predicted value) 2021 reaches a preset threshold value (limit value) is obtained. In the management server 120, the sum 2021 of the predicted traffic volume reaches the limit value after 2 hours from the present time. Therefore, the management server 120 sets _tf1 as 2 hours as the operable period until the predetermined limit value is reached.

The case of FIG. 6D is the same as (B) described above, and the predicted traffic amount used by the VMx to be moved is added to the predicted traffic amount 2032 of the VM on the destination candidate C server. The operable period t _f1 until the calculated value (total sum of predicted values) 2031 reaches a preset threshold value (limit value) is obtained. In the management server 120, the total sum 2031 of the predicted traffic volume reaches the limit value after 5 hours from the present time. Therefore, the management server 120 sets the operational period until reaching a predetermined limit value as t _f1 being 5 hours.

  When a VM is moved to each destination candidate by migration of the VM, the usage amount (traffic amount) of the network device of the destination candidate is the flow to the moved VM (graph (A) in FIG. 6). Only increase. As shown in FIGS. 6B to 6D, in the graph of the resource usage of the network device through which the flow flows to each destination candidate, the network usage before moving the VM is indicated by a solid line. The network usage after moving the VM is indicated by a one-dot chain line. The solid line graph is a value obtained from the predicted load value repository 131 a of the management server 120. The value of the one-dot chain line with respect to the solid line is obtained by adding the network usage of the migration target VM shown in FIG. 6A to the solid line graph (this value is also obtained from the load predicted value repository 131a). Value.

  Here, in the graph of predicted values of the respective movement destination candidates in FIGS. 6B to 6D, the network bandwidth (limit value) is indicated by a dotted line. As shown in (B) to (D) of FIG. 6, the VM is moved to each destination candidate while the network usage graph (one-dot chain line) after VM migration does not exceed the network bandwidth (limit value). It is possible to operate with a later configuration.

Therefore, in the examples of FIGS. 6B to 6D, the value of the operable period t _f1 is 3 hours for the movement destination candidate A and 2 hours for the movement destination candidate B as described above. In the case of the candidate C, it takes 5 hours. Calculated here determined that the operation period t _f1, by comparing the power reduction achieved period t _f obtained in FIG. 5, whether it can move the VM to be migrated to the server of the destination candidate applicable it can.

That means
t _f <t _f1 (6)
What is necessary is just to select the movement destination candidate which becomes. Furthermore, the management server 120 increases the time until the next migration by selecting the destination candidate having the maximum operable period _tf1 among the destination candidates that satisfy the above formula (6). The frequency can be suppressed.

In the above example, whether or not the resource usage reaches the limit value is used as a criterion for obtaining a period in which the configuration after VM migration can be operated. In addition to the above, the following is conceivable, and the following must also be taken into account when calculating the operable period _tf1 .

・ The time when the load on the computer system is further reduced and the number of servers (physical computers) is expected to be further reduced. ・ System configuration such as when the server is reserved by the user or when maintenance by the administrator is scheduled. When it is necessary to review again-In addition, when it is known in advance that the server allocation needs to be changed, the operation possible period t _f1 after migration may be obtained by taking these schedules or schedules into consideration.

  FIG. 7 is a flowchart showing details of the VM migration destination determination processing in step 1004 of FIG. 4B. First, the management server 120 acquires a predicted value of the network resource usage of the migration target VM from the predicted load value repository 131a (step 1101).

  Next, the management server 120 acquires a set of servers (physical computers) that are VM migration destination search candidates obtained in step 1003 (step 1102). The servers that are the migration destinations of VMs basically target all the servers in the data center 100, but servers that are not suitable for migrating VMs are excluded in advance according to the following selection conditions.

-Exclude servers that violate the constraints of placement. For example, if multiple VMs are distributed and created on multiple servers 110-118 to improve reliability (redundancy), these VMs must be the same. Do not place on a physical server.

-Exclude servers with long response times (large RTT) Avoids failing to secure response times for applications on the VM being moved. The response times of the servers 110 to 118 may be set in the configuration information such as the virtual server allocation table 1210.

-Exclude servers with large amounts of data movement Do not move database servers that use large amounts of data. Avoid excessive time required for data transfer during migration.

  After obtaining a set of servers that are migration destination candidates, the management server 120 executes the processing of step 1105 for all migration destination candidate physical servers.

That is, first, according to the procedure described in FIG. 5, when the VM is moved to the corresponding server, the management server 120 obtains a time t _f when the power balance of the VM movement is positive (step 1105-1).

Furthermore, when the VM is moved to the server from the load prediction value repository 131a, the management server 120 acquires the resource usage prediction value of the network device through which the flow of the migration target VM newly passes (Step 1105- 2). Here, the flow information of the migration target VM is obtained from the network management table 127a. Thereafter, according to the procedure described with reference to FIG. 6, an operable period t _{f1 in} which the server can be operated is obtained until the resource usage reaches the limit value in the configuration after migration of the VM (step 1105-3).

Here, when there are a plurality of network devices through which the flow of the migration target VM newly passes, the management server 120 needs to obtain the operable period t _f1 for each network device and select the minimum value among them. (Furthermore, it is necessary to consider the operational period t _f1 obtained from the CPU usage rate).

After determining the operation possible time t _f1, the management server 120, and can be operational time t _f1, compares the power reduction achieved period t _f (step 1105-4), the operation period t _f1 is power reduction achieved period _If it is smaller than tf, it is determined that the VM cannot be moved to the corresponding server because the power cannot be reduced by moving the VM (step 1105-7). On the other hand, if the operation period t _f1 larger than the power reduction achieved period t _f is possible to move the VM to the appropriate server (step 1105-5). In this case, the amount of power Pm1 that can be reduced when the server is operated until the operable period _{tf1 is} obtained by the following equation described with reference to FIG. 5 (step 1105-6).

Pm1 = B-A = Wb × (t f1 -t 0) -A ...... (7)
If there is no movable server after performing the above processing (1105-1 to 1105-7) for all the migration destination candidate physical servers, the VM is not migrated (step 1106, 1108). On the other hand, if there is a server that can be moved, Pm1 of each destination candidate is compared, and it is determined that the VM is moved to the destination server that has the largest Pm1 (step 1107).

  With the above processing, the destination of the VM can be determined in consideration of the fluctuation state of the network traffic and the power balance due to the movement of the VM due to the VM migration.

<Modification 1>
In the above embodiment, when determining whether a VM can be moved, the difference between the amount of power (B) that can be reduced by migration and the amount of power (A) that is necessary for migration is positive (a little power is required for migration). It was possible to reduce it).
Pm = B−A> 0 (5 ′)
However, considering the error in approximate calculation of the above parameter calculation and the risk associated with VM migration (system failure occurs if migration fails for some reason), it is a good idea to perform migration when the amount of power reduction is small is not. Therefore, the minimum value C of the power reduction amount due to the movement of the VM is defined,
Pm = B−A> C (8)
Only in this case, a method of performing VM migration is conceivable. Thereby, it is possible to prevent frequent migration with almost no power reduction amount. The constant C is a threshold value determined by the operation policy of the computer system in the data center 100, and is a value input from the administrator terminal 190 by the system administrator.

<Modification 2>
In the above-described embodiment and the first modification, the VM migration destination determination process is shown in which the power consumption is reduced by consolidating the VMs into one server (case 2 in step 1003 in FIG. 4). On the other hand, the load of the computer system increases, and the migration destination of the VM when the VM that has been performed by one server (physical computer) is distributed to a plurality of servers (1 in the case of step 1003) This can be determined by applying the procedure of the embodiment.

In this case, since the number of devices to be processed increases, the total power consumption of the computer system in the data center 100 increases. Therefore, parts can not be applied determines whether migration by the above-described power reduction achieved period t _f. However, the power amount Pm1 that can be reduced when the server is operated by the server according to the procedure described in Step 1105-6 shown in FIG. 7 for each destination server is calculated (Pm1 becomes negative), and Pm1 is the maximum. By moving the VM to the server (the absolute value is minimum), an increase in power consumption accompanying the movement of the VM can be minimized.

Or in the said embodiment, even if VM after migration reaches the operation possible period _tf1 , it can search a migration destination again by this modification 2.

<Modification 3>
FIG. 13 is a third modification of the present invention, and shows a block diagram of a computer system when determining the placement of VMs between a plurality of data centers.

  In the above embodiment and the first and second modified examples, the method of determining the migration destination of the VM in one data center 100 has been described. The migration destination of a VM across a plurality of data centers can also be determined by the same process as described above. FIG. 13 shows the overall architecture.

  In FIG. 13, 600, 700, and 800 are data centers, 500 is a WAN, and 510 and 511 are clients. Data centers 600, 700, and 800 are connected to clients 510 and 511 through WAN 500. The configuration of the data centers 600, 700, and 800 is the same as that shown in FIG.

  In the data center, taking 600 as an example, 640 is a router at the data center entrance (switches in the data center are omitted), 610, 611 and 612 are servers (physical computers), and 620 is a management server. In each of the servers 610, 611, and 612, virtual servers (hereinafter referred to as VMs) 610a, 610b, 611a, 611b, 612a, and 612b that provide business (or service) to the clients 510 and 511 are executed.

  The management server 620 has configuration information 621 and monitoring information 622 as in FIG. Routers 551 to 554 are installed in the WAN 500 to connect the data centers 600, 700, and 800 and the clients 510 and 511. A management server 520 is also placed in the WAN 500. Between the management servers of each data center 600, 700, 800, and WAN 500, an inter-management server network 590 is arranged and can access each other. Each management server 620, 720, 820 of the data center is connected to a server in each data center via a management network (not shown), as in FIG.

  FIG. 13 shows an example in which the VM 710a of the data center 2 (700) is moved to the VM 810a of the data center 3 (800).

  In the VM migration destination determination between data centers, the management server of the data center (720 in this example) where the migration target VM is operating basically determines the migration destination. Compared with the VM migration destination determination in the data center, the following is different.

  The management server of each data center has a server in the own data center, configuration information of the network equipment (721 in the case of the management server of the data center 2), and monitoring information (722). The configuration and operation information of devices placed in other data centers access information (621, 622, 821, 822) held by other management servers via the management server network 590.

  The WAN 500 is provided by a carrier different from the data center. Therefore, it is difficult for the management server 620 of the data center 600 to directly determine the amount of power that can be reduced in the WAN 500 when the VM is moved. In order to solve the above, regarding the reduction of power consumption of the WAN 500 part, when the management server (720) of the migration source of the VM inquires the management server 520 of the WAN 500 and the flow path of the VM to be moved is changed An interface for inquiring about the power reduction amount Wb2 is provided in the management server 520 of the WAN 500.

  An example of an inquiry interface is as follows.

query_power_difference (flow flow rate, current WAN entrance, current WAN exit, WAN entrance after VM migration, WAN exit after VM migration)
When the VM flow after the movement does not pass through the WAN 500 as a result of the movement of the VM, empty arguments (NIL, NULL, etc.) are given to the entrance and the exit.

By using the above inquiry result in the calculation of the electric power Wb reduced by the movement of the VM shown in FIG. 5, it is possible to determine the movement destination of the VM in consideration of the electric power of the WAN 500 portion that is a black box. That is, the power Wb that is reduced in each data center (700, 800), the amount of power B multiplied by the time t _f to the sum of the power consumption Wb2 is reduced by WAN500 by changing the communication path, to move the VM Therefore, it may be equal to or greater than the amount of electric power A.

  In the example of FIG. 13, since the VM 710a of the data center 2 (700) is moved to the VM 810a of the server 810 of the data center 3 (800), the management server 720 reduces the power wb-1 that is reduced in the data center 2 (700). Ask for. The management server 720 inquires of the management server 820 about the power Wb-2 to be reduced (or increased) in the data center 3 (800). Further, the management server 720 inquires of the management server 520 about the power Wb-3 to be reduced (or increased) in the WAN 500. And the management server 720 can obtain | require the sum total of the acquired said electric power Wb-1 to Wb-3 as electric power Wb reduced by movement of VM.

<Summary>
As described above, according to the present invention, in the optimization of the arrangement of VMs inside and outside the data center,
・ VM movement overhead (power required for VM movement)
It is possible to select a destination server (physical computer) in consideration of network device usage and load fluctuations.

  As described above, even in a computer system with a load fluctuation, it is possible to determine an appropriate VM migration destination in consideration of the resource usage of devices including network devices.

  As described above, the present invention can be applied to a computer system that performs migration of a virtual machine and a management server of the computer system.

50 WAN
DESCRIPTION OF SYMBOLS 100 Data center 110-118 Server 110a-117b Virtual server 121 Configuration information 122 Monitoring information 127 Movement destination determination program 127a Network management table 128 Load value collection program 129 VM management program 131 Load prediction program 131a Load prediction value repository 140 Router 150-154 switch

Claims (12)

A plurality of physical computers each including a processor and a memory; a network device that connects the plurality of physical computers; and a management server that manages the plurality of physical computers, wherein the physical computer includes one or more virtual computers A virtual machine moving method for controlling the virtual machine assigned to the physical computer so that the management server reduces power consumption of the plurality of physical machines,
A first step in which the management server selects the virtual machine to be moved running on a first physical computer ;
A second step in which the management server selects a second physical computer that is a candidate for a destination of the selected virtual computer;
Said management server, information on the network device that connects the movement source of the first physical computer and the second physical computer is selected as a candidate of the destination, required to move the selected virtual machine A third step of calculating a first amount of power;
For the second physical computer selected as the destination candidate, the management server increases the power after moving the selected virtual computer and the first physical computer after moving the virtual computer to be moved. A fourth step of calculating the second power from the difference from the power reduced by the physical computer of
The management server calculates a first time at which a second power amount obtained by multiplying the second power after moving the selected virtual machine by the time is equal to the first power amount. And the steps
After the management server moves the selected virtual machine, it calculates a second time until the amount of resources used by the second physical computer selected as the migration destination candidate reaches a predetermined limit value. A sixth step,
It said management server, and a seventh step of the second time to determine the second physical computer exceeding the first time, the second physical computer to move the virtual machine to which the selected,
A method for moving a virtual machine, comprising: A method of moving a virtual machine according to claim 1 ,
The resource amount is
A method of moving a virtual machine, comprising: a predicted value of a network traffic amount used by the second physical computer after the selected virtual machine has been moved to the selected second physical computer . A method of moving a virtual machine according to claim 1,
The seventh step includes
For each of the second physical computers for which the second time exceeds the first time, a step of calculating a third amount of electric power reduced by moving the virtual computer to the second time after moving the virtual computer When,
Selecting a second physical computer that maximizes the third amount of power as a destination of the virtual computer;
A method for moving a virtual machine, comprising: A method of moving a virtual machine according to claim 1,
The seventh step includes
A method of moving a virtual computer , wherein a second physical computer in which a difference between the second time and the first time exceeds a predetermined threshold is determined as a transfer destination of the selected virtual computer . A method of moving a virtual machine according to claim 1,
The network device is:
A first router connected to a first physical computer group including a first physical computer that operates the virtual computer to be moved;
A second router connected to a second physical computer group including a second physical computer that is a destination candidate, and
A second network connecting the first router and the second router;
The management server
A first management server for managing a first physical computer group under the first router;
A second management server for managing a second physical computer group under the second router;
A third management server for managing power consumption of the second network,
The first management server communicates with the second management server via the second network, and the virtual machine running on the first physical computer under the first router is connected to the second management server. Move to a second physical computer under the router,
The first step includes
The first management server selects the virtual machine to be moved that runs on the first physical computer,
The second step includes
The second management server selects, from the second computer group, a second physical computer that is a candidate for a migration destination of the virtual computer,
The fourth step includes
After the first management server moves the virtual computer to the second computer group selected as the migration destination candidate, the first management server reduces the first physical computer under the first router. An eighth step for determining power;
A ninth step in which the management server makes an inquiry to the second management server that calculates power that increases after the selected virtual machine is moved to the second physical computer selected as the destination candidate;
After the selected virtual machine is moved to the second physical computer of the second computer group selected as the migration destination candidate, the power reduced in the second network is transferred to the third management server. A tenth step for the management server to query
The first management server obtains the sum of the power obtained in the eighth step, the power inquired in the ninth step, and the power inquired in the tenth step as the second power. And the steps
A method for moving a virtual machine, comprising: A plurality of physical computers each having a processor and memory;
  Network equipment connecting the plurality of physical computers;
  A management server including a processor and a memory for managing the plurality of physical computers, wherein the physical computer executes a virtualization unit that provides one or more virtual computers, and the management server includes the plurality of physical computers. A virtual computer system that controls the virtual computer assigned to the physical computer so as to reduce power consumption of the computer,
  The management server
  A migration destination determination unit that selects the migration target virtual machine operating on the first physical computer and selects a second physical computer that is a candidate for the migration destination of the selected virtual computer;
  For the network device that connects the second physical computer selected as the migration destination candidate and the first physical computer that is the migration source, the first power amount required to move the selected virtual computer is calculated. For the second physical computer that is calculated and selected as the destination candidate, the power that increases after moving the selected virtual computer and the first physical computer after the virtual computer to be moved is moved A second power amount obtained by calculating the second power reduced by the computer and multiplying the second power after moving the selected virtual computer becomes equal to the first power amount. A load prediction unit that calculates the time of 1;
  The destination determination unit
  After the selected virtual machine is moved by the physical computer selected as the migration destination candidate, the amount of resources used by the second physical computer selected as the migration destination candidate reaches a predetermined limit value. And calculating a second physical computer whose second time exceeds the first time as a destination of movement of the selected virtual computer. . The virtual machine system according to claim 6,
The resource amount is
A virtual computer system, which is a predicted value of a traffic amount of a network used by the second physical computer after the selected virtual computer is moved to the selected second physical computer . The virtual machine system according to claim 6 ,
The destination determination unit
For each of the second physical computers for which the second time exceeds the first time, calculate a third power amount that is reduced by moving the virtual computer until the second time, A virtual computer system , wherein a second physical computer having the maximum third electric energy is selected as a destination of the virtual computer. The virtual machine system according to claim 6 ,
The destination determination unit
A virtual computer system , wherein a second physical computer in which a difference between the second time and the first time exceeds a predetermined threshold is determined as a migration destination of the selected virtual computer. A management server that includes a processor and a memory and controls a virtual computer assigned to the physical computer so as to reduce power consumption of a plurality of physical computers;
  A migration destination determination unit that selects the migration target virtual machine operating on the first physical computer and selects a second physical computer that is a candidate for the migration destination of the selected virtual computer;
  For a network device that connects the second physical computer selected as the migration destination candidate and the first physical computer that is the migration source, the first power amount required to move the selected virtual computer is calculated. For the second physical computer selected as the migration destination candidate, the power that increases after moving the selected virtual computer and the first physical computer after moving the virtual computer to be moved The second power amount obtained by multiplying the second power after calculating the second power to be reduced by the above and moving the selected virtual machine becomes equal to the first power amount. A load prediction unit that calculates the time of
  The destination determination unit
  After the selected virtual machine is moved by the second physical computer selected as the migration destination candidate, the amount of resources used by the second physical computer selected as the migration destination candidate is a predetermined limit. A second time until the value is reached is calculated, and a second physical computer whose second time exceeds the first time is determined as a migration destination of the selected virtual computer. Management server. The management server according to claim 10,
  The resource amount is
  A management server, which is a predicted value of a traffic amount of a network used by the second physical computer after the selected virtual computer has been moved to the selected second physical computer. The management server according to claim 10,
The destination determination unit
For each of the second physical computers for which the second time exceeds the first time, calculate a third power amount that is reduced by moving the virtual computer until the second time, A management server , wherein a second physical computer that maximizes the third power consumption is selected as a migration destination of the virtual computer .