JP2010117760A - Virtual machine movement management server and virtual machine movement method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently and safely arrange and move a virtual machine by distributing the loads of an overall system. <P>SOLUTION: The predetection of the resource shortage of a physical machine 120 and the layout and movement of a virtual machine 122 which is proper when it is viewed in the long and middle terms is achieved by recognizing and analyzing change in the load of a virtual machine 122 and the physical machine 120 by an approximate expression such as an equation of higher degree. In addition, the movement of the virtual machine 122 is scheduled in such a manner that the load is prevented from being deviated in the virtual machine 122, the physical machine 120 and the network by recognizing and analyzing the change in the load of the network by the approximate expression such as the equation of higher degree. In particular, a movement schedule is set so that the overlap of the peak of a CPU load in the physical machine 120 or the virtual machine 122 is avoided as much as possible. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for managing the placement and movement of virtual machines in a system that implements a virtual environment.

Conventionally, as described in Patent Document 1, when a plurality of virtual machines are arranged on a plurality of physical machines, the measured values of the performance data of the virtual machines and the physical machines are directly compared, and the optimum performance is obtained. Arrangement was calculated.
JP 2005-115653 A

In such conventional technologies, virtual machines can be placed in consideration of the maximum or minimum performance value, but virtual machines can be placed while reducing or avoiding load bias due to overlapping load peaks. I can't. In general, there is a problem that the range of load fluctuation is large at the peak time, and when the same time machines of different jobs are executed on the same physical machine, the fluctuation of the load further increases and the risk of resource shortage increases.
Further, in the on-demand examination and implementation of virtual machine movement, there is a problem that the movement of the virtual machine occurs at the same time, thereby increasing the network load.
Because of these challenges, the performance of the prior art is never optimal.

  In view of the above circumstances, an object of the present invention is to distribute and distribute the load of the entire system, and to arrange and move virtual machines efficiently and safely.

In the present invention, a virtual machine movement schedule is determined in advance in consideration of a virtual machine, a physical machine, and a change in network load as necessary.
Specifically, the virtual machine, the physical machine, and the network load cycle (time-dependent change) as needed are grasped and analyzed using functions. Based on the basis of the determination of the sum and difference values of these functions, the presence or absence of intersections, etc., real-time detection of physical machine resource shortage and appropriate placement of virtual machines over the medium to long term are realized. In addition, when moving virtual machines, use functions to understand and analyze the load cycles of systems related to movement, such as virtual machines and physical machines, so that loads are not biased (no peaks are created). , Schedule the move.
Details will be described later.

  According to the present invention, it is possible to distribute and distribute the load of the entire system, and to arrange and move virtual machines efficiently and safely.

  Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described in detail with reference to the drawings.

  The present invention is a system that calculates future trends from past results and optimizes future virtualization environments based on these trends. The optimization of the virtual environment is to realize a virtual environment in which there is no shortage of resources and load imbalance. In this system, in order to optimize the virtual environment, the virtual environment is periodically analyzed, physical machines that may run out of resources are pre-detected, virtual machines are properly arranged, and virtual machines are moved. Perform scheduling and so on.

  In addition to using this system alone, it is also possible to use it together with a system that moves virtual machines on demand. Even if there is a concern about a shortage of physical machine resources due to a virtual machine being moved on demand, the system optimizes periodically, so it is possible to avoid resource shortage in advance.

≪Configuration≫
FIG. 1 shows a configuration of a virtual machine migration system (hereinafter sometimes simply referred to as “system”) as an example of the present embodiment. In this embodiment, it is realized by the management server 100 and the physical machine 120 that are communicably connected via the network 140. Further, an HDD (Hard Disk Drive: storage unit) 130 is connected to the management server 100.

  The management server 100 includes a memory (storage unit) 101, a CPU (Central Processing Unit: control unit) 113, a NIC (Network Interface Card) 114, and an I / O (Input / Output) 115.

The memory 101 includes a high-order equation processing unit 103, a movement determination unit 107, and a movement processing unit 110 as processing units that embody execution by the CPU 113.
The high-order equation processing unit 103 includes processing units such as a high-order equation calculation unit 104, a threshold value excess determination unit 105, and a peak overlap determination unit 106.
The migration determination unit 107 includes processing units such as a migration virtual machine candidate selection unit 108 and a migration destination physical machine candidate selection unit 109.
The movement processing unit 110 includes processing units such as a schedule adjustment unit 111 and a movement execution unit 112.
The functions of these processing units will be described later.

  The HDD 130 includes a CPU usage rate measurement data table 131, a coefficient table table 132, a CPU usage rate higher-order equation table 133, a virtual / physical machine correlation table 134, machine threshold values table 135, a peak determination value table 136, and a movement. A schedule table 137 is stored. The management server 100 acquires data in these various tables via the I / O 115 and executes data registration for these various tables.

  The physical machine 120 includes a memory 121, a CPU 123, and a NIC 124. One or more virtual machines 122 are stored in the memory 121. In other words, the physical machine 120 has a function of configuring one or more virtual machines.

In the present embodiment, the usage rate of the CPU 123 is used as data for calculating higher-order equations and determining the necessity and timing of movement of the virtual machine 122, but the usage rate of the memory 121 and the I / O Other data such as O usage rate may be used.
In this embodiment, the high-order equation processing unit 103, the CPU usage rate measurement data table 131, the coefficient table table 132, the CPU usage rate high-order equation table 133, and the machine threshold values table 135 are stored in the HDD 130 of the management server 100. Although stored, these may be stored in the physical machine 120 or another machine.

≪Details of various tables≫
Next, details of various tables stored in the HDD 130 will be described.
2 to 8 show the data structures of the various tables 131 to 137 stored in the HDD 130. FIG.

The CPU usage rate measurement data table 131 illustrated in FIG. 2 is a table that stores data obtained by periodically measuring the CPU usage rate of the virtual machine 122 that uses the CPU 123 of the physical machine 120 as the resource usage rate of the virtual machine 122. . In the CPU usage rate measurement data table 131, the date 1311 on which the month and day for measuring the CPU usage rate are registered, and the CPU usage rate (%) measured at the time determined to be periodically measured on that month and day. ) Is registered as a measured value 1312. The CPU usage rate measurement data table 131 is provided for each virtual machine 122, and the physical machine 120 can measure the CPU usage rate of the virtual machine 122 on the physical machine 120 by a known method. This table is created by transmitting the processed data to the management server 100.
The measurement data registered in the CPU usage rate measurement data table 131 may be that of the virtual machine 122 or the physical machine 120 as described above. Further, the entire virtual machine moving system may be used.

  The coefficient table 132 shown in FIG. 3 is a table that stores coefficients for correcting the performance difference between the physical machines 120 constituting the system. The coefficient table table 132 shows the physical machine 1321 in which a name having a function of identifying the physical machine 120 is registered, the specifications (specification information) of the physical machine 120, and the clock frequency of the CPU used in the physical machine Fields such as a CPU 1322 in which (GHz) is registered and a coefficient 1323 in which a coefficient for correcting a performance difference between physical machines is registered according to the specifications are provided. As a coefficient measurement method registered in the coefficient 1323, for example, there is a benchmark measurement method.

  The CPU usage rate high-order equation table 133 shown in FIG. 4 is a table for storing a CPU usage rate cycle of the virtual machine 122 that is approximated by a high-order equation. In the high-order equation table 133 of the CPU usage rate, the date 1331 in which the day (day of the week), the month, etc. when the CPU usage rate was measured are registered, and the coefficient of each term in the high-order equation is registered. Field 1332 is provided. The data used for calculating the higher order equation is data obtained by correcting the CPU usage rate measurement data table 131 with the coefficient table table 132. Then, based on the data, for example, the higher order equation is calculated using a least square method. Note that the high-order equation registered in the high-order equation table 133 of the CPU usage rate may be that of the physical machine 120.

In the present embodiment, high-order equations of daily units, weekly units, and monthly units are held, but yearly units may be added as needed, or only daily units may be held.
In this embodiment, higher-order equations are used to approximate the CPU usage cycle of virtual machines and physical machines, but other functions such as trigonometric functions, complex functions, and exponential logarithmic functions are used. May be.

  The virtual / physical machine correlation table 134 shown in FIG. 5 is a table that stores the correlation of which physical machine has which virtual machine. Further, this table holds an area for setting a flag for each physical machine and virtual machine. In the virtual / physical machine correlation table 134, a physical machine name 1341 in which a name having a function for identifying a physical machine constituting this system is registered, a physical machine flag 1342 in which a flag to be assigned to the physical machine is registered, The field includes a virtual machine name 1343 in which a name having a function for identifying a virtual machine on the physical machine is registered, and a virtual machine flag 1344 in which a flag to be assigned to the virtual machine is registered.

  In the present embodiment, the purpose of using the physical machine flag 1342 is to set a mark of a virtual machine migration destination candidate (migration destination physical machine candidate) in the migration destination physical machine candidate selection unit 109. When a certain physical machine is determined as a virtual machine migration destination candidate, a flag “1” is registered in the physical machine flag 1342, and a flag “0” is registered otherwise. The purpose of use of the virtual machine flag 1344 is to set a mark of a virtual machine candidate (moving virtual machine candidate) to be preferentially moved by the moving virtual machine candidate selection unit 108. When a certain virtual machine is determined as a moving virtual machine candidate, a flag “1” is registered in the virtual machine flag 1344, and a flag “0” is registered otherwise. In addition to these, these flags may be used for the purpose of setting a mark of a virtual machine whose movement is prohibited or a physical machine scheduled for maintenance.

  The various threshold table 135 of the machine shown in FIG. 6 is a table for storing a threshold indicating an allowable range when a certain purpose relating to the realization of the virtual environment is achieved for each physical machine and virtual machine. Specifically, this table stores the upper limit of the CPU usage rate (%) indicating the usage rate of the resource that may fulfill the purpose of operating (arranging) the virtual machine and the purpose of moving the virtual machine. . Basically, the system does not execute a process that accomplishes a purpose that exceeds this upper limit.

  In the various threshold table 135 of the machine, a machine 1353 in which a name having a function of identifying a physical machine and a virtual machine constituting this system is registered, and a virtual machine in which a CPU usage rate indicating an upper limit on the operation of the virtual machine is registered Fields such as an operation upper limit (first threshold) 1351 and a virtual machine movement upper limit (second threshold) 1352 in which a CPU usage rate indicating an upper limit related to the movement of the virtual machine is registered are provided.

The virtual machine operation upper limit 1351 is set only for the physical machine. The virtual machine operation upper limit is an upper limit of the CPU usage rate of a physical machine that is considered to be able to sufficiently allocate CPU resources (resources) to each virtual machine when the virtual machine is operated on the physical machine. When the value of the virtual machine operation upper limit 1351 is exceeded, it is desirable to move some virtual machines from the physical machine to another physical machine.
The virtual machine movement upper limit 1352 is set for a physical machine and a virtual machine. The virtual machine movement upper limit is an upper limit of the CPU usage rate at which a virtual machine can safely move a virtual machine without affecting other virtual machines on the physical machine. Further, in a virtual machine, the upper limit of the CPU usage rate is considered to be that the virtual machine can be safely moved without affecting the running application or the like. If the value of the virtual machine movement upper limit 1352 is exceeded, the movement of the virtual machine may take time or may fail.

These CPU usage rates are for comparison with the values of higher order equations. Also, when setting these CPU usage rates, it is desirable to set them taking into account the overhead and margin of the physical machine.
In this embodiment, the CPU usage rate is used as the threshold value, but other data such as a memory usage rate and an I / O usage rate may be used.
In this embodiment, only two of the virtual machine operation upper limit 1351 and the virtual machine movement upper limit 1352 are held, but other values such as the number of virtual machines that can be executed simultaneously may be held.

  The peak determination value table 136 shown in FIG. 7 is a table that stores a value indicating which portion of the CPU usage rate cycle expressed by a high-order equation is defined as a load peak. The peak determination value table 136 includes a field such as a determination value 1361 in which a determination value that defines a load peak is registered. In this embodiment, the value of the higher-order equation is defined as a peak when the minimum value is 0 and the maximum value is 1, and the portion is 0.85 or more. That is, the determination value registered in the determination value 1361 is 0.85.

  A migration schedule table 137 illustrated in FIG. 8 is a table that stores a migration schedule of a virtual machine. Specifically, information indicating which virtual machine is moved from which physical machine to which physical machine at what time and how long it takes to move is determined as a movement schedule. Therefore, in the movement schedule table 137, a start time 1371 at which the start time of the movement of the virtual machine is registered, a movement time 1372 at which the movement time of the virtual machine is registered, and a name having a function for identifying the moving virtual machine. A registered virtual machine 1373, a name having a function of identifying a physical machine that is a migration source of the virtual machine, a name of a migration source physical machine 1374 to be registered, and a name having a function of identifying a physical machine that is a migration destination of the virtual machine A field such as a migration destination physical machine 1375 to be registered is provided.

≪Overview of higher-order equations≫
With reference to FIG. 9 to FIG. 11, an outline of the higher order equation calculated by the higher order equation calculation unit 104 will be described. Specifically, a high-order equation of the virtual machine at the CPU usage rate is calculated from the data of the CPU usage rate measurement data table 131. Furthermore, the higher order equation of the physical machine is calculated from the higher order equation of the virtual machine. Not only the CPU usage rate but also the memory usage rate and the I / O usage rate can be calculated in the same procedure as described below.

  FIG. 9 shows a graph plotting measurement data of the CPU usage rate corrected by taking into account the specifications of the physical machine. The CPU usage rate measurement data of the virtual machine 122 is registered in the CPU usage rate measurement data table 131, and the registered data is corrected using the coefficient registered in the coefficient 1323 of the coefficient table table 132. Is made. As a result, the corrected CPU usage rate data plot 201 after the correction is made on a graph including the CPU usage rate (%) axis and the time t axis.

  FIG. 10 illustrates a high-order equation of a virtual machine and its graph. The higher-order equation F (x) 202 is obtained by using, for example, the least square method based on the corrected CPU usage data plot 201. In this embodiment, the maximum CPU usage rate of this virtual machine is 40%, and the graph 203 of the high-order equation of the virtual machine drawn by the high-order equation F (x) 202 is a little before 6 o'clock and 18 o'clock. After a while, it becomes the maximum. Further, the virtual machine can be safely moved from 23 o'clock to around 0 o'clock and around 12 o'clock when the CPU usage rate is 15% (upper limit of virtual machine movement).

  FIG. 11 illustrates a higher-order equation of a physical machine, a higher-order equation of a virtual machine on the physical machine, and a graph thereof. A thick line is a graph 204 of a high-order equation of a physical machine, and a thin line is a graph 203 of a high-order equation of a virtual machine on this physical machine. The sum of the higher order equations of the virtual machine is the higher order equation of the physical machine. In the present embodiment, the movement of the virtual machine on the physical machine is necessary when the CPU usage rate of the physical machine exceeds 85% (the upper limit of the virtual machine operation), and the virtual machine is moved by about 18:00. It has been shown that movement is necessary. Also, the virtual machine can be safely moved from this physical machine, or the virtual machine can be safely moved to this physical machine is from 23 o'clock to around 0 o'clock when the CPU usage rate is 65% or less (virtual machine movement upper limit). It is around 12:00.

The higher order equation calculation unit 104 calculates the higher order equation after correcting the values in the CPU usage rate measurement data table 131 in consideration of the specifications for each physical machine. Therefore, even when there are a wide variety of physical machines in a virtual environment, higher-order equations can be used without worrying about the difference in performance between physical machines.
Further, since the higher order equation of the physical machine is calculated from the higher order equation of the virtual machine, the function of measuring the CPU usage rate of the physical machine is not required. However, the CPU usage rate of the physical machine may be measured, and the higher order equation of the physical machine may be directly calculated without using the higher order equation of the virtual machine.
In addition, since the CPU usage cycle of the virtual machine and the physical machine is held in the high-order equation table 133 of the CPU usage rate as a high-order equation, it is possible to hold data for a longer period than when all the measurement data is held.

  Note that the corrected CPU usage rate plots shown in FIG. 9 and the higher-order equations of the virtual machine or physical machine shown in FIGS. 10 and 11 are obtained from actually measured CPU usage rates. Therefore, the CPU usage rate over a certain period in the past, one day in the graphs of FIGS. However, given the current situation that the operational status of this system rarely changes during a day, these plots and higher-order equations are for a certain period in the future, for example one day (one day for each day of the week). It can be said that this represents a cycle (change with time) of the CPU usage rate that can be realized over one week and one month.

<< Process >>
FIG. 12 is a diagram showing a flow of processing for optimizing the virtualization environment in the virtual machine migration system. In order to optimize the virtualization environment, the CPU 113 periodically detects physical machines that may run out of resources, and examines and implements migration of virtual machines as necessary.
The operation described below is performed on the management server 100 by loading a program (processing unit) stored in the memory (storage unit) 101 of the management server 100 into the storage area and executing it by the CPU (control unit) 113. The processing is executed by each embodied processing unit. Each program may be stored in advance in a storage unit (including the HDD 130), or may be introduced through other storage media or communication media (network or carrier wave propagating through the network) when necessary. .
The process in FIG. 12 starts when a command is issued from the CPU 113 at a fixed time (step S301).

  First, all the physical machines in the virtual / physical machine correlation table 134 are examined by executing a loop process (step S302 to step S310). Through this loop process, physical machines that may run out of resources are detected.

  Next, using the high-order equation of the virtual machine acquired from the high-order equation table 133 of the CPU usage rate, the high-order equation of the physical machine is acquired by the procedure described with reference to FIG. The virtual machine operation upper limit of the physical machine is acquired from the virtual machine operation upper limit 1351 of the table 135 (step S303).

  Next, threshold value excess determination is executed, and it is determined whether or not the acquired higher-order equation of the physical machine exceeds the threshold value indicated as the acquired virtual machine operation upper limit (step S105A). The threshold value excess determination unit 105 outputs a determination result based on the threshold value excess determination (step S304).

  If the determination result does not exceed the threshold indicated by the higher-order equation of the physical machine as the virtual machine operation upper limit (No (not exceeded) in step S304), the physical machine is not likely to run out of resources. The process for this physical machine is terminated (step S310), and loop processing (step S302 to step S310) is performed for other physical machines. If it is performed for all physical machines, the process proceeds to step S112A. On the other hand, if the higher order equation of the physical machine exceeds the threshold indicated as the virtual machine operation upper limit (Yes in step S304), it is determined that this physical machine may run out of resources. The process proceeds to step S108A.

  For a physical machine that has been determined to have a resource shortage, moving virtual machine candidate selection by the moving virtual machine candidate selection unit 108 is executed (step S108A). Next, all virtual machines selected by selecting the moving virtual machine candidate are examined by executing a loop process (step S305 to step S309). Through this loop process, the moving virtual machine is detected.

  Next, migration destination physical machine candidate selection is executed to determine whether or not a migration destination physical machine candidate is found (step S109A). The migration destination physical machine candidate selection unit 109 outputs a determination result based on the migration destination physical machine candidate selection (step S307).

If the determination result is that no migration destination physical machine candidate was found (“failure” in step S307), the corresponding part (record) in the migration schedule table 137 is deleted to cancel the migration of the virtual machine. (Step S306), the process returns to Step S109A to search for another destination physical machine candidate. If there is no migration destination physical machine candidate, it is determined that the virtual machine cannot be moved for the virtual machine, and the process of optimizing the virtualization environment is terminated.
On the other hand, if the determination result is that the migration destination physical machine candidate is found (“success” in step S307), it is determined that the migration destination physical machine candidate has been successfully selected, and the process proceeds to step S111A.

  For a virtual machine that has successfully selected a destination physical machine candidate, schedule adjustment is executed to determine whether adjustment of the schedule for moving the target virtual machine to the destination physical machine candidate is possible (step S111A). . The schedule adjustment unit 111 outputs a determination result by schedule adjustment (step S308).

If the determination result indicates that the schedule cannot be adjusted (“failure” in step S308), for example, if the time overlaps with another schedule, the movement schedule table is used to cancel the movement of the virtual machine. The corresponding part (record) in 137 is deleted (step S306), and the process returns to step S109A to search for another destination physical machine candidate. If there is no migration destination physical machine candidate, it is determined that the virtual machine cannot be moved for the virtual machine, and the process of optimizing the virtualization environment is terminated.
On the other hand, if the determination result indicates that the schedule can be adjusted (“success” in step S308), it is determined that the schedule has been adjusted successfully, and the record remains in the movement schedule table 137, and this virtual machine Is finished (step S309), and loop processing (step S305 to step S309) is performed for the other selected virtual machines. If it is performed for all the selected virtual machines, the process proceeds to step S310. As a result, which virtual machine is moved from which physical machine to which physical machine at what time is registered in the movement schedule table 137.

  Next, the migration of the virtual machine is executed according to the schedule in which the adjustment is successful (step S313). This execution is performed by the movement execution unit 112.

  Finally, when the movement of the virtual machine is completed, the corresponding part (record) in the movement schedule table 137 is deleted (step S311), and the process of optimizing the virtualization environment is completed (step S312).

  Because this process periodically optimizes the virtualization environment, even if the optimization of the virtualization environment breaks down due to on-demand movement of virtual machines, it can be optimized again after a certain period of time, Can realize a stable virtualization environment.

  FIG. 13 to FIG. 18 show the flow of FIG. 12 in further detail, and the flow of each processing unit (105, 106, 108, 109, 111) shown in FIG. 1 (step S105A, step S108A, FIG. Step S109A, Step S111A, and Step S106A in FIG. 15).

  FIG. 13 is a diagram showing a detailed flow of the threshold value excess determination (step S105A). Specifically, a physical machine high-order equation (hereinafter simply referred to as “high-order equation” in the description of FIG. 13) calculated from the data of the high-order equation table 133 of the CPU usage rate represents various threshold values of the machine. It is determined whether or not the threshold (that is, the virtual machine operation upper limit) acquired from the table 135 is exceeded. Note that the higher-order equation and the threshold value are given in advance, and this processing starts when they are acquired (step S401).

  The given threshold value depends on the specifications of the physical machine 120. Therefore, in order to correct the threshold value, the value (that is, the coefficient) registered in the coefficient 1323 of the coefficient table table 132 is multiplied to obtain a new threshold value (step S402).

  Next, a graph of F (x) = [new threshold value] and a graph of G (x) = [higher order equation] are set, respectively, and it is determined whether or not they intersect (step S403). If both intersect (Yes in step S403), it is determined that the higher-order equation exceeds the threshold (step S404), and the present process is terminated (step S406). On the other hand, if the two do not intersect (No in step S403), it is determined that the higher-order equation does not exceed the threshold (step S405), and the present process is terminated (step S406).

  As described above, since the threshold value excess determining unit 105 corrects the threshold value in consideration of the specifications for each physical machine, the higher-order equation and the threshold value can be appropriately compared.

Here, for the following explanation, an outline of overlapping load peaks that have a significant effect on optimization of a virtual environment will be described. The overlap of load peaks means that the vicinity of the maximum of the CPU usage rate that the higher-order equations of a plurality of virtual machines or physical machines have overlap.
FIG. 14 is a diagram for explaining the outline of peak overlap determination, which is determination of whether or not the peak portions of two higher-order equations overlap.

  There is a possibility that the maximum value and the minimum value of two higher-order equations to be subjected to peak overlap determination are different (see 511 and 512). Therefore, the higher order equations are corrected so that the maximum value and the minimum value of each higher order equation are 0.5 and 0, respectively (see 513 and 514). If G (x) = [sum of two higher-order equations after correction], the maximum and minimum values of G (x) are 1 and 0 (bold lines 515 and 516), respectively. If G (x) does not exceed the peak determination value, it is determined that the peaks do not overlap (see 515), and if it exceeds the peak determination value, it is determined that the peaks overlap (see 516). The peak determination value is a value registered in the peak determination value table 136, and the peak overlap determination unit 106 executes the peak overlap determination.

  FIG. 15 is a diagram showing a detailed flow of peak overlap determination for realizing FIG. Specifically, the peaks of two high-order equations (a high-order equation of a physical machine or a high-order equation of a virtual machine) acquired from the high-order equation table 133 of the CPU usage rate may be used. It is determined whether or not the locations overlap. Note that two higher-order equations for analyzing peak overlap are given in advance, and this processing starts when they are acquired (step S501).

  First, in order to clarify the location of the peak of each higher order equation, each higher order equation is corrected (step S502). Specifically, correction (for example, correction of scale conversion) is performed so that F (x) = [higher order equation] becomes 0 ≦ F (x) ≦ 0.5. Note that the domain of x at this time is determined by the daily unit, the week unit, or the like. More specifically, for example, it is determined in units of values registered on the date 1331. The higher-order equation that has been corrected is referred to as a corrected higher-order equation.

  Here, G (x) is set to G (x) = [sum of two corrected higher order equations] (step S503). Subsequently, a peak determination value is acquired from the peak determination value table 136 (step S504), and a threshold value excess determination is performed (step S105A). The threshold excess determination unit 105 outputs a determination result based on the threshold excess determination (step S505). This determination of exceeding the threshold is in accordance with that shown in FIG.

  If the determination result exceeds the determination value that is a threshold value (Yes in step S505), it is determined that the peaks overlap (step S506), and the process ends (step S508). On the other hand, if the determination result does not exceed the determination value which is a threshold value (No in step S505), it is determined that the peaks do not overlap (step S507), and this process is terminated (step S508).

  As described above, the peak overlap determination unit 106 corrects the higher-order equations to be compared, so that it is possible to appropriately determine the overlap of peaks even in two higher-order equations having different maximum values.

  FIG. 16 is a diagram showing a detailed flow of moving virtual machine candidate selection (step S108A). Specifically, it selects which virtual machine should be moved from physical machines that are concerned about resource shortage. At that time, a virtual machine whose peak overlaps with the physical machine is preferentially selected. Note that a physical machine in which resource shortage is a concern is given in advance, and this processing starts when it is acquired (step S601).

  First, in order to acquire information on a virtual machine included in a given physical machine, the virtual / physical machine correlation table 134 is referred to, and a list of virtual machines on the physical machine and a high-order equation table 133 of the CPU usage rate are obtained. Refer to the various threshold tables 135 of the machine to check whether the physical machine resource is still insufficient after moving the virtual machine and any higher-order equations of the physical machine and each virtual machine. Then, the value registered in the virtual machine operation upper limit 1351, that is, the virtual machine operation upper limit is obtained (step S602).

  Next, all virtual machines acquired as the virtual machine list are examined by executing a loop process (step S603 to step S606). Through this loop process, virtual machines that may be able to move are detected.

  The virtual machine to be moved preferably has a physical machine (source physical machine) whose peak overlaps. Therefore, for all the acquired virtual machines, it is determined whether or not the peaks overlap with the physical machines (step S106A). The peak overlap determination unit 106 outputs a determination result based on peak overlap determination (step S604).

  When the determination result indicates that the target virtual machine has a peak that overlaps with the physical machine (Yes in step S604), a flag is set in the virtual machine flag 1344 of the virtual / physical machine correlation table 134 for the virtual machine. Stand up (step S605). On the other hand, when the determination result indicates that the target virtual machine does not overlap with the physical machine (No in step S604), the virtual machine flag 1344 is not set. And if the peak duplication determination is not performed about another virtual machine, it will be performed. If it is performed for all virtual machines, the process proceeds to step S607.

  Next, all virtual machines for which the peak overlap determination has been completed are examined by executing a loop process (step S607 to step S611). Through this loop process, virtual machines that may be able to move are detected. However, the loop processing is preferentially performed from the virtual machine with the flag set.

  First, the target virtual machine is set as a movement virtual machine candidate, the virtual machine (that is, itself) is stored in the movement schedule table 137, and the source physical machine (that is, the physical in which the virtual machine currently exists). Machine) is registered in the migration source physical machine 1374 (step S608).

  Next, for the registered virtual machine, G (x) that satisfies G (x) = [higher order equation of physical machine] − [higher order equation of added virtual machine] is obtained (step S609).

  Next, it is determined whether G (x) exceeds the virtual machine operation upper limit (threshold value) acquired from the virtual machine operation upper limit 1352 of the various threshold values table 135 of the machine (step S105A). The threshold value excess determination unit 105 outputs a determination result based on the threshold value excess determination (step S610). The determination of exceeding the threshold is based on the one shown in FIG. 13 (F (x) is a new threshold obtained by correcting the virtual machine operating upper limit according to the specifications of the physical machine).

  If the determination result exceeds the virtual machine operation upper limit that is the threshold value (Yes in step S610), the processing from step S608 to step S105A is repeated for the other virtual machines. As a result, the maximum value of G (x) gradually decreases, and is basically repeated until the value becomes equal to or less than the threshold value.

  If the determination result does not exceed the virtual machine operation upper limit that is the threshold value (No in step S610), the process ends (step S612). As a result, a virtual machine that is loop-processed while exceeding the threshold is selected as a moving virtual machine candidate.

  In this way, the moving virtual machine candidate selection unit 108 moves preferentially from virtual machines whose peaks overlap with those of the physical machine, and thus can efficiently reduce the maximum value of the physical machine peak. This can also reduce the time load bias of the physical machine.

  In this embodiment, the virtual machine whose peak overlaps with the physical machine is a virtual machine that moves preferentially, but the virtual machine with the slowest peak is a virtual machine that moves preferentially. Priorities may be given.

  FIG. 17 is a diagram showing a detailed flow of selecting a migration destination physical machine candidate (step S109A). Specifically, a physical machine (move destination physical machine) selected by the migration virtual machine candidate selection unit 108 and to which each virtual machine registered in the migration schedule table 137 is selected is selected. At that time, a physical machine whose peak does not overlap with the virtual machine is preferentially selected. Note that the virtual machine to be moved (moving virtual machine) and the migration source physical machine have been acquired in advance from the migration schedule table 137, and this processing starts when these are acquired (step S701).

  First, in order to acquire physical machine information, the virtual / physical machine correlation table 134 is referred to, the physical machine list and the high-order equation table 133 of the CPU usage rate are referred to, and the high level of the moving virtual machine and each physical machine is referred to. In order to confirm in advance that there will be no resource shortage as a result of moving the following virtual equation and the moving virtual machine, refer to the various threshold table 135 of the machine, the value registered in the virtual machine operating upper limit 1351, that is, each physical machine The virtual machine operation upper limit is acquired (step S702).

  Next, all physical machines other than the migration source physical machine acquired as the physical machine list are examined by executing a loop process (steps S703 to S707). Through this loop process, a physical machine that may be a destination is detected.

  As the migration destination physical machine, a physical machine that does not have insufficient resources even after the migration of the migration virtual machine is selected. Specifically, first, G (x) that satisfies G (x) = [higher order equation of physical machine] + [higher order equation of moving virtual machine] is obtained in the target physical machine (step S704).

  Next, it is determined whether or not G (x) exceeds the virtual machine operation upper limit (threshold value) acquired from the virtual machine operation upper limit 1351 of the various threshold table 135 of the machine (step S105A). The threshold value excess determination unit 105 outputs a determination result based on the threshold value excess determination (step S705). Note that this determination of exceeding the threshold is in accordance with that shown in FIG. 13 (F (x) is a new threshold obtained by correcting the virtual machine operation upper limit according to the specifications of the target physical machine).

  If the determination result does not exceed the virtual machine operation upper limit that is the threshold (No in step S705), the physical machine is a candidate for the destination physical machine, and the physical machine in the virtual / physical machine correlation table 134 A flag is set in the use flag 1342 (step S706). On the other hand, if the determination result exceeds the upper limit of the virtual machine operation, which is a threshold (Yes in step S705), the physical machine flag 1342 is not set. Then, if it is not judged that the threshold is exceeded for other physical machines, it is done. If it is done for all physical machines other than the destination physical machine. The process proceeds to step S708.

  Next, in all the physical machines, after confirming the resource shortage after moving the virtual machine, it is determined whether at least one flag is set in the physical machine flag 1342 of the virtual / physical machine correlation table 134 (step S708). ). If no flag is set (No in step S708), it is determined that the selection of the migration destination physical machine has failed because there is no appropriate migration destination physical machine (step S715), and this process ends (step S715). S716).

  On the other hand, if at least one flag is set in the physical machine flag 1342 of the virtual / physical machine correlation table 134 (Yes in step S708), the flow advances to step S709 to select an appropriate destination physical machine. move on.

  Next, the physical machine with the flag set is checked by executing a loop process (step S709 to step S711). Through this loop process, an appropriate destination physical machine is detected. However, the loop processing is preferentially performed from a physical machine having a small maximum value of the high-order equation G (x) = [high-order equation of physical machine] + [high-order equation of moving virtual machine].

  Next, a peak overlap determination is made as to whether or not the peak of the moving virtual machine overlaps for all target physical machines (step S106A). The peak overlap determination unit 106 outputs a determination result based on peak overlap determination (step S710).

When the determination result is that the target physical machine has a peak that overlaps with the moving virtual machine (Yes in step S710), G (x) = [higher order equation of the physical machine] + [high of the moving virtual machine] The maximum value of the following equation], that is, the physical machine with the smallest CPU load is selected as the destination physical machine (step S712), and the process proceeds to step S713.
On the other hand, if the determination result of the determination is that the target physical machine does not overlap with the moving virtual machine (No in step S710), the process directly proceeds to step S713.

  Next, of the physical machines with overlapping peaks, the physical machine with the smallest CPU load or the physical machine with no overlapping peaks is determined as the movement destination physical machine, and the determined movement destination physical machine is moved to the movement schedule table 137. It is registered in the physical machine 1375 (step S713), and since there is an appropriate destination physical machine, it is determined that the destination physical machine candidate has been successfully selected (step S714), and this processing is terminated (step S716).

  In this way, the migration destination physical machine candidate selection unit 109 preferentially selects a physical machine whose peak does not overlap with the virtual machine as a migration destination physical machine candidate, thereby reducing the time load bias of the physical machine. Even if there are no physical machines whose peaks do not overlap, the physical machine having the smallest resource usage rate is set as the migration destination physical machine candidate, so that the load unevenness of each machine can be reduced.

  In this embodiment, the highest priority is that the peaks of the virtual machine and physical machine do not overlap, but other conditions such as a low resource usage rate may be set as the highest priority.

  FIG. 18 is a diagram showing a detailed flow of schedule adjustment (step S111A). Specifically, a new movement schedule is created so that the movement schedule newly registered in the movement schedule table 137 does not interfere with other movement schedules. At this time, a time schedule in which the CPU usage rate of the virtual machine and the physical machine related to the movement is low is selected and a movement schedule is set. Note that the migration virtual machine, the migration source physical machine, and the migration destination physical machine have been acquired in advance from the migration schedule table 137, and this processing starts when these are acquired (step S801).

  First, in order to calculate the time during which the virtual machine can be moved, the CPU usage rate higher-order equation table 133 is referred to, and higher-order equations of the moving virtual machine, the source physical machine, and the destination physical machine are obtained, and various threshold values of the machine are obtained. In order to calculate the values registered in the virtual machine movement upper limit 1352 of the migration virtual machine, the migration source physical machine, and the migration destination physical machine with reference to the table 135, that is, the virtual machine migration upper limit and the time required for the migration of the virtual machine, The file size (capacity) of the virtual machine 122 is acquired from the physical machine 120 (step S802).

  Next, in the virtual machine, source physical machine, and destination physical machine, whether the virtual machine can be moved safely, that is, whether each higher order equation is below the upper limit of virtual machine movement judge. At this time, it is determined including whether they are in the same time zone (step S803). If each higher-order equation does not fall below each virtual machine movement upper limit in the same time zone (No in step S803), it is determined that there is no time zone in which the virtual machine can be moved safely and that the schedule adjustment has failed (step S811). ), This process is terminated (step S812).

  If the higher-order equation falls below the upper limit of each virtual machine movement in the same time zone (Yes in step S803), that is, if there is a time zone during which the virtual machine can be safely moved, the virtual machine file size is obtained from the acquired virtual machine file size. Calculate the time it takes to move.

  Next, it is determined whether or not the time width of the time zone in which the virtual machine can be safely moved with respect to the calculated time is sufficient for moving the virtual machine (step S805). For example, when the movement time of the virtual machine takes 10 minutes, the time zone in which the virtual machine can be moved safely also needs a time width of 10 minutes or more.

When the time width is short with respect to the time required for movement (No in step S805), it is determined that there is no time zone in which the virtual machine can be moved safely, and it is determined that the schedule adjustment has failed (step S811), and this process is terminated. (Step S812).
On the other hand, when the time width of the time zone in which the virtual machine can be safely moved has a sufficient time width with respect to the time required for the movement (Yes in step S805), it is assumed that there is a time zone in which the virtual machine can be safely moved, The start time and movement time of the moving virtual machine for that time zone are registered in the movement schedule table 137 (step S806).

  Next, the movement schedule registered in the movement schedule table 137 and another determined movement schedule (determined movement schedule) are sorted by start time (step S807).

  Next, it is determined whether there is a determined movement schedule for moving the virtual machine at the same time as the registered movement schedule (step S808). If there is no determined movement schedule at the same time (No in step S808), it is determined that the schedule adjustment is successful (step S810), and this process is terminated (step S812).

On the other hand, if there is a determined movement schedule at the same time (Yes in step S808), it is determined whether or not the same physical machine is related in the registered movement schedule and the determined movement schedule (step S809). If the same physical machine is involved (Yes in step S809), the movement will collide, so the registered schedule cannot be executed, and it is determined that the schedule adjustment has failed (step S811), and this process is performed. The process ends (step S812).
Conversely, if the same physical machine is not related (No in step S809), it means that the related physical machines are different even in the movement schedule at the same time, and the movement does not collide, so it is determined that the schedule adjustment is successful. (Step S810), and this process is terminated (Step S812).

  Note that when the movements collide, in the two movement schedules, all or part of the time for moving the virtual machine is the same time, and the same physical machine (at least one of the source physical machine and the destination physical machine) It means using.

In the present embodiment, when the registered movement schedule collides with the determined movement schedule, the registered movement schedule is determined to have failed, and the processing is cancelled. However, it is also possible to recalculate the movement schedule by recalculating the movable time zone of the registered movement schedule or another determined movement schedule.
It is also possible to change the migration destination physical machine and reschedule the migration.

≪Summary≫
According to the present embodiment, it is possible to distribute and distribute the load of the entire system, and to arrange and move virtual machines efficiently and safely.
Basically, the movement schedule is adjusted so as to avoid duplication of load peaks, so the risk of resource shortage is greatly reduced.
Even if on-demand virtual machine migration is examined and implemented, an optimal migration schedule can be established by periodically adjusting the migration schedule. Therefore, it is possible to avoid the simultaneous occurrence of the movement of the virtual machine, and the network load is reduced.

≪Others≫
An example of the embodiment of the present invention has been described above, but the present invention is not limited to this and can be appropriately changed without departing from the spirit of the present invention.

  For example, in the present embodiment, the load on the virtual machine and the physical machine is taken into consideration, but in addition to this, the network load on these may be taken into consideration. In this case, the identification information for identifying the physical machine is not the physical machine name as in this embodiment but the identification information of the corresponding network. Also, instead of using the physical machine specification information, the corresponding network specification information is used. Also, instead of calculating a function that approximately represents the load on the virtual machine, a function that approximately represents the load on the corresponding network is calculated. The load on the network is calculated mainly from the amount of packets passing through the network.

  Moreover, the virtual machine of this embodiment handled what was comprised on one physical machine (for example, refer FIG. 5). However, the present invention can also be applied to virtual machines configured on a plurality of physical machines. In this case, if the virtual / physical machine correlation table 134 of FIG. 5 is provided with a field for registering a numerical value indicating how much the file size of the virtual machine is occupied by each physical machine, It is possible to generate a higher-order equation of the virtual machine through processing such as correcting the CPU usage rate in consideration of not only the physical machine specification information but also the ratio.

  In addition, specific configurations of hardware, software, flowcharts, and the like can be changed as appropriate without departing from the spirit of the present invention.

1 illustrates a configuration of a virtual machine migration system that is an example of the present embodiment. 2 shows a data structure of a CPU usage rate measurement data table. 2 is a diagram illustrating a data structure of a coefficient table. The data structure of the high-order equation table of CPU usage rate is illustrated. 2 is a diagram illustrating a data structure of a correlation table of virtual / physical machines. 2 is a diagram illustrating a data structure of various threshold tables of a machine. 2 is a diagram illustrating a data structure of a peak determination value table. 2 is a diagram illustrating a data structure of a movement schedule table. FIG. 6 is a graph plotting measurement data of CPU utilization, which is corrected in consideration of physical machine specifications. FIG. 2 is a diagram illustrating a higher-order equation of a virtual machine and a graph thereof. FIG. 2 illustrates a higher order equation of a physical machine, a higher order equation of a virtual machine on the physical machine, and a graph thereof. It is a figure which shows the flow of optimization of the virtual environment in a virtual machine movement system. It is a figure which shows the flow of determination exceeding a threshold value. It is a figure explaining the outline | summary of peak duplication determination. It is a figure which shows the detailed flow of peak duplication determination. It is a figure which shows the detailed flow of selection of a movement virtual machine candidate. It is a figure which shows the detailed flow of selection of a movement destination physical machine candidate. It is a figure which shows the detailed flow of a schedule adjustment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Management server 120 Physical machine 122 Virtual machine 104 Higher-order equation calculation part 105 Threshold excess determination part 106 Peak duplication determination part 108 Moving virtual machine candidate selection part 109 Destination physical machine candidate selection part 111 Schedule adjustment part 112 Migration execution part 131 CPU Usage rate measurement data table 132 Coefficient table table 133 High-order equation table of CPU usage rate 134 Virtual / physical machine correlation table 135 Machine various threshold tables 136 Peak judgment value table 137 Movement schedule table

Claims (10)

In a virtual machine migration management server that executes control to connect a plurality of physical machines having one or more virtual machines so as to communicate with each other and move a virtual machine on one physical machine to another physical machine,
Load information associating the load on the virtual machine with the time when the load was measured;
A first threshold value for each physical machine that defines an upper limit of operation related to a load on the physical machine, which allows a sufficient resource to be allocated to the virtual machine when the virtual machine on the physical machine is operated;
A second threshold value for each physical machine or virtual machine, wherein an upper limit of movement related to a load on the physical machine or the virtual machine, which allows the virtual machine to be safely moved;
At least identification information of a migration virtual machine that is a virtual machine to be migrated, identification information of a migration source physical machine that is a physical machine that is the migration source of the virtual machine, and a physical machine that is a migration destination of the virtual machine Movement schedule information in which identification information of a movement destination physical machine and movement time required for movement of the virtual machine determined from the capacity of the virtual machine are associated with each other,
A storage unit for storing
A control for periodically acquiring a load on the virtual machine on the physical machine from the physical machine, and generating the load information based on the acquired load;
Based on the generated load information, control for generating a function for a virtual machine that approximates a change with time of a load on the virtual machine and a function for a physical machine that approximates a change with time of a load on the physical machine;
If there is a physical machine for which the physical machine function exceeds the first threshold, the physical machine is set as the movement source physical machine, and the identification information of the movement source physical machine is registered in the movement schedule information;
Control that registers all or part of virtual machines on the migration source physical machine as candidates for the migration virtual machine, and registers identification information of the migration virtual machine that is the candidate in the migration schedule information;
A first threshold value determined for the physical machine as the migration destination by moving the migration virtual machine, and a sum of the function for the physical machine of the physical machine as the migration destination and the virtual machine function of the migration virtual machine If there is a physical machine that does not exceed the control, the control unit registers the physical machine as a candidate for the migration destination physical machine, and registers identification information of the migration destination physical machine that is the candidate in the migration schedule information.
The physical machine function of the migration source physical machine, the virtual machine function of the migration virtual machine, and the physical machine function of the migration destination physical machine registered in the schedule information are respectively the migration source. There is a time zone that does not exceed the second threshold value set for the physical machine, the second threshold value set for the migration virtual machine, and the second threshold value set for the migration destination physical machine, and the migration If the moving time of the virtual machine is included in the time zone, control to move the moving virtual machine in the time zone;
A control unit for executing
A virtual machine migration management server characterized by comprising: The storage unit
Storing coefficient information in which identification information of the physical machine, specification information of the physical machine, and a coefficient determined based on the specification information are associated with each other;
The controller is
By causing the coefficient of the coefficient information to act on the virtual machine function of the virtual machine on the physical machine, or to act on the physical machine function of the physical machine, the function for the virtual machine or the physical machine The virtual machine migration management server according to claim 1, wherein control for correcting the function is executed. The controller is
If there is a peak included in the function for the virtual machine of the moving virtual machine that overlaps with a peak included in the function for the physical machine of the source physical machine, the virtual machine is a candidate for the moving virtual machine. The virtual machine movement management server according to claim 1, wherein control that is preferentially executed is executed. The controller is
If there is a peak included in the physical machine function of the destination physical machine that overlaps with a peak included in the virtual machine function of the migration virtual machine, a physical machine other than the physical machine is moved to the destination physical. The virtual machine migration management server according to claim 1, wherein control for preferentially determining a machine candidate is executed. The controller is
The travel time included in the new travel schedule information stored in the storage unit overlaps in part or in part with the travel time included in other travel schedule information that has already been determined, and is included in the new travel schedule information At least one of the identification information of the migration source physical machine and the identification information of the migration destination physical machine matches the identification information of the migration source physical machine and the identification information of the migration destination physical machine included in the other migration schedule information that has already been determined. The virtual machine movement management server according to claim 1, wherein the virtual machine movement control based on the new movement schedule information is not executed. In a virtual machine migration method in a virtual machine migration management server for performing control to connect a plurality of physical machines having one or more virtual machines so as to be able to communicate and move a virtual machine on a physical machine to another physical machine,
The storage unit of the virtual machine migration management server is
Load information associating the load on the virtual machine with the time when the load was measured;
A first threshold value for each physical machine that defines an upper limit of operation related to a load on the physical machine, which allows a sufficient resource to be allocated to the virtual machine when the virtual machine on the physical machine is operated;
A second threshold value for each physical machine or virtual machine, wherein an upper limit of movement related to a load on the physical machine or the virtual machine, which allows the virtual machine to be safely moved;
At least identification information of a migration virtual machine that is a virtual machine to be migrated, identification information of a migration source physical machine that is a physical machine that is the migration source of the virtual machine, and a physical machine that is a migration destination of the virtual machine Movement schedule information in which identification information of a movement destination physical machine and movement time required for movement of the virtual machine determined from the capacity of the virtual machine are associated with each other,
Remember
The control unit of the virtual machine migration management server is:
Periodically acquiring a load on the virtual machine on the physical machine from the physical machine, and generating the load information based on the acquired load;
Generating, based on the generated load information, a function for a virtual machine that approximates a change with time of a load on the virtual machine and a function for a physical machine that approximates a change with time of a load on the physical machine;
If there is a physical machine for which the physical machine function exceeds the first threshold, the physical machine is set as the movement source physical machine, and identification information of the movement source physical machine is registered in the movement schedule information;
All or part of virtual machines on the migration source physical machine are candidates for the migration virtual machine, and the identification information of the migration virtual machine that is the candidate is registered in the migration schedule information;
A first threshold value determined for the physical machine as the migration destination by moving the migration virtual machine, and a sum of the function for the physical machine of the physical machine as the migration destination and the virtual machine function of the migration virtual machine If there is a physical machine that does not exceed the physical machine as a candidate of the migration destination physical machine, registering the identification information of the migration destination physical machine that is the candidate in the migration schedule information;
The physical machine function of the migration source physical machine, the virtual machine function of the migration virtual machine, and the physical machine function of the migration destination physical machine registered in the schedule information are respectively the migration source. There is a time zone that does not exceed the second threshold value set for the physical machine, the second threshold value set for the migration virtual machine, and the second threshold value set for the migration destination physical machine, and the migration If the moving time of the virtual machine is included in the time zone, moving the moving virtual machine in the time zone;
The virtual machine moving method characterized by performing this. The storage unit of the virtual machine migration management server is
Storing coefficient information in which identification information of the physical machine, specification information of the physical machine, and a coefficient determined based on the specification information are associated with each other;
The control unit of the virtual machine migration management server is:
By causing the coefficient of the coefficient information to act on the virtual machine function of the virtual machine on the physical machine, or to act on the physical machine function of the physical machine, the function for the virtual machine or the physical machine The virtual machine moving method according to claim 6, wherein the step of correcting the function is executed. The control unit of the virtual machine migration management server is:
If there is a peak included in the function for the virtual machine of the moving virtual machine that overlaps with a peak included in the function for the physical machine of the source physical machine, the virtual machine is a candidate for the moving virtual machine. The virtual machine moving method according to claim 6, wherein the step of preferentially determining is executed. The control unit of the virtual machine migration management server is:
If there is a peak included in the physical machine function of the destination physical machine that overlaps with a peak included in the virtual machine function of the migration virtual machine, a physical machine other than the physical machine is moved to the destination physical. The virtual machine migration method according to claim 6, wherein a step of preferentially determining that the machine is a candidate for a machine is executed. The control unit of the virtual machine migration management server is:
The travel time included in the new travel schedule information stored in the storage unit overlaps in part or in part with the travel time included in other travel schedule information that has already been determined, and is included in the new travel schedule information At least one of the identification information of the migration source physical machine and the identification information of the migration destination physical machine matches the identification information of the migration source physical machine and the identification information of the migration destination physical machine included in the other migration schedule information that has already been determined. The virtual machine movement method according to claim 6, wherein the virtual machine movement step based on the new movement schedule information is not executed.

Images