JP2016062140A - Virtual equipment management device, virtual equipment management method, and virtual equipment management program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To restore a virtual machine safely.SOLUTION: A virtual equipment management device has a determination part, a deletion part, and a reconstitution part. The determination part determines whether there is a restoring process being performed for a virtual machine when trouble of the virtual machine or trouble of physical equipment operating the virtual machine is detected. The deletion part deletes correspondence between the virtual machine and a storage region used by the virtual machine or deletes the correspondence and a virtual machine instance of the virtual machine as a first restoring process when there is not the restoring process being performed. After the physical equipment operating the virtual machine is selected, and the correspondence is deleted or the correspondence and the virtual machine instance are deleted, the reconstitution part makes the selected physical equipment form the virtual machine again as a second restoring process.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a virtual device management apparatus, a virtual device management method, and a virtual device management program.

  Examples of IaaS (Infrastructure as a Service) type cloud services include Amazon Elastic Compound Cloud (web site, http://aws.amazon.com/ec2), Rackspace Cloud Server (web site, http: //www.rackspacecloud .com / cloud-hosting-products / servers /).

  Amazon uses a proprietary platform as the foundation of the IaaS cloud service, while Rackspace uses OpenStack (web site, http://www.openstack.org/) of Open Source.

  However, the IaaS platform such as OpenStack is mainly targeted at providing primitive API (Application Programming Interface) for managing virtual devices, and management of physical devices is out of scope, so service providers provide cloud services. It is necessary to consider when doing so.

  Specifically, OpenStack does not particularly support recovery when a physical device on which a virtual device operates fails, and a countermeasure is required by the service provider. As a method adopted in the city, there is a method in which an HA configuration is constructed using a high availability cluster software, such as a packager, and a failover occurs when a physical device fails.

  In addition, in a physical device that operates a virtual machine, there is a method of detecting the down of the virtual machine by using Ping or monitoring a virtual machine control library such as Libvirt. When such a physical device detects that the virtual machine is down, it executes a recovery process for recreating the virtual machine.

Pacemaker web site, [August 27, 2014 search], Internet (URL: http://www.linux-ha.org/wiki/Pacemaker/) Libvirt web site, [searched August 27, 2014], Internet (URL: http://libvirt.org/) D. Mane, "Building A High Availability-Openstack," International Journal of Engineering Research and Applications, Vol.3, Issue 4, pp.269-277, July 2013.

  However, the above-described conventional technique has a problem that the virtual machine may not be safely recovered.

  Specifically, recovery when a physical device fails and recovery when a virtual machine goes down are independent. Also, the recovery means when the failure of the physical device and the failure of the virtual machine overlap is not unified. For this reason, for example, if virtual machine recovery due to a virtual machine down is performed before the forced termination of the failed node at the time of failover due to a physical device failure, the virtual machine can be started twice and the volume can be destroyed There is sex.

  The disclosed technology has been made in view of the above, and aims to safely recover a virtual machine.

  The virtual device management apparatus disclosed in the present application includes a determination unit, a deletion unit, and a re-creation unit. When a failure of a virtual machine or a failure of a physical device that operates the virtual machine is detected, the determination unit determines whether there is a recovery process being performed on the virtual machine. The deletion unit deletes the association between the virtual machine and the storage area used by the virtual machine as the first restoration process when there is no restoration process in progress, or the association Delete the virtual machine instance of the virtual machine. The re-creation unit selects a physical device that operates the virtual machine and deletes the association, or after deleting the association and the virtual machine instance, the selected physical device as a second recovery process To recreate the virtual machine.

  In the virtual device management method disclosed in the present application, the virtual device management apparatus includes a determination step, a deletion step, and a recreation step. In the determination step, when a failure of the virtual machine or a failure of a physical device that operates the virtual machine is detected, it is determined whether there is a recovery process being performed on the virtual machine. The deletion step deletes the association between the virtual machine and the storage area used by the virtual machine as the first restoration process when there is no restoration process in progress, or Delete the virtual machine instance of the virtual machine. In the re-creation process, after selecting the physical device that operates the virtual machine and deleting the association, or after deleting the association and the virtual machine instance, the selected physical device is used as a second recovery process. To recreate the virtual machine.

  The virtual device management program disclosed in the present application causes a computer to execute a determination procedure, a deletion procedure, and a re-creation procedure. The determination procedure determines whether there is a recovery process in progress for the virtual machine when a failure of the virtual machine or a failure of a physical device that operates the virtual machine is detected. The deletion procedure deletes the association between the virtual machine and the storage area used by the virtual machine as the first restoration processing when there is no restoration processing in progress, or Delete the virtual machine instance of the virtual machine. In the re-creation procedure, a physical device that operates the virtual machine is selected, and after the association is deleted, or after the association and the virtual machine instance are deleted, the selected physical device is used as a second recovery process. To recreate the virtual machine.

  According to one aspect of the disclosed virtual device management apparatus, there is an effect that the virtual machine can be safely recovered.

FIG. 1 is a diagram illustrating an example of a configuration of a virtual device management system according to the first embodiment. FIG. 2 is a diagram for explaining virtual device creation processing by the virtual device placement scheduler function unit. FIG. 3 is a diagram for explaining virtual device rearrangement processing by the virtual device placement scheduler function unit. FIG. 4 is a diagram for explaining virtual device rearrangement processing by the virtual device placement scheduler function unit. FIG. 5 is a diagram for explaining a processing operation for creating a virtual device in the virtual device management system. FIG. 6 is a diagram for explaining a processing operation for rearranging virtual devices in the virtual device management system. FIG. 7 is a diagram for explaining a processing operation for rearranging virtual devices in the virtual device management system. FIG. 8 is a diagram for explaining the virtual device placement scheduler DB and the virtual device placement scheduler function unit realized by the virtual device management apparatus. FIG. 9 is a diagram illustrating an example of the data structure of the virtual device arrangement information table. FIG. 10 is a diagram illustrating an example of the data structure of the physical resource information table. FIG. 11 is a diagram for explaining the processing operation by the placement destination selection unit. FIG. 12 is a diagram for explaining a processing operation for detecting a failure of a physical device by the failure determination unit. FIG. 13 is a diagram for explaining the processing operation by the re-creation unit. FIG. 14 is a flowchart illustrating a processing procedure performed by the virtual device placement scheduler function unit when a physical device failure notification is received. FIG. 15 is a flowchart illustrating a processing procedure performed by the virtual device arrangement scheduler function unit when a virtual machine down notification is received. FIG. 16 is a diagram illustrating a computer that executes a virtual device management program.

  Hereinafter, embodiments of a disclosed virtual device management apparatus, virtual device management method, and virtual device management program will be described in detail based on the drawings. The invention disclosed by this embodiment is not limited.

(First embodiment)
FIG. 1 is a diagram illustrating an example of a configuration of a virtual device management system according to the first embodiment. As illustrated in FIG. 1, the virtual device management system includes a user terminal 101, a physical device 103a, a physical device 103b, a physical device 103c, a cloud controller 108, and a virtual device management apparatus 109. In the case where the physical device 103a, the physical device 103b, and the physical device 103c are not distinguished, they are described as the physical device 103. The “physical device” referred to here is a physical server, a storage device, a network device, or the like that can create a virtual device. The case where it is is demonstrated. Further, the number of physical devices 103 included in the virtual device management system is not limited to the number shown in FIG. 1 and can be arbitrarily changed.

  The user terminal 101 is a terminal used by the user, and requests the virtual device management apparatus 109 to create a virtual device according to a user instruction.

  The physical device 103 receives a virtual device creation or deletion request from the cloud controller 108 and creates or deletes an actual virtual device. For example, the physical device 103 receives an instruction to create a virtual device from the cloud controller 108 and creates a virtual device.

  For example, the physical device 103a has a virtual machine control unit (not shown), and creates a virtual machine 104a and a virtual machine 105a. For example, the physical device 103b has a virtual machine control unit (not shown), and creates a virtual machine 104b and a virtual machine 105b. Similarly, the physical device 103c has a virtual machine control unit (not shown), and creates a virtual machine 104c and a virtual machine 105c. In the physical device 103a to the physical device 103c, the virtual machine control unit is realized by a “Nova” function in the case of OpenStack, for example.

  If the physical device 103 is a storage device, for example, the physical device 103 has a virtual volume control unit and creates a virtual volume. When the physical device 103 is a network device, the physical device 103 includes a virtual network control unit, and creates a virtual L2 network, a virtual router, a virtual load balancer, and the like. Note that the virtual network control unit is realized by a “Netron” function in the case of OpenStack, for example.

  Such operation states of the physical device 103 include four states of “in operation”, “standby”, “failing (maintenance)”, and “failing (recovery processing)”. “In operation” indicates that the physical device is in operation. “Reserved” indicates that the physical device is provided as a standby system and is not in operation. “Failure (during maintenance)” indicates that recovery processing of a physical device in which a failure has occurred has not started. This “failing (maintenance)” state is not a state in which a complete stop of a physical device in which a failure has occurred is guaranteed. “During failure (during recovery process)” indicates that the recovery process of the physical device in which the failure has occurred is started. The state of “failing (during recovery process)” is a state in which a complete stop of the physical device in which the failure has occurred is guaranteed. In the virtual device management system, a “reserve” physical device may not be provided.

  In addition, the physical device 103 defines a physical resource capacity that can be used to arrange virtual devices in accordance with the physical resource capacity. Here, the physical resources include, for example, a physical memory, a CPU (Central Processing Unit), a network port, and the like. Since virtual machines have different memory sizes according to flavors (specifying virtual machine specifications), the capacity of physical resources used differs depending on the virtual machine to be created. However, for convenience of explanation, it is assumed below that the capacity of physical resources used is the same for all virtual devices. The capacity of the physical resource used for arranging one virtual device is defined as one unit and is called “one space”. In other words, one virtual device can be arranged in one space, and when one virtual device is created, one of the physical device spaces is consumed.

  In addition, it is assumed that the state of the space in the physical device is managed by three types of “free”, “in use”, and “failure buffer”. Here, “free” indicates a space where no virtual device is arranged. “In use” indicates a space in which virtual devices are arranged. The “failure buffer” indicates a space reserved for failure recovery.

  The physical device 103a includes a virtual machine monitoring module 106a. Similarly, the physical device 103b includes a virtual machine monitoring module 106b, and the physical device 103c includes a virtual machine monitoring module 106c. Note that the virtual machine monitoring modules 106a to 106c are described as the virtual machine monitoring module 106 when they are not distinguished. The virtual machine monitoring module 106 operates on the physical device 103. For example, when a virtual machine failure is detected by a module for monitoring events such as Libvirt, the virtual machine monitoring module 106 notifies the virtual device placement scheduler function unit 111 described later of the failure of the virtual machine.

  In addition, the physical device 103a includes high availability software 107a. Similarly, the physical device 103b includes high availability software 107b, and the physical device 103c includes high availability software 107c. Note that the high availability software 107a to 107c is described as the high availability software 107 when not distinguished from each other. As this highly available software 107, for example, “Pacemaker” or the like can be used. The high availability software 107 detects a failure of the physical device 103 and notifies the virtual device management apparatus 109 of the failure of the physical device. In such a case, the physical device 103 receives an instruction to relocate the virtual device from the cloud controller 108, and relocates the virtual device disposed in the physical device 103 in which the failure has occurred.

  Note that “Pacemaker” has a highly reliable failure detection mechanism, and a countermeasure for split brain has been established. “Pacemaker” detects the split brain state (isolated state) based on the majority rule by the Quorum module or the like.

  The cloud controller 108 is connected to the physical device 103 and the virtual device management apparatus 109. The cloud controller 108 is a device having a CPU (Central Processing Unit), a memory, a data holding area, and a network communication function. The cloud controller 108 receives a virtual device creation request or deletion request from the virtual device management apparatus 109 via an API (Application Programming Interface), and creates or deletes a virtual device based on the received creation request or deletion request. 103 is instructed. For example, the cloud controller 108 is OpenStack or the like.

  The virtual device management apparatus 109 is connected to the user terminal 101, the physical device 103, and the cloud controller 108 via a network (not shown). The virtual device management device 109 is a device having a CPU, a memory, a data holding area, and a network communication function. For example, as shown in FIG. 1, the virtual device management device 109 is a virtual device placement scheduler DB (Data Base). 110 and a virtual device arrangement scheduler function unit 111. When the virtual device management apparatus 109 receives a virtual device creation request from the user terminal, the virtual device management apparatus 109 determines a virtual device placement destination and requests the cloud controller to create a virtual machine. When the virtual device management apparatus 109 receives a physical device failure notification from the high availability software 107 and a virtual machine down notification from the virtual machine monitoring module 106, the virtual device management apparatus 109 uses the information in the virtual device placement scheduler DB 110 to perform recovery processing. Determine the type of. Then, the virtual device management apparatus 109 determines a virtual device placement destination and performs recovery processing such as a virtual machine creation request to the cloud controller.

  The virtual device arrangement scheduler DB 110 is, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The virtual device arrangement scheduler DB 110 stores virtual device arrangement information and physical resource information. The virtual device arrangement information is information indicating on which physical device each virtual machine is arranged and the progress of processing of each virtual machine. The physical resource information is information indicating the operating state of each physical device and the free capacity of the physical resource that the physical device has. Information indicating the operating state of the physical device includes, for example, operating, standby, failure (maintenance), and failure (recovery processing). Information indicating the free capacity of the physical resource includes, for example, free, There is a fault buffer in use. Details of the virtual device arrangement information will be described later with reference to FIG. 9, and details of the physical resource information will be described later with reference to FIG.

  The virtual device arrangement scheduler function unit 111 arranges virtual devices according to business requirements with reference to the operating state of the physical device and the usage state of the physical resource of the physical device. For example, when a virtual device creation request is received from the user terminal 101, the virtual device placement scheduler function unit 111 uses the virtual device creation request and information in the virtual device placement scheduler DB 110 to mediate the creation of the virtual device. To do. Here, the virtual device placement scheduler function unit 111 determines a physical device to place a virtual device in a normal operation of creating a new virtual device such as a virtual machine or a virtual router, and designates the physical device to the cloud controller 108. Request to create a virtual device.

  In the virtual device management system configured as described above, the virtual device management apparatus 109 operates in cooperation with the virtual machine monitoring module 106 and the high availability software 107. Processing in the virtual device management system will be described with reference to FIGS.

  FIG. 2 is a diagram for explaining virtual device creation processing by the virtual device placement scheduler function unit 111. FIG. 2 illustrates a case where virtual machines VM # 11 to VM # 16, VM # 21 to VM # 26, and VM # 31 to VM # 36 are created as virtual devices in three physical devices 103a to 103c. It is assumed that all three physical devices 103a to 103c are in operation. In the example of FIG. 2, the virtual device is a virtual machine, but the virtual device may be another virtual device such as a virtual router.

  As illustrated in FIG. 2, the virtual device placement scheduler function unit 111 selects the physical device 103a as the placement destination of the virtual machines VM # 11 to VM # 16, and physically selects the physical device 103a as the placement destination of the virtual machines VM # 21 to VM # 26. The device 103b is selected, and the physical device 103c is selected as the placement destination of the virtual machines VM # 31 to VM # 36. Then, the virtual device placement scheduler function unit 111 requests the cloud controller 108 for placement. In other words, the virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to create the virtual machines VM # 11 to VM # 16 in the physical device 103a. Further, the virtual device placement scheduler function unit 111 requests the cloud controller 108 to create the virtual machines VM # 21 to VM # 26 in the physical device 103b, and assigns the virtual machines VM # 31 to VM # 36 to the physical device 103c. The cloud controller 108 is requested to create it. It should be noted that the placement logic may be set by each operator according to business requirements, and may be distributed as much as possible or concentrated as much as possible. When it is distributed as much as possible, it is convenient for the performance of the user's virtual device, and when it is concentrated as much as possible, the physical devices that are actually used can be reduced, which is convenient for reducing the operating cost of the operator.

  Further, for example, when a failure occurs in any of the physical devices 103, the virtual device placement scheduler function unit 111 mediates the rearrangement of the virtual devices using information in the virtual device placement scheduler DB 110. Here, the virtual device placement scheduler function unit 111 rearranges virtual devices at the time of failure recovery in cooperation with the high availability software 107 and the cloud controller 108. FIG. 3 is a diagram for explaining virtual device rearrangement processing by the virtual device placement scheduler function unit 111. FIG. 3 illustrates a case where a failure occurs in the physical device 103b in which the VM # 21 to VM # 26 are arranged as virtual devices among the three physical devices 103a to 103c that are operating.

  As illustrated in FIG. 3, the virtual device placement scheduler function unit 111 detects that a failure has occurred in the physical device 103b. In such a case, the virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to clear the virtual machine. Then, the cloud controller 108 clears the virtual machine. Note that the clearing of the virtual machine here means that the association between the virtual machine and the storage area used by the virtual machine is deleted. In other words, the clearing of the virtual machine means deleting the association between the virtual machine and the storage.

  Subsequently, the virtual device arrangement scheduler function unit 111 determines relocation destinations of the virtual machines VM # 21 to VM # 26 using information in the virtual device arrangement scheduler DB 110. In the example illustrated in FIG. 3, the virtual device arrangement scheduler function unit 111 selects the physical device 103a as the relocation destination of the VM # 21, VM # 23, and VM # 25, and the VM # 22, VM # 24, and VM The physical device 103c is selected as the relocation destination of # 26.

  Then, the virtual device placement scheduler function unit 111 requests the cloud controller 108 to rearrange. That is, the virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to create the virtual machines VM # 21, VM # 23, and VM # 25 in the physical device 103a. Further, the virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to create the virtual machines VM # 22, VM # 24, and VM # 26 in the physical device 103c.

  As a result, the physical device 103a reconstructs the virtual machines VM # 21, VM # 23, and VM # 25, and in addition to the virtual machines VM # 11 to VM # 16, the virtual machines VM # 21, VM # 23, And VM # 25 are arranged. Further, the physical device 103c reconstructs the virtual machines VM # 22, VM # 24, and VM # 26, and in addition to the virtual machines VM # 31 to VM # 36, the virtual machines VM # 22, VM # 24, and VM # 26 is arranged.

  As described above, when a failure occurs in the physical device 103b, the virtual device placement scheduler function unit 111 converts the virtual machines VM # 21 to VM # 26 that are placed in the physical device 103b to the physical device 103a and the physical device 103c. Rearrange to In other words, since the virtual device placement scheduler function unit 111 uses a plurality of physical devices as virtual device recovery destinations, it is possible to shorten the virtual device recovery time when a physical device fails. The operator can also set the logic to place virtual machines on as few physical devices as possible even in the event of a failure. In that case, you may set up logic that fills from the node with the least available space. Alternatively, logic that fills from the node with the most available space may be set.

  Further, for example, when a failure occurs in any of the virtual machines 104, the virtual device placement scheduler function unit 111 mediates the rearrangement of the virtual machines using information in the virtual device placement scheduler DB 110. Here, the virtual device placement scheduler function unit 111 rearranges virtual machines at the time of failure recovery in cooperation with the virtual machine monitoring module 106 and the cloud controller 108. FIG. 4 is a diagram for explaining virtual machine relocation processing by the virtual device arrangement scheduler function unit 111. FIG. 4 shows a case where the virtual machine VM # 21 that is operating the physical device 103b is down. It is assumed that all three physical devices 103a to 103c are in operation.

  As illustrated in FIG. 4, the virtual device placement scheduler function unit 111 detects that a failure has occurred in the virtual machine VM # 21 that is operating the physical device 103b. In such a case, the virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to delete the virtual machine instance. Then, the cloud controller 108 deletes the virtual machine instance. In addition, in the cloud controller such as OpenStack, the virtual machine instance scheduler API 111 deletes the association between the virtual machine and the storage by executing the virtual machine instance deletion API. The cloud controller 108 is requested to delete the virtual machine instance and release the association between the virtual machine and the storage.

  Subsequently, the virtual device arrangement scheduler function unit 111 determines the relocation destination of the virtual machine VM # 21 using information in the virtual device arrangement scheduler DB 110. In the example illustrated in FIG. 4, the virtual device arrangement scheduler function unit 111 selects the physical device 103b as a relocation destination of the VM # 21. Then, the virtual device placement scheduler function unit 111 requests the cloud controller 108 to rearrange. That is, the virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to create the virtual machine VM # 21 in the physical device 103b. As a result, the physical device 103b reconstructs the virtual machine VM # 21.

  As described above, when a failure occurs in the virtual machine VM # 21 operated by the physical device 103b, the virtual device placement scheduler function unit 111 deletes the virtual machine instance and then transfers the virtual machine VM # 21 to the physical device 103b. Rearrange.

  As described above, in the virtual device management system, not only the virtual machine can be recovered when a physical device fails, but also the virtual machine can be recovered when the virtual machine is down. Further, in the virtual device management system, when a plurality of physical device failure notifications or virtual machine down failure notifications are duplicated, the virtual machine is safely recovered. After performing the virtual machine instance cleanup step on the device, a new virtual machine is placed.

  Next, processing operations in such a virtual device management system will be described with reference to FIGS. FIG. 5 is a diagram for explaining a processing operation for creating a virtual device in the virtual device management system.

  As shown in FIG. 5, the user terminal 101 transmits a virtual device creation request to the virtual device placement scheduler function unit 111 (step S1). The virtual device arrangement scheduler function unit 111 refers to the virtual device arrangement scheduler DB 110 (step S2) and confirms physical resource information (step S3). That is, the virtual device arrangement scheduler function unit 111 refers to the virtual device arrangement scheduler DB 110 to acquire physical device availability information. As a result, the virtual device placement scheduler function unit 111 determines the physical device 103 as the placement destination for creating the virtual device based on the business requirements, and prepares the API parameter (step S4).

  Next, the virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to cause the determined physical device 103 to create a virtual device (step S5). For example, the virtual device arrangement scheduler function unit 111 calls the cloud controller API by designating the decided arrangement destination. Subsequently, the cloud controller 108 requests the designated physical device 103 to create a virtual device (step S6).

  Then, the physical device 103 creates a virtual device (step S7), and notifies the cloud controller 108 that the creation of the virtual device is completed (step S8). Subsequently, the cloud controller 108 notifies the virtual device arrangement scheduler function unit 111 that the creation of the virtual device has been completed (step S9). Then, the virtual device arrangement scheduler function unit 111 notifies the user terminal 101 that the creation of the virtual device has been completed (step S10).

  FIG. 6 is a diagram for explaining a processing operation for rearranging virtual devices in the virtual device management system. FIG. 6 illustrates an operation in which the virtual device placement scheduler function unit 111 mediates the rearrangement of virtual devices when a failure occurs in any physical device 103. As shown in FIG. 6, in the virtual device management system, the physical device 103a, the physical device 103b, and the physical device 103c mutually monitor the device state (steps S21 and S22). For example, the high availability software 17 of the physical device 103a, the physical device 103b, and the physical device 103c exchanges device states with each other using Heartbeat. Hereinafter, a case where a failure occurs in the physical device 103a will be described.

  Here, when a failure occurs in the physical device 103a, the high availability software 107a on the physical device 103a stops the process on the physical device 103a and notifies the virtual device arrangement scheduler function unit 111 of the failure. Similarly, the physical device 103b and the physical device 103c notify the virtual device arrangement scheduler function unit 111 of the failure of the physical device 103a. Here, when the physical device 103a has completely failed, the physical device 103a cannot notify the virtual device placement scheduler function unit 111 of the occurrence of the failure, but the physical device 103b and the physical device 103c are not connected to the physical device 103a. A failure can be notified to the virtual device arrangement scheduler function unit 111. Therefore, the virtual device arrangement scheduler function unit 111 can know the failure of the physical device 103a. In the example illustrated in FIG. 5, a case where the physical device 103 a completely fails and the occurrence of a failure cannot be notified from the physical device 103 a to the virtual device arrangement scheduler function unit 111 is illustrated.

  It is described as “own node failure notification” that a physical device in which a failure has occurred notifies the virtual device placement scheduler function unit 111 of the failure, and another physical device different from the physical device in which the failure has occurred is a virtual device. Notifying the placement scheduler function unit 111 of a failure is referred to as “other node failure notification”. Further, since the own node failure notification has low reliability, the virtual device placement scheduler function unit 111 receives the own node failure notification, and the “operating state” of the physical device in which the failure has occurred is “operating” or “standby "" Is changed to "Failure (maintenance)", otherwise "Operating status" is not changed. Further, when the virtual device placement scheduler function unit 111 receives the failure notification of its own node, it is not guaranteed that the physical device in which the failure has occurred is completely stopped, and there is a possibility that the virtual machine may be double-started. , Do not start the recovery process. However, the virtual device placement scheduler function unit 111 changes the “operating state” to “failing (during maintenance)” to prevent a new virtual machine from being placed. On the other hand, when the virtual device arrangement scheduler function unit 111 receives the other node failure notification, the “operating state” of the physical device in which the failure has occurred is “operating”, “standby”, and “failing (maintenance)”. If this is the case, change it to “Failure (during recovery process)”. Here, the other node failure notification guarantees the stop of the failed node by STONIT or the like described later. For this reason, the virtual device arrangement scheduler function unit 111 enters the recovery process when receiving the other node failure notification.

  As illustrated in FIG. 6, the physical device 103b notifies the virtual device placement scheduler function unit 111 that a failure has occurred in the physical device 103a (step S23). Then, the virtual device arrangement scheduler function unit 111 receives another node failure notification from the physical device 103b. Here, the virtual device arrangement scheduler function unit 111 refers to the virtual device arrangement scheduler DB 110 to check the operating state of the notification physical device 103a, and the operation state of the notification physical device 103a is “in operation”. The operating state “operating” of the device 103a is changed to “failing (during recovery process)” (step S24). Then, the virtual device arrangement scheduler function unit 111 sends an ACK (ACKnowledgement) response to the physical device 103b (step S25). Then, the virtual device arrangement scheduler function unit 111 starts a recovery process. Similarly, the physical device 103c notifies the virtual device arrangement scheduler function unit 111 that a failure has occurred in the physical device 103a (step S26). Then, the virtual device placement scheduler function unit 111 refers to the virtual device placement scheduler DB 110 to confirm the operating state of the notification physical device 103a, and the operation state of the notification physical device 103a is “failing (during recovery process)”. Therefore, it is determined as retransmission (step S27). Then, the virtual device arrangement scheduler function unit 111 returns an ACK to the physical device 103c (step S28). As described above, the virtual device arrangement scheduler function unit 111 starts the virtual device recovery process according to the first received notification, but also responds with an ACK to the second and subsequent notifications.

  Then, in the failure recovery process, the virtual device placement scheduler function unit 111 first requests the cloud controller 108 to clear all virtual machines operating on the physical device 103a (step S29). Subsequently, the cloud controller 108 deletes the association between the virtual machine and the storage on the DB of the cloud controller 108 (step S30), and notifies the virtual device placement scheduler function unit 111 of the completion of the virtual machine clear (step S30). S31).

  The virtual device arrangement scheduler function unit 111 refers to the virtual device arrangement scheduler DB 110 (step S32) and confirms physical resource information (step S33). Thereby, the virtual device arrangement scheduler function unit 111 acquires the free capacity of the physical resources of the physical device 103b and the physical device 103c, determines the physical device 103 on which the virtual device is to be relocated, and prepares an API parameter (step S34). ). Here, the virtual device arrangement scheduler function unit 111 enables a high-speed recovery by selecting a plurality of physical devices 103 as a recovery destination of the virtual device.

  Next, the virtual device placement scheduler function unit 111 requests the cloud controller 108 to cause the physical device 103b to create a virtual device to be rearranged (step S35). Subsequently, the cloud controller 108 requests the physical device 103b to create a virtual device (step S36). Similarly, the virtual device placement scheduler function unit 111 requests the cloud controller 108 to cause the physical device 103c to create a virtual device to be rearranged (step S37). For example, the virtual device arrangement scheduler function unit 111 calls the cloud controller API by designating the decided arrangement destination. Subsequently, the cloud controller 108 requests the physical device 103c to create a virtual device (step S38). Here, the virtual device placement scheduler function unit 111 calls the API of the cloud controller 108 by designating the selected placement destination. As a result, the cloud controller 108 requests the designated physical device 103 to create a virtual device.

  Then, the physical device 103b creates a virtual device (step S39), and notifies the cloud controller 108 that the creation of the virtual device has been completed (step S40). Subsequently, the cloud controller 108 notifies the virtual device placement scheduler function unit 111 that the creation of the virtual device has been completed (step S41). Similarly, the physical device 103c creates a virtual device (step S42), and notifies the cloud controller 108 that the creation of the virtual device is completed (step S43). Subsequently, the cloud controller 108 notifies the virtual device placement scheduler function unit 111 that the creation of the virtual device has been completed (step S44). In the example illustrated in FIG. 6, the example in which the virtual device placement scheduler function unit 111 rearranges the physical device 103b and the physical device 103c has been described. However, the virtual device placement scheduler function unit 111 performs the rearrangement on either the physical device 103b or the physical device 103c. May be.

  FIG. 7 is a diagram for explaining a processing operation for rearranging virtual devices in the virtual device management system. FIG. 7 illustrates an operation in which the virtual device placement scheduler function unit 111 mediates the rearrangement of the virtual machine when the virtual machine operated by the physical device 103a is down.

  As illustrated in FIG. 7, the physical device 103a detects a virtual machine down on the physical device 103a by the virtual machine monitoring module 106 on the physical device 103a (step S51). Then, the physical device 103a notifies the virtual device placement scheduler function unit 111 that the virtual machine is down (step S52).

  The virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to delete the virtual machine instance (step S53). The cloud controller 108 requests the physical device 103a to delete the virtual machine instance (step S54). The physical device 103a deletes the virtual machine instance (step S55), and notifies the cloud controller 108 of the completion of virtual machine instance deletion (step S56). Subsequently, the cloud controller 108 notifies the virtual device placement scheduler function unit 111 of the completion of virtual machine instance deletion (step S57).

  The virtual device arrangement scheduler function unit 111 that has received the notification of completion of deletion of the virtual machine instance refers to the virtual device arrangement scheduler DB 110 for virtual machine relocation (step S58) and confirms physical resource information (step S59). As a result, the virtual device placement scheduler function unit 111 acquires the free capacity of the physical resources of the physical device 103a, the physical device 103b, and the physical device 103c, determines the physical device 103 on which the virtual machine is to be relocated, and prepares an API parameter. (Step S60). FIG. 7 shows a case where the virtual device arrangement scheduler function unit 111 recreates a virtual machine in the physical device 103a.

  The virtual device arrangement scheduler function unit 111 requests the cloud controller 108 to create a virtual machine (step S61). Subsequently, the cloud controller 108 requests the physical device 103a to create a virtual machine (step S62). Then, the physical device 103a creates a virtual machine (step S63), and notifies the cloud controller 108 of the completion of virtual machine creation (step S64). Subsequently, the cloud controller 108 notifies the virtual device placement scheduler function unit 111 of the completion of virtual machine creation (step S65).

  Next, the virtual device placement scheduler DB 110 and the virtual device placement scheduler function unit 111 realized by the virtual device management apparatus 109 will be described with reference to FIG. FIG. 8 is a diagram for explaining the virtual device placement scheduler DB 110 and the virtual device placement scheduler function unit 111 realized by the virtual device management apparatus 109.

  As shown in FIG. 8, the virtual device arrangement scheduler DB 110 stores a virtual device arrangement information table 110a and a physical resource information table 110b. The virtual device arrangement information table 110a stores virtual device arrangement information indicating on which physical device each virtual device is arranged.

  FIG. 9 is a diagram illustrating an example of the data structure of the virtual device arrangement information table 110a. As illustrated in FIG. 9, the virtual device arrangement information table 110 a stores virtual device arrangement information in which “virtual machine ID”, “physical device ID”, and “progress state” are associated with each other. Here, the “virtual machine ID” stored in the virtual device arrangement information table 110 a indicates an identifier for uniquely identifying a virtual machine created in the physical device 103. For example, data values such as “virtual machine # 11” and “virtual machine # 21” are stored in “virtual machine ID”.

  The “physical device ID” stored in the virtual device arrangement information table 110a indicates an identifier for uniquely identifying the physical device 103. For example, data values such as “physical device # 1” and “physical device # 2” are stored in “physical device ID”.

  The “progress status” stored in the virtual device arrangement information table 110a indicates information indicating the progress status of the virtual machine recovery process. This “progress state” is updated by a virtual machine deletion unit 111d and a re-creation unit 111e described later. Here, in the “progress status”, information indicating the progress of the recovery process of the virtual machine includes “in operation”, “deleting (virtual machine down)”, “deleting (physical device failure)”, “creating” (Virtual machine down) "and" Creating (physical device failure) "are stored. Here, “in operation” indicates that the virtual machine is in operation. That is, “in operation” indicates that the virtual machine recovery processing has not started.

  “Deleting (virtual machine down)” indicates that recovery processing of the virtual machine due to the failure of the virtual machine is being performed. “Deleting (physical device failure)” indicates that virtual machine recovery processing due to a physical device failure is being performed. Here, as a recovery process when the progress state is being deleted, a process of deleting the association between the virtual machine and the storage area used by the virtual machine and a process of deleting the virtual machine instance are performed. Note that the recovery process in the case of deletion is also referred to as “first recovery process”.

  “Creating (virtual machine down)” indicates that a virtual machine recovery process due to a failure of the virtual machine is in progress. “Creating (physical device failure)” indicates that a virtual machine recovery process due to a physical device failure is being performed. Here, as recovery processing when the progress state is being created, a physical device that operates the virtual machine is selected, and processing for causing the selected physical device to recreate the virtual machine is performed. Note that the recovery process when it is being created is also referred to as a “second recovery process”. The second recovery process is performed after the end of the first recovery process.

  For example, in the virtual device arrangement information table 110a shown in FIG. 9, the virtual devices “virtual machine # 11” and “virtual machine # 12” are arranged in the physical device 103 whose identifier is “physical device # 1”. This indicates that the virtual machine # 11 and the virtual machine # 12 are being restored by the virtual machine being down. Here, the virtual machine # 11 is being deleted, and the virtual machine # 12 is being created. In the virtual device arrangement information table 110a shown in FIG. 9, the virtual devices “virtual machine # 21” and “virtual machine # 22” are arranged in the physical device 103 whose identifier is “physical device # 2”. This indicates that the virtual machine # 21 and the virtual machine # 22 are being restored due to a failure of the physical device # 2. Here, the virtual machine # 21 and the virtual machine # 22 are being deleted. Further, the virtual device arrangement information table 110a illustrated in FIG. 9 includes virtual devices “virtual machine # 23”, “virtual machine # 31”, and “virtual machine #” for the physical device 103 whose identifier is “physical device # 3”. 32 ”is arranged, indicating that the virtual machine # 31 and the virtual machine # 32 are operating, and that the virtual machine # 23 is being restored due to a failure of a physical device other than the physical device # 3. In the example illustrated in FIG. 9, the virtual machine # 21, # 22, and # 23 are originally operating in the physical device # 2, and the physical device # 2 has failed. For this reason, the virtual machines # 21, # 22, and # 23 are cleaned up, but since the virtual machines # 21 and # 22 are still being cleaned up, they are associated with the physical device # 2 and are displayed as being deleted. The Since the virtual machine # 23 has already been cleaned up and virtual machine creation has already started on the physical device # 3, it is displayed as being created in association with the physical device # 3.

  Returning to FIG. The physical resource information table 110b stores physical resource information indicating the operating state of each physical device and the free capacity of the physical resource that the physical device has. FIG. 10 is a diagram illustrating an example of a data structure of the physical resource information table 110b. As illustrated in FIG. 10, the physical resource information table 110b stores physical resource information in which “physical device ID”, “operating state”, “free”, “in use”, and “for failure” are associated with each other.

  Here, the “physical device ID” stored in the physical resource information table 110 b indicates an identifier for uniquely identifying the physical device 103. For example, data values such as “physical device # 1” and “physical device # 2” are stored in “physical device ID”.

  The “operating state” stored in the physical resource information table 110b indicates whether the physical device is operating. This “operation state” is updated by a failure determination unit 111c described later. For example, when the physical device is in operation, “in operation” is stored in “operation state”. Although not shown in FIG. 8, when the physical device is provided as a standby system and is not in operation, “standby” is stored in the “operation state”. In addition, when recovery processing of a physical device in which a failure has occurred has not started, “failed (maintenance)” is stored in the “operation state”. Further, when the recovery process of the physical device in which the failure has occurred has been started, “failed (during recovery process)” is stored in the “operation state”.

  “Free” indicates the free capacity of the physical resource capacity of the physical device. For example, values such as “3”, “5”, and “4” are stored in “empty”. Further, “in use” indicates a capacity in use among the capacity of physical resources of the physical device. For example, “in use” stores values such as “1”, “3”, and “2”. Further, “for failure” indicates the capacity reserved for recovery out of the physical resource capacity of the physical device. For example, a value such as “2” is stored in “for failure”.

  As an example, in the physical resource information table 110b shown in FIG. 10, the physical device # 1 is in operation, the free capacity of the physical resource is “3”, and the used capacity is “1”. This indicates that the capacity reserved for recovery is “2”. Further, in the physical resource information table 110b shown in FIG. 10, the physical device # 2 is in operation, the free capacity of the physical resource is “5”, the used capacity is “3”, and is used for recovery. It shows that the secured capacity is “2”. Similarly, in the physical resource information table 110b shown in FIG. 10, the physical device # 3 is in operation, the free capacity of the physical resource is “4”, the capacity in use is “2”, and Indicates that the secured capacity is “2”.

  Returning to FIG. The virtual device placement scheduler function unit 111 includes a creation request reception unit 111a, a placement destination selection unit 111b, a failure determination unit 111c, a virtual machine deletion unit 111d, and a recreation unit 111e.

  The creation request reception unit 111 a receives a virtual device creation request from the user terminal 101. The creation request accepting unit 111a delivers the accepted virtual device creation request to the placement destination selecting unit 111b.

  The placement destination selection unit 111b selects a physical device on which a virtual device is placed when a virtual device is newly created. Here, the placement destination selection unit 111b selects the physical device 103 so that virtual devices are distributed as much as possible. In other words, the placement destination selection unit 111b arranges the virtual devices so that the number of “operating” “free” spaces is leveled.

  FIG. 11 is a diagram for explaining the processing operation by the placement destination selection unit 111b. FIG. 11 illustrates a case where a virtual device is newly created in a virtual device management system having six physical devices # 1 to # 6. Here, the operating state of the physical device # 1 to the physical device # 5 is “operating”, and the operating state of the physical device # 6 is “standby”. The space status of the physical device # 1 is “free” 2, “in use” 1, and “failure buffer” 2. The space status of the physical device # 2 is “free” 0, “in use” 3 and “failure buffer” 2, and the space status of the physical device # 3 is “free” 4, “in use” 0, and “failure buffer” 2. The space status of the physical device # 4 is “free” 0, “in use” 5, and “failure buffer” 2. The space status of the physical device # 5 is “free” 2, “in use”. "0", "Fault buffer" 2, and the space status of the physical device # 6 is "Free" 3, "In use" 0, "Fault buffer" 2.

  For example, when selecting an arrangement destination, the arrangement destination selection unit 111b checks the amount of free space of a physical device whose operation state is “in operation” and identifies an active physical device with the largest available space. More specifically, the placement destination selection unit 111b selects one virtual device (for example, virtual device # 1) from the virtual devices to be created. Then, in the case of the space state shown in FIG. 11, the physical device # 3 whose “empty” is 4 is identified as an active physical device with the most free space. The placement destination selection unit 111b does not use the “failure buffer” during normal operation. Then, the placement destination selection unit 111b selects the specified physical device # 3 as the placement destination of the selected virtual device # 1.

  Subsequently, the placement destination selection unit 111b repeats the process of selecting the placement destination until the placement destinations of all virtual devices requested to be created are selected. For example, the placement destination selection unit 111b selects physical devices so that virtual devices are placed in the numerical order shown in FIG. In this way, the placement destination selection unit 111b equalizes the number of “free” spaces by selecting a part of the space of the operating physical device that has the most free space.

  Further, the arrangement destination selection unit 111b arranges the virtual device in the physical device whose operation state is “standby” when all the space of the physical device whose operation state is “in operation” is filled. For this reason, when the placement destination selecting unit 111b places virtual devices up to the eighth space shown in FIG. 11, the placement destination selecting unit 111b places the virtual devices in the spare physical device. That is, in the example illustrated in FIG. 11, the placement destination selection unit 111 b places virtual devices on physical devices whose operation state is “standby” when nine or more virtual devices are created.

  Then, the placement destination selection unit 111b requests the selected physical device 103 to the cloud controller 108 to create a virtual device.

  Returning to FIG. The failure determination unit 111c detects a failure of a virtual machine or a failure of a physical device that operates the virtual machine. For example, the failure determination unit 111c detects the down virtual machine 104 or the failed physical device 103 in cooperation with the virtual machine monitoring module 106 and the high availability software 107 included in each physical device 103. Then, when the failure determination unit 111c detects a failure of the virtual machine or a failure of a physical device that operates the virtual machine, the failure determination unit 111c determines whether there is a recovery process that has been performed on the virtual machine in advance. To do.

  Here, first, processing for detecting a failure of a physical device that operates a virtual machine by the failure determination unit 111c will be described. FIG. 12 is a diagram for explaining a processing operation for detecting a failure of a physical device by the failure determination unit 111c.

  FIG. 12 illustrates the physical devices 103a to 103c, and a case where a failure occurs in the physical device 103a will be described. In FIG. 12, among the functions of the physical device 103, the physical device 103 a shows a component that functions when a failure of the own device occurs, and the physical device 103 b and the physical device 103 c detect a failure of another device. The component part which functions when it does is shown.

  As illustrated in FIG. 12, high availability software 107 a to 107 c is used for detecting a failure of the physical device 103. All physical devices 103 hold the status of all physical devices 103 in the cluster in a CIB (Cluster Information Base). The high availability software 107 confirms the state of the own physical device using an RA (Resource Agent). Note that RA corresponds to, for example, a virtual volume control unit or a virtual machine control unit.

  Further, the high availability software 107 can know the state of any other physical device in any physical device in the cluster by Heartbeat. For this reason, the high availability software 107 notifies the state in the cluster using the heartbeat packet. With this mechanism, each physical device knows the status of other physical devices. When the Heartbeat packet from a certain physical device is continuously lost, the high availability software 107 considers that the other physical device is down.

  The physical device 103 includes a notification RA and a notification process in order to notify the virtual device arrangement scheduler function unit 111 of a failure that has occurred in the physical device. For example, when the Maker detects a failure of the own physical device, the notification RA is used to notify the virtual device arrangement scheduler function unit 111 of the failure of the own physical device. On the other hand, the notification process is set as a resident process and periodically checks the status of the CIB. When the notification process detects a failure of another physical device, it confirms that the failure node is surely dropped by STONIT, which will be described later, and then indicates that a failure has occurred in the other physical device. Notify The notification by the notification RA and the notification by the notification process are repeated a certain number of times until the ACK is returned from the virtual device arrangement scheduler function unit 111.

  Subsequently, in the virtual device arrangement scheduler function unit 111, the failure determination unit 111c responds with an ACK when receiving the notification, and causes the virtual machine deletion unit 111d and the re-creation unit 111e to execute a virtual device recovery process. The failure determination unit 111c ignores the second and subsequent notifications and returns an ACK for the same content notification. However, the self-node failure notification and the other-node failure notification are handled separately. As a result, it is possible to take a redundancy measure by receiving notifications from a plurality of physical devices and not to repeat the recovery process.

  Note that the high availability software 107 may fail to stop its own physical device. In the case of a virtual machine, a plurality of virtual machines exist at the same time due to recovery, and there is a possibility of data destruction due to simultaneous access to the data area. Therefore, when the highly available software 107 is “Pacemaker”, the failed physical device is surely dropped using the STONITH module. STONIT guarantees that the failed physical device will not continue to operate by stopping the failed physical device via IPMI (Intelligent Platform Management Interface). The majority of physical devices that make up the majority with Quorum activate STONITH to prevent false triggers. Since Quorum is determined by a majority, if a physical device fails when the number of physical devices in the cluster is small, a normal physical device cannot secure the majority. For this reason, it is necessary to perform a reduction work for separating the failed physical device from the cluster. FIG. 12 shows the case where the high availability software 107 is a maker. However, other high availability software may detect the occurrence of a failure or notify the occurrence of the failure using the same mechanism. Is possible. When the failure determination unit 111c is notified of the failure of the physical device in which the failure has occurred, the failure determination unit 111c sets the “operating state” corresponding to the failed physical device in the physical resource information table 110b to “failing”. Update to (in maintenance). On the other hand, for example, when the failure that has occurred in another physical device is notified from the physical device 103b, the failure determination unit 111c sets the “operation state” corresponding to the failed physical device in the physical resource information table 110b to “failure”. Update to “middle (recovery process in progress)”, and cause the virtual machine deletion unit 111d and the re-creation unit 111e to execute the recovery process of the virtual device. In such a case, the failure determination unit 111c notifies the virtual machine deletion unit 111d and the recreation unit 111e that the recovery process is caused by a failure of a physical device that operates the virtual machine. The failure determination unit 111c ignores a failure that has occurred in another physical device following the notification from the physical device 103b because the recovery processing has already been started and is ignored.

  Next, processing for detecting a failure in the virtual machine by the failure determination unit 111c will be described. There are cases where a virtual device goes down due to a process failure or the like in addition to a case where all of the virtual devices operating on the physical device fail due to a physical device failure. In particular, a virtual machine failure has a great influence on the user of the virtual machine because it immediately leads to service interruption. Therefore, in the virtual device management system, not only physical device monitoring by high availability software such as Pacemaker, but also virtual machines are monitored by a virtual machine control library such as Libvirt. Then, the virtual device arrangement scheduler function unit 111 performs virtual machine recovery processing when a virtual machine fails.

  For example, the OpenStack community is a hypervisor in which many Qemu-KVMs are used, and many Libvirts are used as control libraries for virtual machines. Therefore, the virtual machine monitoring module 106 detects that the virtual machine has gone down by acquiring an event from a virtual machine control library such as Libvirt. When the virtual machine monitoring module 106 detects that the virtual machine is down, the virtual machine monitoring module 106 notifies the virtual device arrangement scheduler function unit 111 of the failure. Libvir can acquire not only virtual machine failures but also events such as shutdown of the virtual machine guest OS and shutdown of the host OS. Therefore, the virtual machine monitoring module 106 notifies the virtual device arrangement scheduler function unit 111 of events such as shutdown of the virtual machine guest OS and shutdown of the host OS. In addition, the failure determination unit 111c causes the virtual machine deletion unit 111d and the re-creation unit 111e to execute a virtual device recovery process. In such a case, the failure determination unit 111c notifies the virtual machine deletion unit 111d and the re-creation unit 111e that the recovery process is caused by the virtual machine being down.

  Next, a process for determining whether or not there is a recovery process that has been performed on the virtual machine in advance by the failure determination unit 111c will be described. The failure determination unit 111c may receive a plurality of failure notifications. In other words, failure notifications may compete. For example, in a virtual machine, if the recovery timing is wrong and a virtual machine is started twice, there is a possibility of data destruction due to access from the two virtual machines to the same data area. For this reason, it is necessary to perform an appropriate recovery process according to the current progress of the recovery process. Therefore, the failure determination unit 111c manages the failure notification from the high availability software 107 such as Pacemaker and the failure notification from the virtual machine monitoring module 106 such as the Livevirt monitoring module in accordance with the progress of the recovery process when receiving the notification. The recovery process to be executed is determined.

  For example, when a failure of a virtual machine or a failure of a physical device that operates the virtual machine is detected, the failure determination unit 111c determines whether there is a recovery process being performed on the virtual machine. For example, the failure determination unit 111c refers to the virtual device arrangement information table 110a illustrated in FIG. 9 to determine whether the progress status of the virtual machine operating on the physical device notified of the failure is being deleted or created. Determine. Then, the failure determination unit 111c determines that there is a recovery process being performed on the virtual machine when the progress state of the virtual machine operating on the physical device notified of the failure is being deleted or created. On the other hand, the failure determination unit 111c determines that there is no recovery process being performed on the virtual machine when the progress state of the virtual machine operating on the physical device notified of the failure is not being deleted or created. Below, the case where a failure notification competes will be described in four situations from pattern 1 to pattern 4.

  As pattern 1, a case will be described in which a physical device failure is newly detected when recovery processing due to a failure of a physical device that operates a virtual machine is being performed. In other words, a case where the “progress status” of a virtual machine operating on a physical device in which a failure is newly detected is “deleting (physical device failure)” or “creating (physical device failure)” is shown.

  The failure determination unit 111c discards the received notification as a retransmission before the deletion of the virtual machine on the physical device is completed. More specifically, the failure determination unit 111c, in the virtual device arrangement information table 110a, when the “progress status” of the virtual machine operating on the physical device in which the failure has occurred is “deleting (physical device failure)” It is determined that the notification is a retransmission. In other words, the failure determination unit 111c, when a failure of the physical device is newly detected when the first recovery process due to the failure of the physical device that operates the virtual machine is being performed, the newly detected failure The virtual machine deletion unit 111d is not allowed to execute the first recovery process due to the failure, and the re-creation unit 111e is not allowed to execute the second recovery process due to the newly detected failure.

  If the virtual machine instance is already being created with another physical device, the failure determination unit 111c performs physical device recovery processing using the newly received notification as a new event. More specifically, the failure determination unit 111c, in the virtual device arrangement information table 110a, when the “progress status” of the virtual machine operating on the physical device in which the failure has occurred is “being created (physical device failure)” Perform physical device recovery processing as a new event. In other words, if a failure in the physical device is newly detected when the second recovery process due to the failure in the physical device that operates the virtual machine is being performed, the failure determination unit 111c Stop the recovery process. Then, the failure determination unit 111c causes the virtual machine deletion unit 111d to execute the first recovery process due to the newly detected failure, and recreates the second recovery process due to the newly detected failure. 111e is executed. If the notification is for the same physical device and the virtual machine has already been deleted, the process of newly deleting the virtual machine can be omitted.

  As a pattern 2, a case will be described in which a failure of a virtual machine is newly detected when recovery processing due to a failure of a physical device that operates the virtual machine is being performed. In other words, the “progress status” of a virtual machine in which a down is newly detected is “deleting (physical device failure)” or “creating (physical device failure)”.

  If the failure of the virtual machine is newly detected while the first recovery process is being performed, the failure determination unit 111c performs the first recovery process due to the newly detected failure in the virtual machine deletion unit 111d. And the re-creation unit 111e is not allowed to execute the second recovery process caused by the newly detected failure. For example, since virtual machine cleanup on a physical device is performed after stopping at STONITH, a virtual machine down notification is not basically received. However, when the notification is delayed, a virtual machine down notification may be received during deletion of the virtual machine. In this case, the “progress status” of the virtual machine in which down is newly detected is “deleting (physical device failure)”. In such a case, the failure determination unit 111c determines to continue the virtual machine deletion process being performed.

  In addition, when a failure of the virtual machine is newly detected while the second recovery process is being performed, the failure determination unit 111c causes the re-creation unit 111e to stop the second recovery process being performed. Then, the failure determination unit 111c causes the virtual machine deletion unit 111d to execute the first recovery process due to the newly detected failure, and recreates the second recovery process due to the newly detected failure. 111e is executed. For example, when the virtual machine instance is already being created with another physical device, the failure determination unit 111c handles the newly received notification as a new failure notification. In this case, the “progress status” of the virtual machine in which the down is newly detected is “Creating (physical device failure)”. Thereby, the virtual machine deleting unit 111d deletes the virtual machine instance in addition to deleting the association as the first recovery processing. Then, the re-creation unit 111e causes the selected physical device to re-create the virtual machine as a second recovery process after deleting the association and deleting the instance of the virtual machine.

  As a pattern 3, a case where a physical device failure is newly detected while a recovery process due to a virtual machine failure is being performed will be described. In other words, a case where the “progress status” of a virtual machine operating on a physical device in which a failure is newly detected is “deleting (virtual machine down)” or “creating (virtual machine down)” is shown. In such a case, the failure determination unit 111c cancels the ongoing recovery process when a failure of the physical device that operates the virtual machine is newly detected when there is an ongoing recovery process due to the failure of the virtual machine. Let Then, the failure determination unit 111c causes the virtual machine deletion unit 111d to execute the first recovery process due to the newly detected failure, and recreates the second recovery process due to the newly detected failure. 111e is executed.

  For example, if the “progress status” of a virtual machine running on a physical device in which a failure is newly detected is “deleting (virtual machine down)”, the deletion of the virtual machine instance may not end due to a physical device failure is there. Therefore, the failure determination unit 111c instructs the virtual machine deletion unit 111d to interrupt the virtual machine instance deletion being performed. Then, the failure determination unit 111c causes the virtual machine deletion unit 111d and the re-creation unit 111e to execute the newly notified physical device recovery process.

  For example, even when the “progress status” of a virtual machine operating on a physical device in which a failure is newly detected is “Creating (Virtual Machine Down)”, the deletion of the virtual machine instance does not end due to a physical device failure. there is a possibility. Therefore, the failure determination unit 111c instructs the virtual machine deletion unit 111d to interrupt the virtual machine instance deletion being performed. Then, the failure determination unit 111c causes the virtual machine deletion unit 111d and the re-creation unit 111e to execute the newly notified physical device recovery process.

  As pattern 4, a case will be described in which a failure of a virtual machine is newly detected when recovery processing due to a failure of the virtual machine is being performed. In other words, the “progress status” of a virtual machine in which a down is newly detected is “deleting (virtual machine down)” or “creating (virtual machine down)”.

  If the failure of the virtual machine is newly detected while the first recovery process is being performed, the failure determination unit 111c performs the first recovery process due to the newly detected failure in the virtual machine deletion unit 111d. And the re-creation unit 111e is not allowed to execute the second recovery process caused by the newly detected failure. For example, when the “progress status” of a virtual machine in which a down is newly detected is “deleting (virtual machine down)”, the failure determination unit 111c discards the newly received notification as being retransmitted, Continue to delete the virtual machine instance that is being implemented.

  In addition, when a failure of the virtual machine is newly detected while the second recovery process is being performed, the failure determination unit 111c causes the re-creation unit 111e to stop the second recovery process being performed. Then, the failure determination unit 111c causes the virtual machine deletion unit 111d to execute the first recovery process due to the newly detected failure, and recreates the second recovery process due to the newly detected failure. 111e is executed. Thereby, the virtual machine deleting unit 111d deletes the virtual machine instance in addition to deleting the association as the first recovery processing. For example, when the “progress status” of a virtual machine in which a down is newly detected is “Creating (Virtual Machine Down)”, the failure determination unit 111c interrupts the virtual machine instance creation process that is being performed. The re-creation unit 111e is instructed. Then, the re-creation unit 111e causes the selected physical device to re-create the virtual machine as a second recovery process after deleting the association and deleting the instance of the virtual machine.

  As described above, the failure determination unit 111c uniformly manages the recovery processing caused by the failure of the physical device and the recovery processing of the down virtual machine, which has been performed independently in the past. It is possible to perform reliable recovery processing that prevents heavy startup.

  Returning to FIG. In response to an instruction from the failure determination unit 111c, the virtual machine deletion unit 111d deletes the association between the virtual machine and the storage area used by the virtual machine as the first recovery process. Here, in the case of the first recovery process caused by the failure of the virtual machine, the virtual machine deleting unit 111d deletes the association between the virtual machine and the storage area used by the virtual machine, Delete the machine instance.

  For example, when there is no recovery process in progress, the virtual machine deletion unit 111d deletes the association between the virtual machine and the storage area used by the virtual machine as the first recovery process. Further, the virtual machine deletion unit 111d stops the first recovery process that is being performed in response to an instruction from the failure determination unit 111c. When the first recovery process being performed is stopped, the virtual machine deletion unit 111d newly executes the first recovery process in response to an instruction from the failure determination unit 111c.

  Further, when starting the first recovery process, the virtual machine deleting unit 111d sets “deleting (virtual machine down)” as the “progress state” corresponding to the virtual machine in which a failure has occurred in the virtual device arrangement information table 110a. Alternatively, it is updated to “Deleting (physical device failure)”.

  After the first recovery process ends, the virtual machine deletion unit 111d notifies the re-creation unit 111e that the first recovery process has ended.

  The re-creation unit 111e operates the virtual machine as the second recovery process when the virtual machine deletion unit 111d is notified that the first recovery process has ended after receiving the instruction from the failure determination unit 111c. Select the physical device to be created, and recreate the virtual machine on the selected physical device.

  For example, the re-creation unit 111e selects a physical device for re-creating a virtual device when notified of a failure that has occurred in the physical device. Further, when the re-creation unit 111e is notified of the down of the virtual machine, the re-creation unit 111e selects a physical device for re-creating the down virtual machine. Here, when notified of the failure that has occurred in the physical device, the re-creation unit 111e causes the selected physical device to re-create the virtual machine as a second recovery process after deleting the association. In addition, when the virtual machine is notified that the virtual machine is down, the re-creation unit 111e causes the selected physical device to re-create the virtual machine as a second recovery process after deleting the association and deleting the virtual machine instance. .

  Hereinafter, processing of the re-creation unit 111e when a failure that has occurred in a physical device is notified will be described. Here, the re-creation unit 111e selects physical devices so that they are allocated in order as much as possible to physical devices other than the failed physical device. For example, the re-creation unit 111e specifies a plurality of physical devices having a physical resource free capacity among physical devices other than the failed physical device. Then, the recreating unit 111e selects the physical resources of the identified plurality of physical devices as the relocation destination of the virtual device that is allocated to the physical device in which the failure has occurred.

  FIG. 13 is a diagram for explaining the processing operation by the re-creation unit 111e. FIG. 13 illustrates recovery when the physical device # 4 fails in the virtual device management system having six physical devices # 1 to # 6. Here, the operating state of the physical device # 1 to the physical device # 5 is “operating”, and the operating state of the physical device # 6 is “standby”. The space status of the physical device # 1 is “free” 2, “in use” 1, and “failure buffer” 2. The space status of the physical device # 2 is “free” 0, “in use” 3 and “failure buffer” 2, and the space status of the physical device # 3 is “free” 4, “in use” 0, and “failure buffer” 2. The space status of the physical device # 4 is “free” 0, “in use” 10, and “failure buffer” 2. The space status of the physical device # 5 is “free” 2, “in use”. "0", "Fault buffer" 2, and the space status of the physical device # 6 is "Free" 3, "In use" 0, "Fault buffer" 2.

  For example, the re-creation unit 111e specifies a plurality of physical devices having a physical resource free capacity among physical devices other than the failed physical device. Here, the re-creation unit 111e uses the space of the “failure buffer” in addition to the “free” space when a failure of the physical device occurs. As a result, the re-creation unit 111e allows more physical devices to share the recovery process, including physical devices whose “empty” is zero. The example illustrated in FIG. 13 illustrates a case where the re-creation unit 111e identifies physical devices # 1 to # 3 and physical devices # 5 that are operating.

  Then, the re-creation unit 111e selects the physical resources of the identified plurality of physical devices as the relocation destination of the virtual device disposed in the physical device # 4 where the failure has occurred. For example, the re-creation unit 111e orders the plurality of identified physical devices. Here, the re-creating unit 111e orders the specified plurality of physical devices in descending order of the physical resources of the specified plurality of physical devices. For example, the re-creation unit 111e sets the total space of the “free” space and the “failure buffer” space as the free capacity of the physical resource, and orders the physical devices in descending order of the total space. In the example of FIG. 13, the re-creation unit 111e has a physical device # 3 with a total space of 6, a physical device # 1 with a total space of 4, a physical device # 5 with a total space of 4, and a total space of 2 shows a case where the physical device # 2 is ordered in the order.

  Subsequently, the re-creation unit 111e repeats the process of selecting the physical resource of the physical device selected based on the order as the relocation destination of each of the virtual devices arranged in the failed physical device. As an example, the re-creation unit 111e selects physical devices so that virtual devices are rearranged in the numerical order shown in FIG. Here, at the time of rearrangement, it can be put in the order of increasing free capacity, or it can be put in different physical devices as much as possible until there is no free capacity. In the former case, the purpose is to make the free space after rearrangement uniform, and in the latter case, the purpose is to rearrange as many physical devices as possible and restore them quickly. FIG. 13 is an example assuming the latter logic. More specifically, the re-creation unit 111e transfers the physical resources of the physical devices selected in the order of physical device # 3, physical device # 1, physical device # 5, and physical device # 2 to the virtual device relocation destination. The process of selecting as is repeated. Here, the re-creating unit 111e arranges virtual devices in order on each physical device until there is no “free” space or “failure buffer” space. In addition, the re-creation unit 111e skips physical devices that have run out of space.

  Note that the re-creation unit 111e has a physical device whose operation state is “standby” until the “free” space and the “failure buffer” space of all physical devices whose operation state is “active” are filled. Do not select. In this way, the virtual device placement scheduler function unit 111 prepares in advance a “failure buffer” space that is not used when creating a virtual device, in addition to the “empty” space. High-speed recovery is possible by rearranging virtual devices. Further, if not all the virtual devices arranged in the physical device in which the failure has occurred can be rearranged, the re-creating unit 111e “free” space and “failure buffer” space of the identified physical device The relocation destination is selected for each virtual device within the range that can be relocated.

  In addition, the cluster configuration of Pacemaker is configured with a maximum of about 8 units to detect a failure. Further, since the virtual device placement scheduler function unit 111 may create a virtual device in another physical device across the cluster, the physical device requested to be recreated may be larger than the size of the cluster. Also, there may or may not be a spare machine used when everything is buried. Since the cluster configuration is N-Act, 0-Sby and there is no need to prepare a Stand-by machine, the utilization efficiency of physical equipment can be increased.

  Then, the re-creation unit 111e requests the selected physical device 103 to the cloud controller 108 to create a virtual device.

  Well, when the virtual machine goes down, the re-creation unit 111e may select a physical device that re-creates the virtual machine in the same manner as when a failure occurred in the physical device is notified, or it goes down. The down virtual machine may be recreated on the physical device on which the virtual machine was operating.

  In addition, the re-creation unit 111e stops the second recovery process that is being performed in response to an instruction from the failure determination unit 111c. When the second recovery process that is being performed is stopped, the re-creation unit 111e newly performs the second recovery process in response to an instruction from the failure determination unit 111c. Also in this case, the re-creation unit 111e performs the second recovery process when the virtual machine deletion unit 111d is notified that the first recovery process has been completed after receiving the instruction from the failure determination unit 111c. To do.

  Further, when starting the second recovery process, the re-creation unit 111e sets “progress state (virtual machine down)” to “progress state” corresponding to the failed virtual machine in the virtual device arrangement information table 110a or Update to "Creating (physical device failure)". In addition, when the second recovery process is completed, the re-creation unit 111e updates the “progress status” corresponding to the virtual machine in which the failure has occurred to “in operation” in the virtual device arrangement information table 110a.

  In the example illustrated in FIG. 13, a case has been described in which the re-creation unit 111e orders the plurality of identified physical devices in descending order of physical resource capacity of the identified plurality of physical devices. It is not limited to this. For example, the re-creation unit 111e may arbitrarily order the specified plurality of physical devices regardless of the free capacity of the physical resource.

  FIG. 14 is a flowchart illustrating a processing procedure performed by the virtual device arrangement scheduler function unit 111 when a physical device failure notification is received. As illustrated in FIG. 14, the failure determination unit 111c determines whether a physical device failure notification has been received (step S101). Here, when the failure determination unit 111c determines that a physical device failure notification has been received (step S101, Yes), the failure determination unit 111c determines whether there is a recovery process being performed on the virtual machine (step S102). . If the failure determination unit 111c determines that a physical device failure notification has not been received (No in step S101), the failure determination unit 111c determines whether a physical device failure notification has been received.

  If the failure determination unit 111c does not determine that there is a recovery process in progress for the virtual machine (No in step S102), the failure determination unit 111c proceeds to step S107. On the other hand, if the failure determination unit 111c determines that there is a recovery process that is being performed on the virtual machine (Yes in step S102), whether the recovery process that is being performed on the virtual machine is due to a failure of the physical device. It is determined whether or not (step S103). Here, if the failure determination unit 111c does not determine that the recovery process being performed on the virtual machine is caused by a failure of the physical device (No in step S103), the failure determination unit 111c proceeds to step S106.

  If the failure determination unit 111c determines that the recovery process being performed on the virtual machine is caused by a failure of the physical device (step S103, Yes), the failure determination unit 111c determines whether the physical device failure notification is a retransmission (step S104). . For example, the failure determination unit 111c determines that the physical device failure notification is retransmission when the progress state of the virtual machine running on the physical device notified of the failure is being deleted. On the other hand, the failure determination unit 111c determines that the physical device failure notification is not retransmission when the progress state of the virtual machine running on the physical device notified of the failure is not being deleted. In other words, the failure determination unit 111c determines that the physical device failure notification is not retransmission when the progress state of the virtual machine running on the physical device to which the failure is notified is in operation or being created.

  If the failure determination unit 111c determines that the physical device failure notification is retransmission (Yes in step S104), the failure determination unit 111c discards the notification (step S105) and ends the processing. On the other hand, when the failure determination unit 111c does not determine that the physical device failure notification is retransmission (No in step S104), the failure determination unit 111c stops the process being executed (step S106). Then, the failure determination unit 111c causes the virtual machine deletion unit 111d and the re-creation unit 111e to perform recovery processing (step S107).

  FIG. 15 is a flowchart illustrating a processing procedure performed by the virtual device arrangement scheduler function unit 111 when a virtual machine down notification is received. As illustrated in FIG. 15, the failure determination unit 111c determines whether a virtual machine down notification is received (step S201). If the failure determination unit 111c determines that a virtual machine down notification has been received (step S201, Yes), the failure determination unit 111c determines whether there is a recovery process in progress for the virtual machine (step S202). . If the failure determination unit 111c determines that a virtual machine down notification has not been received (No in step S201), the failure determination unit 111c determines whether a virtual machine down notification has been received.

  If the failure determination unit 111c does not determine that there is a recovery process in progress for the virtual machine (No in step S202), the failure determination unit 111c proceeds to step S206. On the other hand, if the failure determination unit 111c determines that there is a recovery process being performed on the virtual machine (Yes in step S202), is the recovery process being performed on the virtual machine the first recovery process? It is determined whether or not (step S203). Here, if the failure determination unit 111c does not determine that the recovery process being performed on the virtual machine is the first recovery process (No at Step S203), the failure determination unit 111c stops the process being performed (Step S204). ). Then, the failure determination unit 111c causes the virtual machine deletion unit 111d and the re-creation unit 111e to perform recovery processing (step S206).

  On the other hand, when the failure determination unit 111c determines that the recovery process being performed on the virtual machine is the first recovery process (Yes in step S203), the failure determination unit 111c discards the notification (step S205) and ends the process. To do.

  As described above, the virtual device management apparatus 109 according to the first embodiment performs unified means as a recovery method when a physical device fails and a virtual machine goes down, and even if a physical device failure and a virtual machine go down overlap. The virtual machine status is managed, and recovery is performed so that the virtual machine does not start twice.

  For example, the virtual device management apparatus 109 according to the first embodiment newly detects a physical device failure when the first recovery process due to the failure of the physical device that operates the virtual machine is being performed. In this case, it is determined that the notification is retransmission, and the first recovery process and the second recovery process due to the newly detected failure are not performed.

  In addition, the virtual device management apparatus 109 according to the first embodiment newly detects a physical device failure when the second recovery process due to the failure of the physical device that operates the virtual machine is being performed. In such a case, the second recovery process being performed is stopped. Then, the virtual device management apparatus 109 according to the first embodiment performs physical device recovery processing as a new event.

  Further, the virtual device management apparatus 109 according to the first embodiment, when a failure of a physical device that operates a virtual machine is newly detected when there is an ongoing recovery process due to a failure of the virtual machine, Stop the ongoing recovery process. Then, the virtual device management apparatus 109 according to the first embodiment performs a first recovery process and a second recovery process caused by a newly detected failure.

  In addition, when a virtual machine failure is newly detected while the first recovery process is being performed, the virtual device management apparatus 109 according to the first embodiment is configured to perform the first operation caused by the newly detected failure. The first recovery process and the second recovery process are not executed.

  In addition, when a virtual machine failure is newly detected when the second recovery process is being performed, the virtual device management apparatus 109 according to the first embodiment performs the second recovery process being performed. The first recovery process and the second recovery process caused by the newly detected failure are performed.

  As described above, the virtual device management apparatus 109 according to the first embodiment manages the status of the virtual machine so that the virtual machine does not start twice even if the physical device failure and the virtual machine down overlap. Perform recovery. As a result, the virtual device management apparatus 109 according to the first embodiment can safely recover the virtual machine.

  In addition, the virtual device management apparatus 109 according to the first embodiment identifies a plurality of physical devices having a free physical resource capacity among physical devices other than the failed physical device. Then, the virtual device management apparatus 109 according to the first embodiment selects the physical resources of the identified plurality of physical devices as relocation destinations of the virtual devices arranged in the physical device in which the failure has occurred. That is, the virtual device management apparatus 109 according to the first embodiment uses a plurality of physical devices as recovery destinations of virtual devices. Thereby, the virtual device management apparatus 109 according to the first embodiment can reduce the time until the virtual device is restored.

  More specifically, in the conventional method, when a failure occurs in a physical device by combining a cluster with N-Act and M-Sby, the system is failed over to a Standby machine by the function of highly available software such as Pacemaker, and OpenStack or the like. The virtual device was reconstructed based on the cloud controller database. Here, in the conventional method, the failover using the HA cluster software has a problem that it takes time to recover all the virtual devices because a new virtual device is reconstructed in one Standby machine.

  On the other hand, in the virtual device management apparatus 109 according to the first embodiment, a cluster is formed with N-Act and 0-Sby, and when a failure occurs in a physical device, the failure is detected by the function of the highly available software. The virtual device management apparatus 109 is notified of a physical device failure without failing over. The virtual device management apparatus 109 determines a plurality of physical devices to be rearranged for each virtual device, designates the physical devices to be placed, and requests the cloud controller 108 to recreate. Then, the cloud controller 108 creates a virtual device in the designated physical device.

  As described above, the virtual device management apparatus 109 according to the first embodiment recovers at high speed by re-creating a virtual device operating on the failed physical device in a plurality of physical devices. In other words, since the virtual device management apparatus 109 uses a plurality of physical devices as recovery destinations, the virtual device recovery time when a physical device fails is reduced. For example, when there are three migration destination physical devices, the recovery processing time can be shortened to 1/3.

  Also, the virtual device management apparatus 109 according to the first embodiment performs failure detection by combining clusters with N-Act and 0-Sby with high availability software such as Pacemaker. Here, since the virtual device management apparatus 109 according to the first embodiment can perform failure recovery beyond the cluster frame, the number of migration destination physical devices can also be larger than the cluster size. Thereby, the recovery time can be further shortened.

  Furthermore, in the virtual device management system, the failure detection clusters are N-Act and 0-Sby, so there is no need to prepare a physical device for standby, and an increase in the number of physical devices can be suppressed.

  In the above-described embodiment, the placement destination selection unit 111b has been described as selecting the physical device 103 so that the virtual devices are distributed as much as possible during the normal operation of creating a new virtual device. The embodiment is not limited to this. For example, the placement destination selection unit 111b may select the physical device 103 so that the virtual device is placed without being distributed during a normal operation for creating a new virtual device.

(Second Embodiment)
Although the embodiments of the present invention have been described so far, the present invention may be implemented in other embodiments besides the above-described embodiments. Therefore, other embodiments will be described below.

(System configuration)
Also, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part of the distribution / integration may be functionally or physically distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

(program)
It is also possible to generate a virtual device management program in which the processing executed by the virtual device management apparatus 109 according to the above embodiment is described in a language that can be executed by a computer. In this case, when the computer executes the virtual device management program, the same effect as in the above embodiment can be obtained. Further, the virtual device management program is recorded on a computer-readable recording medium, and the virtual device management program recorded on the recording medium is read by the computer and executed, thereby realizing the same processing as in the above embodiment. May be. An example of a computer that executes a virtual device management program that realizes the same function as the virtual device management apparatus 109 shown in FIG. 1 and the like will be described below.

  FIG. 16 is a diagram illustrating a computer 1000 that executes a virtual device management program. As illustrated in FIG. 16, the computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. For example, a mouse 1051 and a keyboard 1052 are connected to the serial port interface 1050. For example, a display 1061 is connected to the video adapter 1060.

  Here, as shown in FIG. 16, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. The virtual device management program described in the above embodiment is stored in, for example, the hard disk drive 1031 or the memory 1010.

  Further, the virtual device management program is stored in, for example, the hard disk drive 1031 as a program module in which a command executed by the computer 1000 is described. Specifically, the determination procedure for executing the same information processing as the failure determination unit 111c described in the above embodiment, the deletion procedure for executing the same information processing as the virtual machine deletion unit 111d, and the same as the re-creation unit 111e A program module 1093 in which a re-creation procedure for executing the information processing is described is stored in the hard disk drive 1031.

  Data used for information processing by the virtual device management program is stored as program data 1094 in, for example, the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the hard disk drive 1031 to the RAM 1012 as necessary, and executes the above-described procedures.

  Note that the program module 1093 and the program data 1094 related to the virtual device management program are not limited to being stored in the hard disk drive 1031, but are stored in a removable storage medium, for example, by the CPU 1020 via the disk drive 1041 or the like. It may be read out. Alternatively, the program module 1093 and the program data 1094 related to the virtual device management program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and the network interface 1070 is stored. Via the CPU 1020.

(Other)
Note that the virtual device management program described in this embodiment can be distributed via a network such as the Internet. The specific program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, or a DVD, and being read from the recording medium by the computer.

109 Virtual device management apparatus 110 Virtual device placement scheduler DB
110a Virtual device arrangement information table 110b Physical resource information table 111 Virtual device arrangement scheduler function unit 111a Creation request reception unit 111b Placement destination selection unit 111c Failure determination unit 111d Virtual machine deletion unit 111e Recreation unit 1000 Computer 1010 Memory 1011 ROM
1012 RAM
1020 CPU
1030 Hard disk drive interface 1031 Hard disk drive 1040 Disk drive interface 1041 Disk drive 1050 Serial port interface 1051 Mouse 1052 Keyboard 1060 Video adapter 1061 Display 1070 Network interface 1080 Bus 1091 OS
1092 Application program 1093 Program module 1094 Program data

Claims (8)

A determination unit that determines whether or not there is a recovery process in progress for the virtual machine when a failure of the virtual machine or a failure of a physical device that operates the virtual machine is detected;
When there is no restoration process in progress, as the first restoration process, the association between the virtual machine and the storage area used by the virtual machine is deleted, or the association and the virtual machine A deletion unit for deleting a virtual machine instance;
After selecting the physical device that operates the virtual machine and deleting the association, or after deleting the association and the virtual machine instance, the virtual machine is added to the selected physical device as a second recovery process. A virtual device management apparatus comprising: a re-creation unit for re-creation.   The determination unit, when a failure of a physical device that operates the virtual machine is newly detected when there is an ongoing recovery process due to the failure of the virtual machine, cancels the ongoing recovery process. , Causing the deletion unit to execute the first recovery process due to a newly detected failure, and causing the re-creation unit to execute the second recovery process due to a newly detected failure. The virtual device management apparatus according to claim 1.   The determination unit, when the failure of the physical device is newly detected when the second recovery process due to the failure of the physical device that operates the virtual machine is being performed, The recovery process is stopped, the first recovery process caused by the newly detected failure is executed by the deletion unit, and the second recovery process caused by the newly detected failure is performed by the re-creation unit. The virtual device management apparatus according to claim 1, wherein the virtual device management apparatus is executed.   When the first recovery process due to a failure of a physical device that operates the virtual machine is being performed, the determination unit detects a newly detected failure when the failure of the physical device is newly detected. 2. The first recovery process caused by a failure is not executed by the deletion unit, and the second recovery process caused by a newly detected failure is not executed by the re-creation unit. The virtual device management apparatus described. If a failure of the virtual machine is newly detected when the second recovery process is being performed, the determination unit cancels the second recovery process being performed and is newly detected Causing the deletion unit to execute the first recovery process due to a failure, causing the re-creation unit to execute the second recovery process due to a newly detected failure,
The deletion unit deletes the virtual machine instance in addition to deleting the association as the first recovery process,
The re-creation unit causes the selected physical device to re-create the virtual machine as the second recovery process after deleting the association and the virtual machine instance. Virtual device management device.   If the failure of the virtual machine is newly detected when the first recovery process is being performed, the determination unit performs the first recovery process caused by the newly detected failure as the deletion unit. The virtual device management apparatus according to claim 1, wherein the second recovery process caused by a newly detected failure is not executed by the re-creation unit. A virtual device management method executed by a virtual device management apparatus,
The virtual device management apparatus is
A determination step of determining whether there is a recovery process in progress for the virtual machine when a failure of the virtual machine or a failure of a physical device that operates the virtual machine is detected;
When there is no restoration process in progress, as the first restoration process, the association between the virtual machine and the storage area used by the virtual machine is deleted, or the association and the virtual machine A deletion step of deleting the virtual machine instance;
After selecting the physical device that operates the virtual machine and deleting the association, or after deleting the association and the virtual machine instance, the virtual machine is added to the selected physical device as a second recovery process. A virtual device management method comprising: a re-creation step for re-creation. A determination procedure for determining whether there is a recovery process in progress for the virtual machine when a failure of the virtual machine or a failure of a physical device that operates the virtual machine is detected;
When there is no restoration process in progress, as the first restoration process, the association between the virtual machine and the storage area used by the virtual machine is deleted, or the association and the virtual machine Deletion procedure to delete the virtual machine instance,
After selecting the physical device that operates the virtual machine and deleting the association, or after deleting the association and the virtual machine instance, the virtual machine is added to the selected physical device as a second recovery process. A virtual device management program that causes a computer to execute a re-creation procedure for re-creation.

Images