JP6386485B2 - Management system, management method and management program - Google Patents

Description

  The present invention relates to a management system, a management method, and a management program.

  A conventional layer 2 (L2) network is generally constructed in the same location (site). On the other hand, in recent years, an L2 tunnel via a WAN (Wide Area Network) or a provider network such as VXLAN (Virtual eXtensible Local Area Network) or EVPN (Ethernet (Ethernet) Virtual Private Network) is set. With the development of technology, networks constructed at different sites can be connected by L2 tunnels and operated as the same subnet. In addition, with the development of virtual machine technology, it has become possible to freely change the server location.

  With these technologies, if it becomes difficult to continue server operation at the site where the server is operating due to a disaster, etc., the server can be migrated (migrated) to a site built at a remote location, It became possible to continue providing. For example, there may be a case where a power failure or a link disconnection occurs around the site where the server is currently operated due to the occurrence of a disaster such as an earthquake. At this time, the immediate network cannot be used due to a disaster or the like, and there is a grace period of several seconds to several minutes before the network becomes unavailable. This time is the timing for moving the server to the backup site.

  Next, a procedure for migrating a server operating in the form of a virtual machine (VM) to a remote site for backup will be described.

  First, a VM OS (Operating System) image is copied to the memory of the migration destination physical server. At the start of migration, synchronization of memory information is started between the migration source VM and the migration destination VM. Here, during the synchronization of the memory information, the memory information changes in the migration source VM by packet processing or application processing. For this reason, synchronization is performed by sequentially transmitting the memory information in which the change has occurred to the migration destination VM. When the synchronization has progressed to some extent, the operation of the migration source VM is stopped, the memory synchronization is completed, and the operation of the migration destination VM is restarted (for example, see Non-Patent Document 1).

  Then, after the VM migration is completed, it is necessary to execute processing for changing the destination of the packet that has reached the migration source VM to the migration destination VM so that the packet addressed to the VM arrives at the migration destination site. .

  Here, in a network system using a VXLAN protocol or MPLS (Multi Protocol Label Switching) -EVPN protocol that connects sites connected via a provider network with an L2 tunnel, a gateway is arranged between the site and the provider network. By terminating the tunnel. When sending a packet through the provider network, the source site gateway attaches a header that can pass through the provider network (a tunneling protocol header) to the packet, and then sends the packet to the destination site gateway. Forward. The destination gateway that received the packet removes the header from the packet and transfers the packet to the destination.

  For this reason, in the case of a network system connected via an L2 tunnel, it is necessary to change the gateway of the packet transfer destination from the gateway of the migration source site to the gateway of the migration destination site as the destination change processing to the migration destination VM. .

  Specifically, the migration status of a conventional VM in a network system that performs network connection by applying the VXLAN protocol will be described. FIG. 28 is a diagram for explaining the migration status of a conventional VM in a network system to which the VXLAN protocol is applied.

  When the VXLAN protocol is applied, as shown in FIG. 28, a VTEP (VXLAN Tunnel End Point) is arranged at the end of the tunnel at each site. Each VTEP has a transfer table Tp that associates the MAC (Media Access Control) address of the VM with the IP address of the VTEP of the site where the VM is located, such as VTEP (2) of the site G, for example. Then, each VTEP refers to the transfer table Tp, attaches a header with the destination address as the IP address of the VTEP corresponding to the destination VM, and transfers the encapsulated packet.

  In this case, the VM that has completed the migration transmits a broadcast packet to each VTEP, and changes the IP address of the VTEP corresponding to the VM (see the cell Mp in FIG. 28). As a result, each VTEP changes the destination of the header to be attached to the IP address of the migration destination gateway, and transfers the packet to the migration destination site.

  For example, the VM (1) that has completed the migration from the site E to the site F is sent to the VTEP (2) of the site G via the VTEP (3) of the site F, and the broadband cast with its own MAC address as the source address Send the packet. In a network system connected by an L2 tunnel, a transfer destination of a packet addressed to a certain MAC address is determined based on arrival port information of a packet having the corresponding MAC address as a transmission source address. For this reason, when the VTEP (2) receives a broadcast packet whose source is the MAC address of the VM (1) via the port corresponding to the VTEP (3), the VTEP (2) sets the MAC address of the VM (1). The corresponding transfer destination is changed to the IP address of VTEP (3) (see table Tp).

  In this way, the VM that has completed the migration changes the transfer destination of the packet addressed to the VM to the migration destination by transmitting a broadcast packet from the migration destination site. In the case of a network system to which the EVPN protocol is applied, the packet is transferred to the destination site via the MPLS tunnel. In this case, when the migration of the VM is completed, the label of the tunnel used for packet transfer is changed from the MPLS tunnel connected to the migration source gateway to the MPLS tunnel connected to the migration destination gateway. Switching takes place.

C. Clark, et al., “Live Migration of Virtual Machines,” NSDI'05 Proceedings of the 2nd Symposium on Networked Systems Design & Implementation, pp273-286, 2005

  The conventional technique has a problem that the amount of discarded packets increases when the distance between the sites is long. For example, it is desirable that the distance between these two sites be long so that the impact of a disaster that can actually be experienced by the site where the server is operating does not reach the backup migration destination site. For this reason, the distance between sites may be several hundred kilometers to several thousand kilometers away, for example.

  As described above, in order to switch the packet transfer destination to the migration destination after the VM migration, the VM after the migration transmits a broadcast packet from the migration destination site. Then, when this packet passes, each gateway changes the forwarding table. As the VM migrates, the VM at the migration source site stops at the designated timing, so packets addressed to the VM that reach the migration source site after the VM stop are discarded without being received.

  However, when the distance between the migration destination site and another site is long, it takes a long time for the broadcast packet transmitted from the migration destination site to pass through the gateway of each site. In other words, the period during which the forwarding table of each gateway is not updated becomes longer.

  For this reason, if the period during which the forwarding table of each gateway is not updated increases because the distance between the migration destination site and other sites is long, the amount of packets discarded at the migration source site increases. There was a problem.

  The present invention has been made in view of the above, and in a network system in which sites are connected by a tunnel route, a management system and a management that can reduce the influence of packet discard that occurs at the time of virtual machine migration between sites An object is to provide a method and a management program.

  In order to solve the above-described problems and achieve the object, a management system according to the present invention performs migration of virtual machines between sites in a network system having a plurality of sites and tunnel paths connecting the plurality of sites. A management system having a management device for managing and a transfer device for transferring packets, wherein the management device is a progress of the migration status of the virtual machine when the virtual machine is migrated to a device operating at another site A migration progress management unit that monitors the transfer status of the packet to the virtual machine to be migrated according to the progress of the migration status of the virtual machine, and the packet sent to the virtual machine during the migration of the virtual machine. A transfer destination change control for sending a transfer destination change notification for changing to a buffer to be buffered to the transfer apparatus at least during the migration of the virtual machine. When the transfer device receives the transfer destination change notification, the transfer device corresponds to the virtual machine that is the destination of the packet and the transfer destination corresponding to the virtual machine according to the transfer destination change notification. And a transfer control unit that changes a transfer destination corresponding to the virtual machine to be migrated to the buffer in the attached transfer table.

  According to the present invention, in a network system in which sites are connected by a tunnel route, it is possible to reduce the influence of packet discard that occurs when a virtual machine migrates between sites.

FIG. 1 is a diagram schematically illustrating an example of a configuration of a management system according to the first embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the management apparatus illustrated in FIG. FIG. 3 is a block diagram showing an example of the configuration of the transfer apparatus shown in FIG. FIG. 4 is a sequence diagram illustrating the flow of packet transfer during VM migration by the management system shown in FIG. FIG. 5 is a diagram illustrating an outline of a configuration of a management system according to Example 1 of the first embodiment. FIG. 6 is a schematic diagram illustrating an example of a configuration of a management system according to Example 1 of the first embodiment. FIG. 7 is a block diagram showing a configuration of the tunnel termination apparatus shown in FIG. FIG. 8 is a diagram for explaining the data structure of the transfer table stored in the storage unit shown in FIG. FIG. 9 is a diagram for explaining an example of identifier information of the tunnel terminating device shown in FIG. FIG. 10 is a diagram for explaining another example of the identifier information of the tunnel terminating device shown in FIG. FIG. 11 is a schematic diagram illustrating the flow of packet transfer during VM migration by the management system shown in FIG. FIG. 12 is a schematic diagram for explaining the flow of packet transfer during VM migration by the management system shown in FIG. FIG. 13 is a schematic diagram illustrating the flow of packet transfer during VM migration by the management system shown in FIG. FIG. 14 is a schematic diagram illustrating the flow of packet transfer during VM migration by the management system shown in FIG. FIG. 15 is a diagram illustrating an outline of another configuration of the management system according to Example 1 of the first embodiment. FIG. 16 is a schematic diagram illustrating a packet transfer flow during VM migration by the management system according to the second example of the first embodiment. FIG. 17 is a schematic diagram illustrating a packet transfer flow during VM migration by the management system according to the second example of the first embodiment. FIG. 18 is a schematic diagram illustrating a packet transfer flow during VM migration by the management system according to the second example of the first embodiment. FIG. 19 is a diagram illustrating an outline of a configuration of a management system according to Example 3 of the first embodiment. FIG. 20 is a diagram illustrating an outline of the operation of the management system according to the second embodiment. FIG. 21 is a diagram for explaining a packet transfer flow during VM migration in the management system according to Example 1 of the second embodiment. FIG. 22 is a diagram illustrating a packet transfer flow during VM migration in the management system according to Example 1 of the second embodiment. FIG. 23 is a diagram illustrating the flow of packet transfer during VM migration in the management system according to Example 1 of the second embodiment. FIG. 24 is a sequence diagram illustrating the flow of packet transfer during VM migration by the management system according to Example 1 of the second embodiment. FIG. 25 is a diagram for explaining a packet transfer flow during VM migration in the management system according to Example 2 of the second embodiment. FIG. 26 is a diagram illustrating a packet transfer flow during VM migration in the management system according to Example 2 of the second embodiment. FIG. 27 is a diagram illustrating an example of a computer in which a management system is realized by executing a program. FIG. 28 is a diagram for explaining the migration status of a conventional VM in the network system.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment. Moreover, in description of drawing, the same code | symbol is attached | subjected and shown to the same part.

[Embodiment 1]
Regarding the management system according to the first embodiment, a schematic configuration of the entire management system, a flow of processing in the management system, and a specific example will be described. Note that the number of sites and the number of devices described below are merely examples, and are not limited thereto.

[Management system configuration]
First, the configuration of the management system according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram schematically illustrating an example of a configuration of a management system according to the first embodiment.

  As illustrated in FIG. 1, the management system according to the first embodiment includes a migration target VM 1, a management device 2, a transfer device 3, a buffer 4, a buffer control device 5, and a terminal 6. The management system shown in FIG. 1 manages the migration status of the VM 1 in a network system having a plurality of sites and L2 tunnels (tunnel paths) connecting the plurality of sites. Techniques such as VXLAN and EVPN can be applied as a method for constructing the L2 tunnel. In each site, a LAN (Local Area Network) is formed by Ethernet (registered trademark) as an example.

  The VM 1 is a virtual device that executes a Web server, a DB server, and the like, and communicates with the terminal 6 to provide various services. The VM 1 is migrated across sites based on an instruction from a controller (not shown) that controls the network system to a physical server that accommodates the VM 1. For example, the VM 1 in FIG. 1 is migrated from a physical server installed at the migration source site to a physical server installed at the migration destination site. In the first embodiment, the migration of the VM1 is assumed to be a live migration in which the entire VM1 is moved to another physical server while the VM1 is in operation. The physical server includes a communication interface, a processor, a memory, a virtual switch, and the like, and controls the VM1.

  The management apparatus 2 manages the migration of the VM 1 between sites. Specifically, the management device 2 monitors the progress of the migration status of the VM 1 when the VM 1 migrates to a device operating at another site, and grasps the progress status of the migration (for example, what percentage has progressed). . Further, the management apparatus 2 changes the transfer destination of the packet addressed to the migration target VM 1 to the buffer 4 that buffers the packet transmitted to the VM 1 during migration of the VM 1 in accordance with the progress of the migration. The change notification is transmitted to the transfer apparatus 3 at least during the migration of the VM 1.

  The transfer device 3 transfers the arrived packet to the transfer destination according to the transfer table in which the MAC address of the VM and the transfer destination IP address corresponding to the VM are associated with each other. When receiving the notification indicating the change of the packet transfer destination accompanying the migration of the VM1, the transfer device 3 changes the transfer destination of the VM1 according to the notification. The transfer device 3 is a tunnel termination device that terminates an L2 tunnel, such as a virtual switch, a virtual software switch, a physical switch, or a gateway.

  The buffer 4 buffers packets transmitted to the VM 1 during migration of the VM 1. Note that there are actually a plurality of buffers 4. A buffer for buffering packets addressed to the migration target VM 1 among the plurality of buffers 4 may be selected in advance, or the controller or the management device 2 that controls the network system buffers the packets addressed to the migration target VM 1. You may make it select the buffer 4 to perform.

  In accordance with the received notification, the buffer control device 5 controls the buffering of the packet addressed to the migration target VM 1 to the buffer 4, while transmitting the packet buffered in the buffer 4 to the VM 1 that has been migrated. . The buffer control device 5 is notified of the MAC address of the migration target VM 1 from the controller or the management device 2 that controls the network system. The buffer control device 5 buffers the packet addressed to the notified MAC address in the buffer 4.

  The terminal 6 is an information processing apparatus that accesses the VM 1 operating at the migration source or migration destination site and uses various services. The terminal 6 is, for example, a personal computer or a smartphone. The packet transmitted to the VM 1 by the terminal 6 arrives at the transfer device 3 corresponding to the terminal 6 and is transferred from the transfer device 3 to the VM 1 via the other transfer device 3.

[Configuration of management device]
Next, the configuration of the management device 2 will be described. FIG. 2 is a block diagram showing an example of the configuration of the management apparatus 2 shown in FIG. As illustrated in FIG. 2, the management device 2 includes a control unit 21, a communication unit 22, and a storage unit 23. In some cases, the management device 2 includes the buffer 4 and functions as the buffer control device 5.

  The control unit 21 has an internal memory for storing a program that defines various processing procedures and the necessary data, and executes various processes using these. For example, the control unit 31 is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 21 includes an identifier acquisition unit 211, a buffer selection unit 212, a migration progress management unit 213, a transfer destination change control unit 214, and a communication control unit 215.

  The identifier acquisition unit 211 acquires identifier information of each device in the network system. Specifically, the identifier acquisition unit 211 acquires in advance the MAC address of the migration target VM 1 and the identifier information related to the tunnel connected to the migration destination site. The identifier information regarding the tunnel is, for example, the IP address of VTEP in the case of VXLAN, and the LSP (Label Switched Path) of the MPLS tunnel toward the PE (Provider Edge) in the case of MPLS-EVVPN.

  For example, the identifier acquisition unit 211 acquires identifier information of each device by being input from an operator to an input interface (not shown) of the management device 2. Or the identifier acquisition part 211 acquires the identifier information of each apparatus by exchanging information with each apparatus by signaling. Alternatively, the identifier acquisition unit 211 connects to an external database that stores identifier information of each device, and acquires identifier information of each device in the network system. Note that information necessary for extracting information from the external database is also included in the identifier information.

  The buffer selection unit 212 selects the buffer 4 that buffers packets addressed to the migration target VM1. The buffer selection unit 212 notifies the buffer control device 5 that controls the selected buffer 4 of the MAC address of the migration target VM 1 to start buffering of the migration target VM 1 packet to the buffer control device 5. Notice. Note that the buffer 4 for buffering packets addressed to the migration target VM 1 may be set in advance, or may be selected by an apparatus other than the management apparatus 2.

  The migration progress management unit 213 manages the progress of the migration status of the migration target VM 1. In other words, the migration progress management unit 213 monitors the progress of the migration status of the VM1. For example, the migration progress management unit 213 provides information such as how much the migration degree (migration degree) has progressed, how fast the migration is being performed, and how long the migration is to be completed. Monitor.

  The transfer destination change control unit 214 sends a transfer destination change notification for changing the transfer destination of the packet destined for the migration target VM1 to the buffer 4, at least during the migration of the VM1, according to the progress of the migration status of the VM1. 3 to send.

  Specifically, the transfer destination change control unit 214 sends a first transfer destination change notification for changing the transfer destination of a packet destined for the migration target VM1 to the buffer control device 5 during migration of the VM1. 3 to send. In this case, the transfer destination change control unit 214 determines the timing for transmitting the first transfer destination change notification based on the progress status of the migration progress management unit 213.

  For example, the transfer destination change control unit 214 receives information indicating how much the migration of VM1 has reached from the migration progress management unit 213, and the migration degree of VM1 exceeds a predetermined threshold (for example, 70%) In addition, it is determined that it is time to transmit the first transfer destination change notification.

  Alternatively, the transfer destination change control unit 214 measures the traffic status of the network system, and determines the predetermined based on information obtained by combining the measured information (for example, communication delay time) and the migration status of the VM 1 by the migration progress management unit 213. Is used to calculate the timing for transmitting the first transfer destination change notification.

  For example, the transfer destination change control unit 214 measures the traffic situation between the migration source site and the migration destination site. When the traffic during this period is free, the transfer destination change control unit 214 lowers the predetermined threshold regarding the degree of migration of VM1, and the degree of migration exceeds the changed predetermined threshold (for example, 65%). In this case, it is determined that it is time to transmit the first transfer destination change notification. In other words, considering the traffic situation, when the traffic is free, the timing of the first transfer destination change notification is advanced in accordance with the advance of the time until the completion of migration.

  In addition, the transfer destination change control unit 214 notifies the buffer control device 5 that the packet addressed to the migration target VM 1 buffered in the buffer 4 is transferred to the migrated VM 1 after the migration of the VM 1 is completed. To do. Then, the transfer destination change control unit 214 transmits a second transfer destination change notification for changing the transfer destination of the packet destined for the migration target VM 1 to the migration destination site after the migration of the VM 1 is completed to each transfer device 3. To do.

  The communication control unit 215 controls communication processing in the communication unit 22 to be described later, and communicates various types of information with other devices. The communication control unit 215 also has a function of establishing an L2 tunnel with an opposing device.

  The communication unit 22 is a communication interface that transmits and receives various types of information to and from a device connected via a network. For example, the communication unit 22 transmits a packet transfer instruction to the buffer control device 5 or receives a packet from the buffer 4.

  The storage unit 23 is realized by a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. In the storage unit 23, a processing program for operating the management device 2, data used during execution of the processing program, and the like are stored in advance, or temporarily stored for each processing.

[Configuration of transfer device]
Next, the configuration of the transfer device 3 will be described. FIG. 3 is a block diagram showing an example of the configuration of the transfer device 3 shown in FIG. As illustrated in FIG. 3, the transfer device 3 includes a control unit 31, a communication unit 32, and a storage unit 33. In some cases, the transfer device 3 includes the buffer 4 and functions as the buffer control device 5.

  The control unit 31 is an electronic circuit such as a CPU or MPU, and executes various processes according to a program that defines various processing procedures. The control unit 31 includes a transfer processing unit 311, a transfer control unit 312, and a communication control unit 313.

  The transfer processing unit 311 transfers the arrived packet to the transfer destination according to a transfer table 331 (described later).

  The transfer control unit 312 controls packet transfer processing and, when receiving a transfer destination change notification, changes the transfer destination corresponding to VM1 in the transfer table to the buffer 4 according to the transfer destination change notification.

  The communication control unit 313 controls communication processing in the communication unit 32 to be described later, and communicates various information with other devices. The communication control unit 313 may have a function of establishing an L2 tunnel with an opposing device.

  The communication unit 32 is a communication interface that transmits and receives various types of information to and from devices connected via a network.

  The storage unit 33 is realized by a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk, and includes a processing program for operating the transfer device 3 and data used during execution of the processing program. Remembered. The storage unit 33 stores a transfer table 331 in which a packet destination VM and a transfer destination corresponding to the VM are associated with each other.

[Flow of packet transfer during VM migration]
With reference to FIGS. 1 and 4, the flow of packet transfer during VM migration by the management system shown in FIG. 1 will be described. FIG. 4 is a sequence diagram illustrating the flow of packet transfer during VM migration by the management system shown in FIG.

  First, when the VM1 migration process (step S1 in FIG. 4) starts, the management apparatus 2 uses the migration progress management unit 213 to manage (monitor) the progress of the migration status of the VM1 (steps S1 and S2 in FIG. 4). To start. Subsequently, the management device 2 selects a buffer as shown in (1) of FIG. 1 (step S3 of FIG. 4). Then, as shown in (2) of FIG. 1, the buffer selection unit 212 of the management device 2 notifies the MAC address of the migration target VM 1 to the buffer control device 5 that controls the selected buffer 4 (step of FIG. 4). S4), notifying the start of buffering of the packet addressed to VM1.

  In the management apparatus 2, the migration progress management unit 213 continues to manage the progress of the migration status of the VM 1 (step S5 in FIG. 4), and determines whether it is time to send the first transfer destination change notification (FIG. 4). 4 step S6). For example, the transfer destination change control unit 214 determines that it is time to transmit the first transfer destination change notification when the migration degree of the VM1 exceeds a predetermined threshold (for example, 70%). If the transfer destination change control unit 214 determines that it is not time to transmit the first transfer destination change notification (step S6: No in FIG. 4), the management device 2 returns to step S5 and continues to manage the progress of the migration status. To do.

  On the other hand, when the transfer destination change control unit 214 determines that it is time to transmit the first transfer destination change notification (step S6 in FIG. 4: Yes), the management apparatus 2 sets the migration target VM1 as the destination. A first transfer destination change notification for changing the transfer destination of the packet to be transferred to the buffer control device 5 is transmitted to each transfer device 3 (step S7 in FIG. 4).

  Each transfer device 3 that has received the first transfer destination change notification changes the transfer destination corresponding to the migration target VM 1 in accordance with the first transfer destination change notification (step S8 in FIG. 4). That is, as shown in (3) of FIG. 1, each transfer device 3 sets the route information so that the packet P addressed to the migration target VM 1 is directed to the buffer. Thereby, each transfer device 3 transfers the packet P transmitted from the terminal 6 to the migration target VM 1 to the buffer control device 5 (step S9 in FIG. 4), and the buffer control device 5 receives (4 in FIG. 1). As shown in FIG. 4, the packet P addressed to the migration target VM 1 is buffered in the buffer 4 (step S 10 in FIG. 4).

  The management apparatus 2 determines whether the migration of the VM 1 to the migration destination site is completed (Step S11 in FIG. 4). When the management apparatus 2 determines that the migration of the VM1 to the migration destination site has not been completed (step S11: No in FIG. 4), the process returns to the step S11, and the migration of the VM1 to the migration destination site is completed. Until it is done, the determination process of step S11 is performed.

  When the management apparatus 2 determines that the migration of the VM 1 to the migration destination site has been completed (step S11 in FIG. 4: Yes), as shown in (5A) of FIG. 4 is instructed to transfer the packet P addressed to the migration target VM 1 buffered to 4 to the management apparatus 2 (step S12 in FIG. 4).

  As a result, the buffer control device 5 transfers packets addressed to the migration target VM 1 buffered in the buffer 4 to the management device 2 in the order of arrival in the buffer 4 as shown in FIG. 1 (5B) (FIG. 4). Step S13). The buffer control device 5 transfers the packet to the management device 2 until there is no packet P addressed to the migration target VM 1 in the buffer 4.

  Note that the buffer control device 5 also buffers the packet addressed to the VM 1 in the buffer 4 while the packet transfer to the management device 2 is being performed. The packet transfer between the buffer control device 5 and the management device 2 may be performed via an L2 tunnel using an L2 tunneling protocol (for example, IEEE802.1ah). The L2 tunnel may be set in advance or may be established at the time of transfer.

  The management device 2 inputs the transferred packet P addressed to VM1 to VM1 of the migration destination site (step S14 in FIG. 4). Note that the buffer control device 5 may directly transfer a packet addressed to the migration target VM 1 to the VM 1 that has been migrated.

  Subsequently, the management apparatus 2 transmits a second transfer destination change notification that the transfer destination change control unit 214 changes the transfer destination of the packet destined for the migration target VM 1 to the migration destination site to each transfer apparatus 3. (Step S15 in FIG. 4).

  Each transfer device 3 that has received the second transfer destination change notification changes the transfer destination corresponding to the migration target VM 1 in accordance with the second transfer destination change notification (step S16 in FIG. 4). That is, as illustrated in (6) of FIG. 1, each transfer apparatus 3 sets route information so that a packet addressed to VM1 is directed to VM1 at the migration destination site. Thereby, each transfer device 3 transfers the packet transmitted by the terminal 6 to the VM 1 of the migration destination site (step S17 in FIG. 4).

[Effect of Embodiment 1]
In the first embodiment, during the migration of the VM1, the transfer device 3 is caused to change the transfer destination of the packet whose destination is the migration target VM1. For this reason, according to the first embodiment, since the transfer of the packet addressed to VM1 to the migration source site can be stopped without waiting for the migration of VM1 to be completed, the amount of packets discarded at the migration source site Can be reduced.

  Further, in the first embodiment, during migration of VM1, by causing each transfer device 3 to change the transfer destination of a packet destined for the migration target VM1 to the buffer control device 5, the terminal during migration of VM1 Packets addressed to VM 1 transmitted from 6 etc. are buffered in the buffer 4. In the first embodiment, after the migration of the VM 1 is completed, the buffer control device 5 is caused to transfer the buffered packet to the VM 1 that has been migrated. Therefore, according to the first embodiment, the VM 1 that has been migrated is transmitted from each terminal 6 during migration of the VM 1 regardless of the distance between the other site including the migration source site and the migration destination site. The packet addressed to VM1 can be received immediately after activation without damaging the packet.

  When using the L2 tunnel technology, a network system provided by a third party such as a provider is often used. In such a provider network, considering the cost, the bandwidth varies and the best effort type line that cannot always secure a large-capacity bandwidth is used. There is a problem that the time required for transfer varies depending on traffic conditions. For this problem, according to the first embodiment, by buffering packets addressed to VM1 transmitted during migration of VM1, VM1 transmits during migration regardless of the length of time until the completion of migration. The received packet addressed to the VM1 can be received immediately after activation without damaging the packet.

  In some cases, the external controller executes various operations in place of the management device 2. For example, a controller (not shown) may transmit a transfer destination change notification to the transfer device 3 in accordance with the progress of the VM1 migration status by the management device 2 or the VM1 migration completion. Alternatively, the controller may select the buffer 4 that buffers packets addressed to the migration target VM 1 and notify the buffer control device 5 that controls the buffer 4 of the MAC address of the migration target VM 1.

  In addition, the VM 1, the transfer device 3, and the buffer control device 5 may perform the transfer destination change notification operation by the management device 2. For example, when the tunnel termination device at the site where the management device 2 is located receives the MAC address of the migration target VM 1 and the identifier information of the site where the buffer control device 5 is located from the management device 2, Instead, the buffer control device 5 may be notified of the start of buffering of the packet of the migration target VM 1 or the first transfer destination change notification may be transmitted to each transfer device 3.

  When STP (Spanning Tree Protocol) is used, the buffer control device 5 notified of the MAC address of the VM 1 transmits a broadcast packet to each transfer device 3 as a first transfer destination change notification. The transfer destination of the packet destined for the migration target VM 1 may be changed to the buffer control device 5. In this case, the buffer control device 5 transfers the broadcast packet having the transmission source address set to the MAC address of the VM 1 to the transfer device 3. When the transfer device 3 receives the broadcast packet, the transfer device 3 looks at the arrival port of the packet and the address of the transmission source of the packet, and performs setting to output the packet as the transfer destination address to the port.

  Further, the VM 1 itself that has completed the migration may transmit a broadcast packet to each transfer apparatus 3 as a second transfer destination change notification. When the transfer device 3 receives a broadcast packet whose source address is the VM1 MAC address, the transfer device 3 associates the arrival port of the packet with the source address of the packet, and sets the VM1 MAC address. When a destination packet arrives, a setting is made to output the packet to the port.

  Examples 1 to 3 will be shown below, and the specific configuration and operation of the first embodiment will be described. The first embodiment is an example in which the buffer 4 is in a different site from the migration source site and the migration destination site. The second embodiment is an example in which the buffer 4 is in the migration destination site, and the management device 2 at the same site performs packet buffering control. The third embodiment is an example in which the buffer 4 is in the migration destination site, and the tunnel terminating device in the same site performs packet buffering control.

[Example 1 of the first embodiment]
First, Example 1 of Embodiment 1 will be described. FIG. 5 is a diagram illustrating an outline of the configuration of the management system according to the first embodiment.

  As shown in FIG. 5, the network system managed by the management system according to the first embodiment connects the sites A to D via a network 10 such as a provider network. A plurality of sites A to D are connected by L2 tunnels. Tunnel termination devices 3A to 3D are arranged at the end of the L2 tunnel at each site A to D. The tunnel termination devices 3A to 3D terminate the L2 tunnel and also have a function as the transfer device 3 that transfers the arrived packet to the transfer destination according to the transfer table. In each site A to D, there is also a communication device such as a switch that transfers the arrived packet to the transfer destination according to the transfer table.

  In the first embodiment, an example in which the VM 1 is migrated from the site A to the site C is shown. FIG. 5 illustrates a case where the migration source site A and the migration destination site C have management apparatuses 2A and 2C, respectively. FIG. 5 illustrates a case where the buffer 4 and the buffer control device 5 exist at the site D. Further, the management devices 2A and 2C obtain in advance the identifier information of the L2 tunnel toward the tunnel termination function 3C connected to the migration destination site C of VM1 from all tunnel termination devices 3A to 3D belonging to the same subnet. . Further, the identifier information refers to information necessary for packet transfer such as an IP address and an MPLS label of the tunnel termination device, as will be described later.

  In the first embodiment, during the migration of the VM1, the transmission path of the packet addressed to the migration target VM1 is set as indicated by arrows Y1 to Y3 so that the packet addressed to the migration target VM1 is transferred to the buffer 4. Yes.

  First, the tunnel termination devices 3A to 3D, which are the entrances and exits of each L2 tunnel, receive the first transfer destination change notification so that the packet addressed to the migration target VM 1 is directed to the site D where the buffer 4 exists. Change (see arrow Y1).

  Then, the tunnel termination device 3A of the migration source site A changes the transmission path of the packet addressed to the migration target VM1 in the migration source site A to a route toward the tunnel termination device 3A (see arrow Y2).

  Further, the tunnel termination device 3D of the site D where the buffer control device 5 (buffer 4) is located changes the transmission path of the packet addressed to the migration target VM 1 in the site D to a route toward the buffer control device 5 (arrow). Y3), the arrival of the packet to the buffer control device 5 is made efficient. When the buffer 4 is present in the tunnel termination device 3D at the site D, it is not necessary to change the transmission route of the packet addressed to the VM1 at the site D to the route toward the buffer control device 5 (cell M1 in FIG. 5). reference).

  Subsequently, after the migration-completed VM1 receives a packet addressed to the VM1 in the buffer 4, each of the tunnel terminating devices 3A to 3D transmits the packet addressed to the VM1 only to the VM1 of the migration destination site C. To change the transmission path of the packet addressed to VM1.

  Next, the specific configuration of the management system and the flow of operation processing in the first embodiment will be described. FIG. 6 is a schematic diagram illustrating an example of a configuration of a management system according to Example 1 of the first embodiment.

  In the management system 100A shown in FIG. 6, the management apparatus 2A that manages the progress of the migration status of the migration target VM 1 is located at the migration source site A (see cell M2). Each of the tunnel termination devices 3A to 3D has a forwarding table (for example, the forwarding table T1 of the cell M3).

[Configuration of tunnel terminating equipment]
Next, the configuration of the tunnel termination devices 3A to 3D will be described. FIG. 7 is a block diagram showing a configuration of tunnel termination apparatus 3D shown in FIG. The other tunnel termination devices 3A to 3C have the same configuration as the tunnel termination device 3D.

  As illustrated in FIG. 7, the tunnel termination device 3D includes a control unit 131, a communication unit 132, and a storage unit 133. The tunnel termination device 3D has a buffer 4 (not shown) and has the function of the buffer control device 5.

  The control unit 131 is an electronic circuit such as a CPU or MPU, similar to the control unit 31 in FIG. 3, and executes various processes according to a program that defines various processing procedures. The control unit 131 includes a transfer processing unit 1311, a transfer control unit 1312, a communication control unit 1313, an intra-site route setting unit 1314, and a buffering control unit 1315.

  The transfer processing unit 1311 has the same function as the transfer processing unit 311. The transfer processing unit 1311 performs processing for transferring a packet to be transferred to another site via the L2 tunnel.

  Specifically, when the packet to be transferred arrives, the transfer processing unit 1311 searches for the identifier information of the L2 tunnel connected to the site where the destination exists based on the MAC address of the destination of the packet. Then, the transfer processing unit 1311 passes the packet through the found L2 tunnel. In this case, the transfer processing unit 1311 attaches the tunneling protocol header to the transfer target packet, and then transfers the packet to the destination site. Also, the transfer processing unit 1311 removes the tunneling protocol header from the received packet, and transfers the packet to the destination. The transfer processing unit 1311 can employ a MAC-in-MAC method to encapsulate a packet and transfer the packet to a transfer destination.

  The transfer control unit 1312 has the same function as the transfer control unit 312 and, in the transfer table 1331 (described later), heads to the destination MAC address and the tunnel termination devices 3A to 3D connected to the site where the destination exists. The identifier information of the L2 tunnel is managed in association with each other.

  Similar to the communication control unit 313, the communication control unit 1313 controls communication processing in the communication unit 132 described later, and communicates with other devices in the site D or other sites A to D via the network 10. Various information is communicated with

  The intra-site route setting unit 1314 sets a packet transmission route within the site D to which the tunnel terminating device 3D is connected. For example, when the transfer destination of a packet destined for the migration target VM 1 is changed to the tunnel termination device 3D itself, the intra-site route setting unit 1314 sends the communication unit 132 to the switch in the site D. A broadcast packet whose source is the MAC address of VM1 is transmitted.

  The buffering control unit 1315 controls the buffering of the packet to the buffer 4 according to the received notification, and transmits the packet buffered in the buffer 4 to the outside.

  The communication unit 132 is a communication interface that transmits and receives various types of information to and from devices connected via a network. Specifically, it is configured by a communication interface that is connected to each terminal 6 in the site to which the tunnel terminating device 3D is connected, for example, via a LAN, or a communication interface that is connected to the network 10 via, for example, a WAN.

  Similarly to the storage unit 33, the storage unit 133 is realized by a semiconductor memory element or a storage device, and stores a processing program for operating the tunnel termination device 3D, data used during execution of the processing program, and the like. The storage unit 133 stores a transfer table 1331 in which the identifier information of the VM that is the destination of the packet is associated with the identifier information of the transfer destination that corresponds to the VM.

  FIG. 8 is a diagram for explaining the data configuration of the transfer table 1331 stored in the storage unit 133 shown in FIG. As illustrated in the forwarding table T1 of FIG. 8, in the forwarding table 1331, the MAC address of the VM is associated with the identifier information of the tunnel terminating device at the site where the VM is located. For example, before migration of VM1, since VM1 is located at migration source site A, the MAC address of VM1 is associated with identifier information of tunnel termination device 3A of site A.

  In the case of a network system to which the VXLAN protocol is applied, as shown in FIG. 9, the tunnel termination device 3D has a VM1 with a VM1 MAC address (for example, “aa: bb: cc: dd: ee: ff”). The forwarding table Ta associated with the IP address (for example, “192.168.11.1”) of the tunnel terminating device 3A connected to the existing LAN (site A) is managed. Further, in the case of a network system to which the MPLS-EVVPN protocol is applied, as shown in FIG. 10, the tunnel termination device 3D has a VM 1 MAC address (for example, “aa: bb: cc: dd: ee: ff”), It manages the forwarding table Tb associated with the MPLS tunnel label (for example, “Label: 124”) connected to the tunnel terminating device 3A of the site A where the VM1 exists.

[Flow of packet transfer during VM migration]
Next, the flow of packet transfer during VM migration in the management system 100A shown in FIG. 6 will be described. FIGS. 11 to 14 are schematic diagrams for explaining the flow of packet transfer during VM migration by the management system 100A shown in FIG. In the management system 100A, it is determined in advance that the buffer 4 of the tunnel termination device 3D is a buffer for buffering packets addressed to VM1 at the time of VM migration, and the tunnel termination device 3D performs buffering for the buffer 4. It also has a function as a buffer control device 5 that performs control.

  First, when the live migration of the VM 1 starts, the management device 2A manages the progress of the live migration of the VM 1 in the migration progress management unit 213 (see (1) in FIG. 11).

  When the management apparatus 2A determines that it is time to transmit the first transfer destination change notification based on the progress of live migration of the VM 1 (see the cell M2a in FIG. 11) (see the cell M2a in FIG. 11), The tunnel termination apparatus 3A connected to the same site A is notified as the first transfer destination change notification of the MAC address of the VM 1 being migrated and the address of the tunnel termination apparatus 3D connected to the site where the buffer 4 exists ( (See (2) in FIG. 11). For example, when the migration degree of VM1 exceeds a threshold (for example, 70%) (see the cell M2a in FIG. 11), the transfer destination change control unit 214 notifies the first transfer destination change notification.

  In the tunnel termination device 3A connected to the migration source site A, the intra-site route setting unit 1314 acquires the MAC address of the VM 1 being migrated, and then transmits a broadcast packet having the MAC address as the transmission source to the migration source site A. Send in. As a result, the switch in the site A changes the transmission destination of the packet addressed to the VM 1 being migrated to the tunnel terminating device 3A. As a result, the tunnel terminating device 3A at the site A can receive a packet addressed to the MAC address of the VM 1 being migrated from a terminal existing in the migration source site A, and connects it to the migration destination site D. Can be efficiently transferred to the tunnel terminating device 3D (see (2 'in FIG. 11)).

  Then, the tunnel termination device 3A notifies the other tunnel termination devices 3B to 3D of the first transfer destination change notification, and the tunnel termination devices 3B to 3D having received the notification transfer packets addressed to the VM 1 during live migration. The destination is changed to the address of the tunnel termination device 3D of the site D where the buffer 4 exists (see (3) in FIG. 11). Specifically, as shown in the cell M4 in FIG. 11, the tunnel termination device 3B uses the identifier information of the tunnel termination device associated with the MAC address of the VM1 in the forwarding table T1 as the tunnel termination device 3A at the site A. To the tunnel terminating device 3D at the site D (see the forwarding table T1a). The forwarding table is similarly changed in the other tunnel termination devices 3A, 3C, and 3D.

  Further, the tunnel terminating device 3A notifies the tunnel terminating device 3D of the site D where the buffer 4 exists so that the packet whose destination is the MAC address of the VM 1 being migrated is recorded in the buffer 4 (FIG. 11). (Refer to (4)).

  When a packet addressed to the VM 1 being migrated arrives, each of the tunnel terminating devices 3A to 3C checks the MAC address, and transmits it toward the site D where the buffer 4 exists according to the forwarding table T1a (see the cell M4 in FIG. 12). (Refer to (5) in FIG. 12). Then, the tunnel terminating device 3D at the site D stores the packet addressed to the VM 1 being migrated in the buffer 4 in the tunnel terminating device 3D (see (6-1) in FIG. 13).

  After the migration of VM1 to site C is completed, VM1 of site C sets up L2 tunnel 7 between buffer 4 and VM1 of site D, and then sends a notification prompting packet transfer via L2 tunnel 7 The data is transmitted to the tunnel termination device 3D (see (7) in FIG. 14). After receiving this notification, the tunnel terminating device 3D of the site D transmits the packet addressed to the VM1 in the buffer 4 to the VM1 of the site C via the L2 tunnel 7, and the packet addressed to the VM1 in the buffer 4 disappears. The VM 1 is notified to transmit a broadcast packet (see (8) in FIG. 14).

  As a result, the VM1 transmits a broadcast packet in the L2 network. The switch and tunnel terminating devices 3A to 3D in the L2 network that received this broadcast packet change the transfer destination of the packet addressed to the MAC address of VM1 to the migration destination site C of VM1 ((9) in FIG. 14). reference). Specifically, as shown in the cell M6 in FIG. 14, the tunnel termination device 3B uses the identifier information of the tunnel termination device associated with the MAC address of VM1 in the forwarding table T1a, as the tunnel termination device 3D at the site D. To the tunnel terminating device 3C at site C (see transfer table T1b). The forwarding table is similarly changed in the other tunnel termination devices 3A, 3C, and 3D.

  In the management system 100A, each device performs the processing in the flow described with reference to FIGS. 11 to 14, so that the VM 1 that has been migrated loses the packet addressed to the VM 1 transmitted from each terminal during the migration of the VM 1. It becomes possible to receive immediately after activation.

  Further, as illustrated in FIG. 15, the buffer 4 may exist in a device different from the tunnel termination device 3D. FIG. 15 is a diagram illustrating an outline of another configuration of the management system according to Example 1 of the first embodiment. In the configuration shown in FIG. 15, the tunnel termination device 3 </ b> D at the site D is independent of the buffer 4 in the site D and the buffer control device 5 that controls the buffer 4. An L2 tunnel 7D is set in advance between the tunnel termination device 3D and the buffer control device 5.

  In the example illustrated in FIG. 15, when the first transfer destination change notification is received, the tunnel termination devices 3A to 3D change the packet addressed to the migration target VM 1 so as to go to the site D (see arrow Y1). The tunnel termination device 3A of the original site A only needs to change the transmission path of the packet addressed to VM1 in the migration source site A to a route toward the tunnel termination device 3A (see arrow Y2).

  The tunnel termination device 3D at the site D can transfer the packet addressed to the VM 1 being migrated to the buffer 4 via the L2 tunnel 7D (see arrow Y3A), and the transfer processing from the tunnel termination device 3D to the buffer 4 is made efficient. it can. Further, the tunnel terminating device 3D does not need to change the route in the site D, and a packet addressed to the VM 1 transmitted from a terminal or the like in the site D is first transferred to the tunnel terminating device 3D (see arrow Y4). ) And transferred to the buffer control device 5 by the tunnel termination device 3D (see arrow Y3A).

  In this way, even when the buffer 4 exists in a device different from the tunnel termination device 3D, the L2 tunnel 7D is set in advance between the tunnel termination device 3D and the buffer control device 5, and the L2 tunnel By transferring the packet to be buffered via 7D, the buffering of the packet to the buffer 4 can be made efficient. The tunnel termination device 3D may transfer the packet to the buffer 4 using Mac-in-Mac without going through the L2 tunnel 7D.

[Example 2 of the first embodiment]
Next, the specific configuration of the management system and the flow of operation processing in Example 2 of Embodiment 1 will be described. FIGS. 16 to 18 are schematic diagrams for explaining a packet transfer flow at the time of VM migration by the management system 100B in the second example of the first embodiment.

  The management system 100B is different from the management system 100A shown in FIG. 6 in that the management device 2C that manages the progress of the migration status of the VM 1 is located at the migration destination site C. Further, the management device 2C has a buffer 4 therein, and also performs buffering control for the buffer 4.

[Flow of packet transfer during VM migration]
First, when live migration of VM1 starts from site A, the management apparatus 2C manages the migration status of VM1, and when the degree of migration exceeds a threshold, a broadcast packet having the MAC address of VM1 being migrated as a transmission source is transmitted. It transmits within the L2 network (see (1) in FIG. 16).

  Each of the tunnel termination devices 3A to 3D that has received the broadcast packet changes the transfer destination of the packet addressed to the VM 1 being migrated to the tunnel termination device 3C connected to the migration destination site C (see (2) in FIG. 16). . For example, as shown in the cell M7 in FIG. 16, the tunnel termination device 3B sends the identifier information of the tunnel termination device associated with the MAC address of the VM1 from the tunnel termination device 3A at the site A to the site in the forwarding table T1. Change to C tunnel terminating device 3C (see transfer table T1b).

  The switch in the site C that has received the broadcast packet changes the transmission destination of the packet addressed to the VM 1 being migrated to the tunnel termination device 3C connected to the migration destination site C (see (3) in FIG. 16).

  Subsequently, the management device 2C receives a packet destined for the MAC address of the VM 1 being migrated via the tunnel termination device 3C in accordance with the transfer destination change of the packet addressed to the VM 1 to the site C by each transfer device. In this case, it is recorded in the buffer 4 (see (4) in FIG. 17).

  When the management apparatus 2C detects the migration completion of the VM1, the management apparatus 2C transmits the packet addressed to the VM1 buffered in the buffer 4 to the VM1 (see (5) in FIG. 18). In this case, the management apparatus 2C may transmit the packet to the VM1 via an L2 tunnel set between the management apparatus 2C and the VM1. In addition, when the VM 1 is managed by the management device 2C, the management device 2C may transmit the packet to the VM 1 by inter-process communication. Further, the management apparatus 2C may transfer the packet to the VM 1 using Mac-in-Mac.

  When the packet transmission in the buffer 4 to the VM 1 is completed, the management device 2C requests the VM 1 to transmit a broadcast packet (see (6) in FIG. 18). In response to this, the VM 1 transmits a broadcast packet in the L2 subnet as a second transfer destination change notification (see (7) in FIG. 18). The tunnel terminating devices 3A to 3D and each switch that have received the broadcast packet from VM1 start forwarding packets addressed to VM1 toward site C.

  In the management system 100B, each device performs processing in the flow described with reference to FIGS. 16 to 18, so that the VM 1 that has been migrated loses the packet addressed to the VM 1 transmitted from each terminal during the migration of the VM 1. It becomes possible to receive immediately after activation.

[Example 3 of the first embodiment]
FIG. 19 is a diagram illustrating an outline of a configuration of a management system 100C according to Example 3 of the first embodiment. The management system 100C is different from the management system 100A shown in FIG. 6 in that the tunnel termination device 3C of the migration destination site C has the buffer 4 and also functions as the buffer control device 5.

  In the management system 100C, when the first transfer destination change notification is received, the tunnel termination devices 3A to 3D change the packets addressed to the migration target VM1 so as to go to the site C where the buffer 4 exists (arrow Y1C). reference). Specifically, as shown in the cell M10, the tunnel termination device 3B sends the identifier information of the tunnel termination device associated with the MAC address of the VM1 from the tunnel termination device 3A of the site A to the site in the forwarding table T1. Change to C tunnel terminating device 3C (see transfer table T1b). The forwarding table is similarly changed in the other tunnel termination devices 3A, 3C, and 3D.

  Then, the tunnel termination device 3A of the migration source site A changes the transmission path of the packet addressed to VM1 in the site A to a route toward the tunnel termination device 3A (see arrow Y2). Since the buffer 4 is provided in the tunnel termination device 3C at the site C, it is not necessary to change the transmission path of the packet addressed to the VM1 at the site C. A packet addressed to VM1 transmitted from a terminal or the like in site C is first transferred to tunnel terminating device 3C (see arrow Y3C) and buffered in buffer 4 of tunnel terminating device 3C. The subsequent flow is the same as in the first and second embodiments.

[Embodiment 2]
In the first embodiment, the case where the packet addressed to the VM1 is transferred to the buffer 4 during the migration of the VM1 has been described. However, in addition to the buffer 4, the packet addressed to the VM1 may be transferred to the migration source site. . Thus, in the second embodiment, a configuration will be described in which a packet addressed to VM1 is transferred to both the buffer 4 and the migration source site during migration of VM1.

  FIG. 20 is a diagram illustrating an outline of the operation of the management system according to the second embodiment. As shown in FIG. 20, in the management system according to the second embodiment, compared to the management system shown in FIG. 5, during migration of VM1, packets addressed to migration target VM1 are added to buffer 4 and migrated from The transmission path of the packet addressed to the migration target VM1 is set as indicated by arrows Y21 to Y25 so that it is also transferred to the other site.

  First, the management device 2A changes the transfer destination of the packet destined for the migration target VM1 to the buffer control device 5 and the VM1 of the migration source site A for each of the tunnel termination devices 3A to 3D and each switch. A first transfer destination change notification is transmitted.

  Then, the tunnel termination devices 3A to 3D, which are the entrances and exits of each L2 tunnel, receive the first transfer destination change notification so that the packet addressed to the migration target VM1 is transferred to the site D where the buffer 4 exists and the migration source. (See arrow Y21 and arrow Y22). Subsequently, the tunnel terminating device 3A connected to the migration source site A provides an L2 tunnel 7A with the VM 1 being migrated, and forwards the packet addressed to the VM 1 via the L2 tunnel 7A (see arrow Y25). .

  Similarly to FIG. 5, the tunnel termination device 3A of the migration source site A changes the transmission path of the packet addressed to VM1 in the migration source site A to a route toward the tunnel termination device 3A (see arrow Y23). ). Then, the tunnel termination device 3D of the site D where the buffer control device 5 (buffer 4) is located changes the transmission path of the packet addressed to VM1 in the site D to a route toward the buffer control device 5 (see arrow Y24).

  Subsequently, after the migration-completed VM1 receives the packet addressed to the VM1 in the buffer 4, each of the tunnel termination devices 3A to 3D transmits the packet addressed to the VM1 only to the VM1 of the migration destination site C. The transmission path of the packet addressed to VM1 is changed.

[Effect of Embodiment 2]
In the second embodiment, during migration of VM1, each transfer device 3 is added with a migration source site in addition to buffer control device 5 as a transfer destination of a packet destined for migration target VM1. Thus, the packet addressed to VM1 is buffered in the buffer 4, and the packet addressed to VM1 is received and migrated to the migration destination even during migration of VM1. Therefore, in the second embodiment, after the migration of the VM 1 is completed, the buffer control device 5 should transfer only the packets in the buffer 4 that have arrived at the buffer 4 after the VM 1 of the migration source site A is stopped. It ’s enough.

  As described above, the second embodiment has the same effect as the first embodiment, and can reduce the amount of packets transferred from the buffer 4 to the VM 1 after the migration is completed. Therefore, according to the second embodiment, it is possible to shorten the packet transfer time from the buffer 4 to the VM 1 after the migration is completed, and accordingly, the time until the transfer destination is changed in the tunnel termination devices 3A to 3D after the migration is completed. Can also be shortened.

  The specific configuration and operation of the second embodiment will be described below with reference to Examples 1 and 2. The first embodiment is an example in which a packet addressed to the VM 1 is transferred to both the migration source site and the buffer 4 in a different site from the migration source site and the migration destination site during migration. The second embodiment is an example in which a packet addressed to the VM 1 is transferred to both the migration source site and the buffer 4 in the migration destination site during migration.

[Example 1 of Embodiment 2]
First, Example 1 of Embodiment 2 will be described. 21 to 23 are diagrams for explaining a packet transfer flow during VM migration in the management system 200A according to Example 1 of the second embodiment. The management system 200A shown in FIG. 21 has the same configuration as the management system 100A (see FIG. 6) in which the tunnel termination device 3D at the site D has the buffer 4, and the operation of each device is different.

  First, when the live migration of the VM1 starts, the management apparatus 2A manages the progress of the live migration of the VM1 (see (1) in FIG. 21).

  When the management apparatus 2A determines that it is the timing to transmit the first transfer destination change notification based on the progress of the live migration of the VM 1 (see the cell M2a in FIG. 21), the tunnel termination connected to the same site A In the apparatus 3A, the MAC address of the VM 1 being migrated, the address of the tunnel termination apparatus 3D connected to the site D where the buffer 4 exists, and the address of the tunnel termination apparatus 3A of the migration source site A are transferred to the first transfer destination. Notification is made as a change notification (see (2) in FIG. 21).

  Then, the tunnel termination device 3A notifies the other tunnel termination devices 3B to 3D of the first transfer destination change notification, and the tunnel termination devices 3B to 3D having received the notification transfer packets addressed to the VM 1 during live migration. The destination is changed to the tunnel termination device 3D connected to the site D where the buffer 4 exists and the tunnel termination device 3A of the migration source site A (see (3) in FIG. 21). Specifically, as shown in the cell M21 of FIG. 21, the tunnel termination device 3B uses the identifier information of the tunnel termination device associated with the MAC address of the VM1 in the forwarding table T1 as the tunnel termination device 3A of the site A. To the tunnel termination device 3D of the site D (refer to the transfer table T2a). The forwarding table is similarly changed in the other tunnel termination devices 3A, 3C, and 3D.

  Furthermore, the tunnel terminating device 3A notifies the tunnel terminating device 3D of the site D having the buffer 4 so that the packet destined for the MAC address of the VM 1 is recorded in the buffer 4 (see (4) in FIG. 21). ).

  As a result, when a packet addressed to the VM 1 being migrated arrives, each of the tunnel terminating devices 3A to 3D checks the MAC address, and according to the forwarding table T2a (see the cell M21 in FIG. 21), the buffer 4 Transmission is performed toward the existing site D and the migration source site A (see (5) in FIG. 21). The tunnel termination device 3A of the migration source site A transfers the packet addressed to the VM 1 being migrated via the L2 tunnel 7A (see (5 ′) in FIG. 21). On the other hand, the tunnel terminating device 3D at the site D stores the packet addressed to the VM1 in the buffer 4 (see (6) in FIG. 21).

  After completing the migration of VM1 to site C, VM1 of site C sets up L2 tunnel 7 between buffer 4 and VM1 of site D, and then tunnels a notification that prompts packet transfer via L2 tunnel 7. It transmits to the termination | terminus apparatus 3D (refer (7) of FIG. 22). After receiving this notification, the tunnel terminating device 3D of the site D transmits a packet addressed to the VM1 in the buffer 4 to the VM1 of the site C through the L2 tunnel 7, and when there is no packet in the buffer 4, the VM1 Is notified to transmit a broadcast packet (see (8) in FIG. 22). At this time, it is sufficient for the tunnel terminating device 3D to transfer only the packets in the buffer 4 that have arrived at the buffer 4 after the VM 1 of the migration source site A is stopped.

  As a result, the VM 1 transmits a broadcast packet in the L2 network as a second transfer destination change notification (see (9) in FIG. 23). The received tunnel termination device 3C starts transmission of a D-Plane packet toward VM1 (see (10) in FIG. 23). In other words, the tunnel termination device 3C transfers the packet addressed to VM1 that has arrived at the tunnel termination device 3C to the destination VM1 of site C.

  At the same time, the tunnel termination device 3C notifies the broadcast packet to the other tunnel termination devices 3A, 3B, 3D. The tunnel terminating devices 3A, 3B, and 3D receive it and stop the transfer to the migration source site A (see (11) in FIG. 23). In other words, the tunnel termination devices 3A to 3D change the transfer destination of the packet addressed to the MAC address of VM1 to the tunnel termination device 3C of the migration destination site C of VM1.

  Specifically, as shown in the cell M22 in FIG. 23, the tunnel termination device 3B uses the identifier information of the tunnel termination device associated with the MAC address of the VM1 in the forwarding table T2a, and the tunnel termination device 3A at the site A. And the tunnel termination device 3D at the site D is changed to the tunnel termination device 3C at the site C (refer to the transfer table T2b). The forwarding table is similarly changed in the other tunnel termination devices 3A, 3C, and 3D.

  FIG. 24 is a sequence diagram for explaining the flow of packet transfer during VM migration by the management system 200A. Steps S21 to S26 shown in FIG. 24 are steps S1 to S6 shown in FIG. In the management system 200A, since the tunnel termination device 3D has the buffer 4 inside and has the function of the buffer control device 5, the buffering start notification in step S24 is transmitted to the tunnel termination device 3D at the site D. The

  If the management device 2A determines that it is time to transmit the first transfer destination change notification (step S26: Yes), the management device 2A sets the transfer destination of the packet destined for the migration target VM1 as the function of the buffer control device 5. A first transfer destination change notification to be changed to the tunnel termination device 3D of the site D and the migration source site A is transmitted to each of the tunnel termination devices 3A to 3D (step S27). Each of the tunnel termination devices 3A to 3D changes the transfer destination corresponding to the migration target VM1 to the tunnel termination device 3D and the migration source site A in accordance with the first transfer destination change notification (step S28).

  As a result, each of the tunnel termination devices 3A to 3D transfers the packet transmitted by the terminal 6 to the VM 1 to the tunnel termination device 3D having the buffer 4 (steps S29-1 to S29-3), and the migration source It is also transferred to the tunnel terminating device 3A at the site A (step S30-1 to step S30-3).

  Next, the tunnel terminating device 3A transfers the packet addressed to the VM 1 to the VM 1 being migrated at the site A (step S31-1 to step S31-3). Further, the tunnel terminating device 3D buffers the packet addressed to the VM 1 in the buffer 4 (step S32).

  When migration of VM1 is completed at the migration destination site A (step S33), the VM1 of site A instructs the tunnel termination device 3D to transfer a packet addressed to VM1 in the buffer 4 (step S34). In response to this instruction, the tunnel termination device 3D transfers, to the VM 1 in the order in which it arrived at the buffer 4, among the packets addressed to the VM 1 in the buffer 4, which arrived at the buffer 4 after the stop of the VM 1 at the site A. (Step S35).

  Subsequently, the VM 1 of the site C transmits a broadcast packet to the tunnel terminating device 3C as the second transfer destination change notification (step S36). The tunnel termination device 3C receives the broadcast packet, and changes the transfer destination of the packet addressed to the MAC address of the VM1 to the tunnel termination device 3C of the migration destination site C (step S37). Then, the tunnel termination device 3C notifies the broadcast packet to the other tunnel termination devices 3A, 3B, 3D (step S38). The tunnel termination devices 3A, 3B, and 3D receive the broadcast packet and change the transfer destination of the packet addressed to the MAC address of the VM1 to the tunnel termination device 3C of the migration destination site C (step S39).

  In the management system 200A, each device performs processing according to the flow described in FIGS. 21 to 24, so that the VM 1 that has been migrated does not impair the packets addressed to the VM 1 transmitted from each terminal during the migration of the VM 1. In addition to being able to receive data, the amount of packets transferred to the VM 1 after the migration is completed can be reduced, so that the time required to change the transfer destination in the tunnel terminating devices 3A to 3D after the migration is completed can be shortened.

[Example 2 of Embodiment 2]
Next, Example 2 of the second embodiment will be described. FIG. 25 and FIG. 26 are diagrams for explaining a packet transfer flow during VM migration in the management system 200B according to the second embodiment. The management system 200B shown in FIG. 25 has the same configuration as the management system 100C shown in FIG. 19, and the operation of each device is different.

  First, when the live migration of the VM1 starts, the management device 2A manages the progress of the live migration of the VM1 (see (1) in FIG. 25).

  When the management apparatus 2A determines that it is the timing to transmit the first transfer destination change notification based on the progress of the live migration of the VM 1 (see the cell M2a in FIG. 21), the tunnel termination connected to the same site A In the device 3A, the MAC address of the VM 1 being migrated, the address of the tunnel termination device 3C of the migration destination site C connected to the site C where the buffer 4 exists, and the address of the tunnel termination device 3A of the migration source site A, (See (2) in FIG. 25).

  The tunnel termination device 3A notifies the other tunnel termination devices 3B to 3D of the first transfer destination change notification, and the tunnel termination devices 3B to 3D that have received the notification indicate the transfer destination of the packet addressed to the VM 1 during live migration. The tunnel termination device 3C connected to the migration destination site C where the buffer 4 exists and the tunnel termination device 3A of the migration source site A are changed (see (3) in FIG. 25). Specifically, as shown in the cell M23 in FIG. 25, the tunnel termination device 3B uses the identifier information of the tunnel termination device associated with the MAC address of the VM1 in the forwarding table T1 as the tunnel termination device 3A at the site A. To the tunnel termination device 3C at the site C (refer to the transfer table T2c). The forwarding table is similarly changed in the other tunnel termination devices 3A, 3C, and 3D.

  As a result, when a packet addressed to the VM 1 being migrated arrives, each of the tunnel termination devices 3A to 3D checks the MAC address, and in accordance with the forwarding table T2c (see the cell M23 in FIG. 25), the site C where the buffer 4 exists and It transmits toward the migration source site A (see (4) in FIG. 25). Note that the tunnel termination device 3A of the migration source site A may transfer a packet addressed to the VM 1 being migrated via the L2 tunnel 7A between the tunnel termination device 3A and the VM 1 being migrated.

  On the other hand, the management apparatus 2A notifies the tunnel termination apparatus 3C of the site C to discard the packet destined for the MAC address of the VM 1 being migrated via the tunnel termination apparatus 3A (( See 5)). As a result, the tunnel terminating device 3C at the site C discards the packet addressed to VM1 that has arrived at the tunnel terminating device 3C (see (6) in FIG. 25).

  Then, the VM 1 of the site C that has finished the live migration transmits a broadcast packet as the second transfer destination change notification (see (7) in FIG. 26). The tunnel terminating device 3C that has received the broadcast packet starts transmission of a D-Plane packet toward the VM 1 (see (8) in FIG. 26). Note that the tunnel termination device 3C is in a state where the VM1 is stopped at the site A and the packet addressed to VM1 from the other tunnel termination devices 3A, 3B, 3D is transferred to the tunnel termination device 3C. A packet addressed to VM1 that has arrived at the terminating device 3C may be buffered in the buffer 4.

  At the same time, the tunnel termination device 3C notifies the broadcast packet to the other tunnel termination devices 3A, 3B, 3D. The other tunnel termination devices 3A, 3B, 3D receive the notification and stop the packet transfer to the migration source site A (see (9) in FIG. 26). Specifically, as shown in the cell M24 of FIG. 26, the tunnel termination device 3B uses the identifier information of the tunnel termination device associated with the MAC address of the VM1 in the forwarding table T2c, and the tunnel termination device 3A at the site A. And the tunnel termination device 3C at the site C is changed to the tunnel termination device 3C at the site C (refer to the transfer table T2d). The forwarding table is similarly changed in the other tunnel termination devices 3A, 3C, and 3D.

  As in Example 2 of the second embodiment, during live migration, packets addressed to VM1 are migrated from site A to site C. Therefore, until VM1 stops at site A, VM1 is stored in buffer 4 at site C. There is no need to buffer destination packets. Therefore, the tunnel termination device 3C of the migration destination site C discards the packet whose destination is the MAC address of the VM 1 being migrated until the VM 1 stops at the site A, thereby buffering the amount of packets. Is reduced.

[System configuration, etc.]
Each component of each illustrated device is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part of the distribution / integration is functionally or physically distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or a part of each processing function performed in each device can be realized by a CPU and a program that is analyzed and executed by the CPU, or can be realized as hardware by wired logic.

  In addition, among the processes described in this embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
FIG. 27 is a diagram illustrating an example of a computer in which a management system is realized by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to the display 1130, for example.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the network integration system is implemented as a program module 1093 in which a code executable by a computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration in the network integrated system is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

  The setting data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 and executes them as necessary.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN, WAN, etc.). Then, the program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

  As mentioned above, although embodiment which applied the invention made | formed by this inventor was described, this invention is not limited with the description and drawing which make a part of indication of this invention by this embodiment. That is, other embodiments, examples, operation techniques, and the like made by those skilled in the art based on this embodiment are all included in the scope of the present invention.

1 VM
2, 2A, 2C Management device 3 Transfer device 3A-3D Tunnel termination device 4 Buffer 5 Buffer control device 6 Terminal 21, 31, 131 Control unit 22, 32, 132 Communication unit 23, 33, 133 Storage unit 211 Identifier acquisition unit 212 Buffer selection unit 213 Migration progress management unit 214 Transfer destination change control unit 215, 313, 1313 Communication control unit 311, 1311 Transfer processing unit 312, 1312 Transfer control unit 331, 1331 Transfer table 100A to 100C, 200A, 200B Management system 1314 Site Inner path setting unit 1315 Buffering control unit

Claims (7)

In a network system having a plurality of sites and a tunnel path connecting the plurality of sites, a management system having a management device that manages migration of virtual machines between sites, a transfer device that transfers packets, and a buffer control device Because
The management device
A migration progress management unit that monitors the progress of the migration status of the virtual machine when the virtual machine is migrated to an apparatus operating at another site;
A transfer destination that changes a transfer destination of a packet addressed to a virtual machine to be transferred to a buffer that buffers a packet transmitted to the virtual machine during the migration of the virtual machine according to progress of the migration status of the virtual machine A transfer destination change control unit that transmits a change notification to at least the transfer device during migration of the virtual machine;
Have
When the transfer device receives the transfer destination change notification, the transfer device transfers the migration in the transfer table in which the virtual machine that is the packet destination and the transfer destination corresponding to the virtual machine are associated with each other according to the transfer destination change notification. a transfer destination corresponding to the target virtual machine to have a transfer control unit to change the buffer,
The buffer control device includes:
A buffering controller for controlling buffering of packets to the buffer according to the received notification;
A transfer processing unit for transmitting a packet buffered in the buffer to a transfer destination;
Have
The transfer destination change control unit notifies the buffer controller of the start of buffering of a packet addressed to the migration target virtual machine during the migration of the virtual machine, and notifies the transfer device of the migration target A first transfer destination change notification for changing a transfer destination of a packet addressed to a virtual machine to the buffer control device and the migration source site is transmitted, and after the migration of the virtual machine is completed, the buffer control device is sent to the buffer control device. That the packet addressed to the virtual machine to be migrated is transferred to the virtual machine that has been migrated, and the forwarding destination of the packet addressed to the virtual machine to be migrated is notified to the transfer device. Send a second transfer destination change notification to the destination site,
When the transfer control unit receives the first transfer destination change notification, the transfer control unit changes the transfer destination corresponding to the migration target virtual machine in the transfer table to the buffer control device and the transfer source site, and When the transfer destination change notification of 2 is received, the transfer destination of the packet addressed to the migration target virtual machine is changed to the migration destination site,
When the transfer processing unit receives the second transfer destination change notification, the transfer processing unit transfers only the packets that have reached the buffer after the migration target virtual machine is stopped, among the packets in the buffer. Management system.   The transfer device is a tunnel termination device at each site that terminates the tunnel path at the migration destination site,
  The buffer control device is provided at the migration destination site,
  Among the transfer devices, the transfer device at the migration destination site receives the first transfer destination change notification and receives the second transfer destination change notification. The management system according to claim 1, wherein the packet is discarded.   The transfer destination change control unit is predetermined based on timing when the degree of migration of the virtual machine exceeds a predetermined threshold, or information combining the measurement information of the traffic status of the network system and the migration status of the virtual machine. The management system according to claim 1, wherein the transfer destination change notification is transmitted at a timing calculated using the algorithm. The transfer device is a tunnel termination device at each site that terminates the tunnel path,
The tunnel termination device has a route setting unit for setting a transmission route of a packet in a site to which the tunnel termination device is connected,
The path setting unit, when the tunnel termination device receives the first transfer destination change notification, when the tunnel termination device is the tunnel termination device of the migration source site, the migration within the migration source site. The transmission path of the packet addressed to the target virtual machine is set to a path directed to the tunnel termination device of the migration source site, and when the tunnel termination device is the tunnel termination device of the site where the buffer control device is located, the buffer 2. The management system according to claim 1 , wherein a transmission path of a packet addressed to the migration target virtual machine in a site where the control apparatus is located is set as a path toward the buffer control apparatus. A layer 2 tunnel exists between the tunnel termination device at the site where the buffer control device is located and the buffer control device,
When the tunnel termination device at the site where the buffer control device is located receives the first transfer destination change notification, the buffer control device sends a packet addressed to the virtual machine to be migrated via the layer 2 tunnel. The management system according to claim 4, wherein the management system is transferred to the management system. Management executed by a management device that manages migration of virtual machines between sites, a transfer device that transfers packets, and a buffer control device in a network system having a plurality of sites and tunnel paths connecting the plurality of sites A method,
The management device monitoring the progress of the migration status of the virtual machine when the virtual machine migrates to a device operating at another site;
A buffer for buffering packets sent to the virtual machine during the migration of the virtual machine by the management apparatus in accordance with the progress of the migration status of the virtual machine A transfer destination change control step of transmitting a transfer destination change notification to be changed to at least the transfer device during the migration of the virtual machine;
When the transfer device receives the transfer destination change notification, the transfer in the transfer table in which a virtual machine that is a packet destination and a transfer destination corresponding to the virtual machine are associated with each other according to the transfer destination change notification A transfer control step of changing the transfer destination corresponding to the target virtual machine to the buffer;
The buffer control device controlling the buffering of packets to the buffer according to the received notification;
A transfer processing step in which the buffer control device transmits a packet buffered in the buffer to a transfer destination;
Including
In the transfer destination change control step, the management device notifies the buffer control device of the start of buffering of the packet addressed to the virtual machine to be migrated during the migration of the virtual machine, and the transfer device A first transfer destination change notification for changing a transfer destination of a packet addressed to the virtual machine to be migrated to the buffer control device and the migration source site, and after the completion of the migration of the virtual machine, the buffer control device To transfer the packet addressed to the migration target virtual machine buffered in the buffer to the virtual machine that has been migrated, and to the transfer device addressed to the migration target virtual machine Send a second transfer destination change notification to change the packet transfer destination to the migration destination site,
When the transfer device receives the first transfer destination change notification in the transfer control step, the transfer destination corresponding to the migration target virtual machine in the transfer table is transferred to the buffer control device and the transfer source site. And when the second transfer destination change notification is received, change the transfer destination of the packet addressed to the virtual machine to be migrated to the migration destination site,
When the buffer control device receives the second transfer destination change notification in the transfer processing step, only the packets that reach the buffer after the migration-target virtual machine is stopped among the packets in the buffer. The management method characterized by transferring . A management program for causing a computer to function as the management device , the transfer device, or the buffer control device according to any one of claims 1 to 5 .

Images