US20210149701A1 - Network updates for virtual machine migration - Google Patents
Network updates for virtual machine migration Download PDFInfo
- Publication number
- US20210149701A1 US20210149701A1 US16/689,320 US201916689320A US2021149701A1 US 20210149701 A1 US20210149701 A1 US 20210149701A1 US 201916689320 A US201916689320 A US 201916689320A US 2021149701 A1 US2021149701 A1 US 2021149701A1
- Authority
- US
- United States
- Prior art keywords
- host
- notification
- virtual machine
- source host
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4641—Virtual LANs, VLANs, e.g. virtual private networks [VPN]
- H04L12/4675—Dynamic sharing of VLAN information amongst network nodes
- H04L12/4679—Arrangements for the registration or de-registration of VLAN attribute values, e.g. VLAN identifiers, port VLAN membership
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/1886—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast with traffic restrictions for efficiency improvement, e.g. involving subnets or subdomains
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/02—Topology update or discovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45591—Monitoring or debugging support
Definitions
- the disclosure is generally related to virtualization systems, and is more specifically related to network updates for virtual machine migration.
- Data centers may include clusters of multiple hosts (e.g., physical servers) in racks.
- Hypervisors may operate on each host to create and run virtual machines (VMs).
- VMs emulate computer systems and may be referred to as guest machines.
- the hosts in the clusters may be connected via one or more wired (e.g., Ethernet) and/or wireless (e.g., WiFi) networks (e.g., the Internet, local area network (LAN)).
- wired e.g., Ethernet
- wireless e.g., WiFi
- a VM on a source host machine may be migrated to a destination host machine within the cluster.
- various components of the networks may be updated with address updates for the VM.
- FIG. 1 depicts a high-level diagram of an example system architecture operating in accordance with one or more aspects of the disclosure
- FIG. 2 depicts a flow diagram of an example method for network updates for virtual machine migration, in accordance with one or more aspects of the disclosure
- FIG. 3 depicts a block diagram of an example computer system, in accordance with one or more aspects of the disclosure
- FIG. 4 depicts a flow diagram of another example method for network updates for virtual machine migration by a notification component, in accordance with one or more aspects of the disclosure
- FIG. 5 depicts a flow diagram of another example method for network updates virtual machine migration by a notification component, in accordance with one or more aspects of the disclosure
- FIG. 6 depicts a block diagram of an illustrative computing device operating in accordance with the examples of the disclosure.
- a virtual machine may execute via a source hypervisor on a source host in a managed virtualization environment.
- the VM may be associated with a first address (e.g., a media access control (MAC) address tagged with a virtual local area network (VLAN) address) that corresponds to the source host.
- the source host may be connected to other hosts via a network.
- components e.g., switches
- the network may inspect the packets and determine that the first address of the VM is coming from a port associated with the source host. This information may be added to a forwarding table maintained by each of the components of the network. As such, when the components receive packets from other hosts that include a destination address set to the address of the VM, the components may forward the packets using the port to the source host.
- a topology of the network may change, for example, when a VM executing on the source host is migrated to a destination host.
- a VM may be migrated from one host to another host for a variety of reasons.
- the source host may be overloaded from a compute capacity perspective and one or more of the source host's VMs may be migrated to another host that is less overloaded.
- a VM may be migrated to another host for network latency efficiencies when a destination host with sufficient capacity is located. Migrating VMs while they execute can raise issues related to minimizing the downtime of the VM during the migration.
- Latency remains one of the notable issues surrounding VM migration. For example, both the time to migrate a VM and the VM response time after migration can be at issue.
- Post-copy migration is a technique utilized to improved latency during VM migration. Post-copy migration can improve latency by starting the VM on the destination host while maintaining communication with the source host. Provided good connectivity between the source host and the destination host, post-copy migration provides a partial solution to the latency issue.
- the network (such as a local area network (LAN)) should consider the destination host as the new location for the VM.
- the network Prior to migration, the network had previously observed that packets from the VM were being sent from an address and port corresponding to the source host. However, the VM may now be at a new location in the network (e.g., destination host) and may send packets associated with a different port (of the destination host). Thus, packets from the VM may arrive from a different direction on the different port (e.g., incoming port) at components in the network.
- LAN local area network
- the destination host may broadcast a notification packet (e.g., reverse address resolution protocol (RARP) packet) using the address of the VM as the source address field in the packet.
- RARP reverse address resolution protocol
- the notification packet may cause the components of the network to update their forwarding tables with the address and the different port (of the destination host) to enable packets to be sent from and forwarded to the VM at its new location (e.g., on the destination host) in the network.
- RARP reverse address resolution protocol
- the notification packets are sent periodically for a set number of times at specified intervals. As such, even if the network is temporarily congested, eventually the notification packets should reach all of the network. However, the notification packets may get lost and cause network disruption for the VM. If a post-copy migration strategy is implemented, a significant strain is placed on the network in terms of communication bandwidth, which increases the chances of a notification packet getting lost or delayed. This is especially true in a network where multiple VMs may be migrated at the same time. Such a network address update disruption may be observed as bad performance, which may be hard to distinguish from other downtime sources and may be difficult to debug and recover from efficiently.
- a VM may be migrated from a source host to a destination host, resulting in the packets from the VM arriving at a different port on the components in the network.
- the destination host may broadcast a notification packet including a source address field with the address of the VM.
- an indication of successful migration to the destination host may be sent to the hypervisor on the source host.
- a monitoring component e.g., at the source host, destination host, or a virtualization manager
- the monitoring component may then send a notification to the destination hypervisor over a migration channel established between the source host and the destination host.
- the notification sent by the monitoring component over the migration channel can be an update notification to notify the destination hypervisor of the incoming packets address to the VM at the source host.
- the update notification can identify an origin of the incoming packet to the destination hypervisor.
- the destination hypervisor may include a notification component that causes another notification packet for the VM to be re-transmitted out on the network based on the received update notification from the monitoring component.
- either one or both of the source host and/or the virtualization manager may also include a notification component.
- the notification packet that is re-transmitted by the notification component may be referred to herein as a retransmitted notification packet.
- the retransmitted notification packet may be directly sent to the specified origin (e.g., endpoint) of the packet using a unicast address.
- the retransmitted notification packet may be broadcast to the entire network.
- the techniques disclosed herein may improve performance of the network by reducing communication downtime of a VM after the VM is migrated to a destination host. Downtime may be reduced by quickly detecting that an address of a migrated VM has not been updated at one or more components in a network of the VM, and causing action to be taken to update the pertinent components in the network with the ports associated with the VM at the new location (e.g., destination host). As a result, network components are more efficiently and quickly updated with a new address of a migrated VM, reducing communication disruptions that may result from lost or delayed notification packets.
- FIG. 1 illustrates an example system architecture 100 in which implementations of the disclosure may operate.
- the system architecture 100 may include a virtualization manager 110 , a plurality of host systems 120 A (hereinafter “source host 120 A”) and 120 B (hereinafter “destination host 120 B”), a client device 130 , and one or more endpoint devices 150 coupled via a network 140 .
- the network 140 may be a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof.
- LAN local area network
- WAN wide area network
- Network 140 may include a wireless infrastructure, which may be provided by one or more wireless communications systems, such as a wireless fidelity (WiFi) hotspot connected with the network 140 and/or a wireless carrier system that can be implemented using various data processing equipment, communication towers, etc. Additionally or alternatively, network 140 may include a wired infrastructure (e.g., Ethernet).
- WiFi wireless fidelity
- network 140 may include a wired infrastructure (e.g., Ethernet).
- the source host 120 A and the destination host 120 B may comprise one or more processors communicatively coupled to memory devices and input/output (I/O) devices.
- the source host 120 A and the destination host 120 B may run a plurality of VMs by executing a hypervisor 122 A and 122 B, respectively, to abstract the physical layer, including processors, memory, and I/O devices, and present this abstraction to the VMs as virtual devices.
- hypervisor 122 A may run VM 124 .
- the VM 124 may execute a guest operating system that may utilize the underlying virtual devices, including virtual processors, virtual memory, and virtual I/O devices.
- the hypervisors 122 A and 122 B may create, run, manage, and monitor various aspects of VMs operation, including the processing, and storage, memory, and network interfaces.
- hypervisors 122 A and 122 B may communicate with the virtualization manager 110 using a Virtual Desktop and Server Management (VDSM) daemon (not shown).
- VDSM daemon may include an application programming interface (API) with which the virtualization manager 110 interfaces.
- the virtualization manager 110 may be hosted by a computer system and include one or more computer programs executed by the computer system for centralized management of the system architecture 100 .
- the virtualization manager 110 may comprise various interfaces, including administrative interface, reporting interface, and/or application programming interface (API) to communicate with the client device 130 , the source host 120 A, and the destination host 120 B of system architecture 100 , as well as to user portals, directory servers, and various other components, which are omitted from FIG. 1 for clarity.
- API application programming interface
- Virtualization manager 110 may provide VM migration management to migrate existing VMs from one host (e.g., source host 120 A) to a second host (e.g., destination host 120 B).
- an administrator may use the client device 130 (e.g., laptop, desktop, tablet, smartphone, server) to request migration of the VM 124 to the destination host 120 B.
- the migration may include copying various data structures (e.g., disks, volumes, etc.) associated with the VM 124 from the source host 120 A to the destination host 120 B, starting the VM 124 on the destination host 120 B, and/or stopping the VM 124 on the source host 120 A.
- the VDSM or any suitable application executing on the source host 120 A and the destination host 120 B may provide status notifications to the virtualization manager 110 that indicate the operating state of the hypervisors 122 A and 122 B and/or the VMs 124 .
- the hypervisors 122 A and 122 B may be connected to each other via the network 140 and may send notifications back and forth without sending the notifications to the virtualization manager 110 .
- the hypervisors 122 A, 122 B are connected via a migration communication channel 121 established during a migration process of the VM 124 .
- the status notification may be transmitted by the VDSM or other application when the VM 124 is successfully migrated to the destination host 120 B and starts to execute on the destination host 120 B via the hypervisor 122 B.
- the VDSM or any suitable application may provide a notification that indicates that the hypervisor 122 B or the VM 124 sent out a notification packet that includes the VM address in a source address field over the network.
- Sending the notification packet may cause components (e.g., switches) to identify the address of the VM and update their forwarding tables with the different incoming ports at which the packets from the VM arrived.
- the notification packet may not result in the network fully updating (e.g., when a notification packet is lost) and the techniques disclosed herein may be used to detect and resolve such broadcast issues.
- the source host 120 A may include a monitoring component 125 and may be referred to as a monitoring node.
- the monitoring component 125 may be implemented as computer instructions stored on one or more memories and executed by one or more processing devices of the source host 120 A.
- the monitoring component 125 on the source host 120 A may be part of the hypervisor 122 A.
- the monitoring component 125 on the source host 120 A may interface with the hypervisor 122 A.
- the monitoring nodes may be various networking components (e.g., host systems in a cluster, switches, relays, etc.) in the network 140 .
- the destination host 120 B may also include a monitoring component 125 (not shown).
- the destination host 120 B may include a notification component 127 and may be referred to as a notification node.
- the notification component 127 may be implemented as computer instructions stored on one or more memories and executed by one or more processing devices of the designation host 120 B.
- the notification component 127 on the destination host 120 B may be part of the hypervisor 122 B.
- the notification component 127 on the destination host 120 B may interface with the hypervisor 122 B.
- the source host 120 A may also include a notification component 127 (not shown).
- the virtualization manager 110 may also include a monitoring component 125 and a notification component 127 , which may be the same as the monitoring component 125 and notification components 127 of the source host 120 A and the destination hot 120 B.
- the monitoring component 125 and/or the notification component 127 of the virtualization manager 110 may be implemented as computer instructions stored on one or more tangible, non-transitory computer-readable media executed by one or more processing devices of the computer system hosting the virtualization manager 110 .
- the monitoring component 125 and the notification component 127 are used to perform one or more actions for network updates for VM migration.
- the monitoring component 125 may receive an indication from the virtualization manager 110 or the hypervisor 122 B of the destination host 120 B. This received indication may indicate that the VM 124 migrated successfully and started on the destination host 120 B. The indication may also include the address of the VM 124 that is executing on the destination host 120 B.
- the hypervisor 122 A may exit or shut down after receiving the indication that the VM migrated successfully (if there are no other VMs running on the source host 120 A).
- the source hypervisor 122 A may drop packets (“incoming packets”) received at the source host 120 A that are addressed to the VM 124 that has migrated.
- the hypervisor 122 A remains active after migration is complete in order to monitor for incoming packets at the source host 120 A with a destination address field set to the address of the VM.
- the techniques disclosed herein may enable improving performance of the system 100 by detecting when a notification packet has not been delivered for the VM 124 that is migrated (and thus, packets destined for the VM were dropped at the source host 120 A) and resolving the notification issue via efficient re-transmission of notification packets to endpoints 150 in the network 140 .
- the monitoring component 125 can keep track of the incoming packets arriving at the source host 120 A. In response to receiving the indication of successful VM migration, the monitoring component 125 may begin to monitor the network 140 for incoming packets that have a destination address field set to the address of the VM. For example, the monitoring component 125 may monitor a network interface card (NIC) of the source host 120 A for the received incoming packets. In implementations of the disclosure, the incoming packets may be sent from one or more of the endpoint devices 150 of the network 140 . The endpoint devices 150 may be other hosts in the network 140 or other nodes capable of communicating across network 140 .
- NIC network interface card
- the indication may include an instruction to the monitoring component 125 to monitor for a specific type of incoming packets (e.g., a reverse address resolution protocol (RARP) packet type).
- a specific type of incoming packets e.g., a reverse address resolution protocol (RARP) packet type.
- RARP reverse address resolution protocol
- This particular packet type may be used when a change is made to a topology of the network 140 , and thus, may enhance robustness of detection at the monitoring components 125 by looking for packets that include the address of the VM and have the packet type indicative of a change in the network 140 . In other words, also inspecting for the packet type may help the monitoring components 125 not detect an older message sent by the VM before the network change that resulted from migration.
- RARP reverse address resolution protocol
- the monitoring component 125 may send a notification to the destination hypervisor 122 B over the migration channel 121 established between the source host 120 A and the destination host 120 B.
- the migration channel 121 may be a communication channel established between the source hypervisor 122 A and the destination hypervisor 122 B to communicate with respect to migration.
- the migration channel 121 is used to communicate regarding state of memory pages of the VM 124 during the migration process.
- the migration channel 121 may be further utilized in implementations of the disclosure to communicate notifications regarding packets for the VM 124 that are received at the source host 120 A after migration of the VM to the destination host 120 B.
- the notification sent by the monitoring component 125 over the migration channel 121 may be an update notification to notify the destination hypervisor 122 B of the incoming packets at the source host 120 A.
- the update notification can identify a origin (e.g., endpoint 150 ) of the incoming packet to the destination hypervisor 122 B.
- the identification of the origin may be a network address or other identifier of one or more of the endpoint devices 150 of network 140 .
- the update notification can be a binary flag that indicates that a packet directed to the VM 124 was dropped by the source hypervisor 122 A. In this example, the identification of the origin of the dropped packet is not provided by the monitoring component 125 , which can help reduce network bandwidth used by the update notifications sent by the monitoring component 125 .
- the destination hypervisor 122 B receives the update notification from the monitoring component 125 .
- the destination hypervisor 122 B may include a notification component 127 .
- the notification component can cause another notification packet for the VM 124 to be re-transmitted based on the update notification received from the monitoring component 125 .
- the notification component 127 may also be part of the virtualization manager 110 and/or the source host 120 A.
- the notification packet that is re-transmitted by the notification component 127 may be referred to herein as a retransmitted notification packet.
- the retransmitted notification packet can be sent to the specified origin (e.g., endpoint 150 ) of the packet.
- the retransmitted notification packet may be broadcast to the entire network 140 .
- the notification component 127 may detect that a number of addresses corresponding to the dropped packets at the source host 120 A exceeds a specified threshold value (also referred to herein as a dropped packet threshold value). When the dropped packet threshold value is exceeded by the number of dropped packets and/or by the number of endpoints 150 identified by the monitoring component 125 , the notification component 127 may then notify the originating endpoints 150 (e.g., as a multicast or broadcast communication) via the retransmitted notification packet. This notification can be in lieu of sending individual notifications (via unicast address transmissions) to each originating endpoint 150 . In some implementations, the notification component 127 may utilize broadcast (or multicast) communication for multiple addresses, and unicast for a single address.
- a specified threshold value also referred to herein as a dropped packet threshold value.
- the notification component 127 may send retransmitted notification packets as a broadcast to the entire network 140 for a first interval of time and then may switch to sending retransmitted notification packets individually to unicast addresses of the identified endpoints 150 associated with dropped packets after the first interval of time. In some implementations, the notification component 127 may utilize a broadcast communication to send the retransmitted notification packet if the notification component 127 is not able to keep up with a rate of incoming update notifications being received from the monitoring component 125 .
- the notification component 127 may send retransmitted notification packets using a multicast address to particular segments or partitions of the network 140 . For example, if the notification component 127 identifies that a particular segment or partition of the network 140 is associated with the dropped packets at the source host 120 A (e.g., due to a faulty network cable or connection), then the notification component 127 may send the retransmitted notification packet to those endpoints 150 associated with the segment/partition using a multicast address corresponding to that segment/partition.
- FIG. 2 depicts a flow diagram of an example method 200 for network updates for VM migration, in accordance with one or more aspects of the disclosure.
- Method 200 and each of its individual functions, routines, subroutines, or operations may be performed by one or more processing devices of the computer device executing the method.
- method 200 may be performed by a single processing thread.
- method 200 may be performed by two or more processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method.
- the processing threads implementing method 200 may be synchronized (e.g., using semaphores, critical sections, and/or other thread synchronization mechanisms).
- the processes implementing method 200 may be executed asynchronously with respect to each other.
- method 200 may be performed by monitoring component 125 executed by one or more processing devices of the source host 120 A and/or the virtualization manager 110 .
- Method 200 may begin at block 202 .
- a processing device may receive an indication over a network that a VM successfully migrated from a source host to a destination host.
- the indication may indicate that the VM migrated successfully and started on the destination host.
- the indication may also include the address of the VM that is executing on the destination host.
- the indication may include an instruction to a monitoring component to monitor for a specific type of incoming packets (e.g., a reverse address resolution protocol (RARP) packet type).
- RARP reverse address resolution protocol
- This particular packet type may be used when a change is made to a topology of the network, and thus, may enhance robustness of detection at the monitoring components by looking for packets that include the address of the VM and have the packet type indicative of a change in the network.
- RARP reverse address resolution protocol
- the processing device may, responsive to the indication that the VM successfully migrated from the source host to the destination host, monitor incoming packets to the source host for an incoming packet having a VM address of the VM in a destination address field.
- a monitoring component may begin to monitor the network for incoming packets that have a destination address field set to the address of the VM. For example, the monitoring component may monitor a NIC of the source host for the received incoming packets.
- the processing device may, upon determining that one or more of the incoming packets to the source host include the destination address field having the VM address, provide, to the destination host, a notification that the one or more of the incoming packets having the VM address were received at the source host.
- the notification indicates that an update of the virtual machine address should be performed by one or more endpoints of the network (e.g., by the endpoint(s) from which the above incoming packets have originated or by all endpoints of the network).
- the destination hypervisor may include a notification component that causes the notification packet for the VM to be re-transmitted based on the received update notification from the monitoring component.
- the notification packet that is re-transmitted by the notification component may be referred to herein as a retransmitted notification packet.
- the retransmitted notification packet can be sent to the specified origin (e.g., endpoint) of the packet.
- the retransmitted notification packet may be broadcast to the entire network.
- FIG. 3 depicts a block diagram of an example computer system 300 , in accordance with one or more aspects of the disclosure.
- Computer system 300 may be the same or similar to the source host 120 A, destination host 120 B, or the virtualization manager 110 and may include one or more processing devices 305 and one or more memory devices 350 .
- computer system 300 may include indication receiving module 310 , monitoring module 320 , and notification sending module 330 .
- computer system 300 may be communicatively coupled to the source host 302 , and destination host 304 via the network 140 . Although shown as separate from and connected to the source host 302 (e.g., source host 120 A of FIG.
- the computer system 300 may be part of the source host 302 (e.g., source host 120 A). In addition, although shown as separate from and connected to the destination host 304 (e.g., destination host 120 B of FIG. 1 ), the computer system 300 may be part of the destination host 304 (e.g., destination host 120 B). Furthermore, the computer system 300 may be part of the virtualization manager (e.g., virtualization manager 110 of FIG. 1 ).
- the virtualization manager e.g., virtualization manager 110 of FIG. 1 ).
- the indication receiving module 310 may receive an indication over the network 140 that a VM 124 successfully migrated from the source host 302 to the destination host 304 .
- Successfully migrating may refer to the VM 124 successfully starting on the destination host.
- the indication may be received from the virtualization manager 110 or the destination host 304 .
- the indication may include a VM address of the VM 124 executing on the destination host.
- the hypervisor 122 A may remain active on the source host 120 A after receiving the indication that the migration was successful to enable monitoring for incoming packets to the source host 302 .
- the monitoring module 320 may, responsive to the indication that the VM 124 successfully migrated from the source host 302 to the destination host 304 , start to monitor incoming packets 306 of source host 302 for an incoming packet 306 that includes a destination address field 308 having the VM address at the source host 302 .
- the notification sending module 330 may, upon determining that one or more of the incoming packets 306 at the source host 302 include the destination address field 308 having the VM address, provide a notification to the destination host 304 that one or more incoming packets 306 were received at the source host 302 .
- This notification to the destination host 304 is to facilitate retransmission of notification packets on the network to cause the VM address to be update to transmit to the destination host 304 .
- the retransmission of a notification packet by the destination host 304 may facilitate updating of forwarding tables in switches of the network 140 with at least the new ports at which packets having the VM address arrive (e.g., the forwarding tables updated entries for the VM address with different incoming ports associated with packets received from the VM 124 on the destination host 304 ).
- FIG. 4 depicts a flow diagram of an example method 400 for network updates for VM migration at a notification component, in accordance with one or more aspects of the disclosure.
- Method 400 includes operations performed by the destination host 120 A and/or the virtualization manager. In some implementations, method 400 may be performed by the source host 120 A, as well. Method 400 may be performed in the same or a similar manner as described above in regards to method 200 . Method 400 may be performed by processing devices of the destination host 120 B executing the hypervisor 122 B and the notification component 127 .
- Method 400 may begin at block 402 .
- the processing device may receive an update notification from a monitoring component.
- the update notification can indicate that a packet directed to a migrated VM was received at a source host and dropped by the source host hypervisor.
- the processing device may determine whether a number of received update notifications at the destination host meets or exceeds a threshold value.
- the processing device may broadcast a retransmitted notification packet on the network responsive to the threshold value being met or exceeded.
- the threshold value may be determined by an administrator of the managed virtualization environment. Utilization of the threshold may provide a baseline where it is more economical (in terms of network bandwidth and/or load) to send a single broadcast message for the notification packet over the network to all components, instead of sending such notification packet to individual components on a one-on-one basis. The utilization of the threshold can help reduce overall network communications, while balancing against the cost of sending an unnecessary notification packet to components that may have already updated the address of the VM.
- the processing device may transmit a retransmitted notification packet to individual endpoints on the network responsive to the threshold value not being exceeded.
- the update notification from the monitoring component may identify the endpoints associated with the received packets at the source host. The identified endpoints can then be individually contacted with the retransmitted notification packet using a unicast address of the endpoint.
- FIG. 5 depicts a flow diagram of another example method 500 for network updates for VM migration at a notification component, in accordance with one or more aspects of the disclosure.
- Method 500 includes operations performed by the destination host 120 A and/or the virtualization manager. In some implementations, method 500 may be performed by the source host 120 A, as well. Method 500 may be performed in the same or a similar manner as described above in regards to methods 200 and 400 . Method 500 may be performed by processing devices of the destination host 120 B executing the hypervisor 122 B and the notification component 127 .
- Method 500 may begin at block 502 .
- the processing device may receive an update notification from a monitoring component.
- the update notification can indicate that a packet directed to a migrated VM was received at a source host and dropped by the source host hypervisor.
- the processing device may determine whether a number of received update notifications at the destination host exceeds a threshold rate.
- the threshold rate may include a ratio of number of update notifications received over a determined period of time.
- the processing device may broadcast a retransmitted notification packet on the network responsive to the threshold rate being met or exceeded.
- the threshold rate may be determined by an administrator of the managed virtualization environment.
- the processing device may transmit a retransmitted notification packet to individual endpoints on the network responsive to the threshold rate not being exceeded.
- the update notification from the monitoring component may identify the endpoints associated with the received packets at the source host. The identified endpoints can then be individually contacted with the retransmitted notification packet using a unicast address of the endpoint.
- FIG. 6 depicts a block diagram of a computer system operating in accordance with one or more aspects of the disclosure.
- computer system 600 may correspond to a computing device within system architecture 100 of FIG. 1 .
- the computer system 600 may be the source host 120 A or the destination host 120 B.
- the computer system 600 may be hosting the virtualization manager 110 .
- the computer system 600 may be included within a data center that supports virtualization. Virtualization within a data center results in a physical system being virtualized using VMs to consolidate the data center infrastructure and increase operational efficiencies.
- a VM may be a program-based emulation of computer hardware.
- the VM may operate based on computer architecture and functions of computer hardware resources associated with hard disks or other such memory.
- the VM may emulate a physical computing environment, but requests for a hard disk or memory may be managed by a virtualization layer of a host system to translate these requests to the underlying physical computing hardware resources. This type of virtualization results in multiple VMs sharing physical resources.
- computer system 600 may be connected (e.g., via a network, such as a Local Area Network (LAN), an intranet, an extranet, or the Internet) to other computer systems.
- Computer system 600 may operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment.
- Computer system 600 may be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, switch or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device.
- PC personal computer
- PDA Personal Digital Assistant
- STB set-top box
- web appliance a web appliance
- server a server
- network router switch or bridge
- any device capable of executing a set of instructions that specify actions to be taken by that device.
- the computer system 600 may include a processing device 602 , a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically-erasable programmable ROM (EEPROM)), and a data storage device 616 , which may communicate with each other via a bus 608 .
- a volatile memory 604 e.g., random access memory (RAM)
- non-volatile memory 606 e.g., read-only memory (ROM) or electrically-erasable programmable ROM (EEPROM)
- EEPROM electrically-erasable programmable ROM
- Processing device 602 may be provided by one or more processors such as a general purpose processor (such as, for example, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of types of instruction sets) or a specialized processor (such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).
- CISC complex instruction set computing
- RISC reduced instruction set computing
- VLIW very long instruction word
- ASIC application specific integrated circuit
- FPGA field programmable gate array
- DSP digital signal processor
- Computer system 600 may further include a network interface device 622 .
- Computer system 600 also may include a video display unit 610 (e.g., an LCD), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generation device 620 .
- a video display unit 610 e.g., an LCD
- an alphanumeric input device 612 e.g., a keyboard
- a cursor control device 614 e.g., a mouse
- signal generation device 620 e.g., a signal generation device.
- Data storage device 616 may include a non-transitory computer-readable storage medium 624 on which may store instructions 626 encoding any one or more of the methods or functions described herein, including instructions implementing monitoring component 125 and/or notification component 127 of FIG. 1 for implementing method 200 , method 400 , and method 500 .
- Instructions 626 may also reside, completely or partially, within volatile memory 604 and/or within processing device 602 during execution thereof by computer system 600 , hence, volatile memory 604 and processing device 602 may also constitute machine-readable storage media.
- While computer-readable storage medium 624 is shown in the illustrative examples as a single medium, the term “computer-readable storage medium” shall include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of executable instructions.
- the term “computer-readable storage medium” shall also include any tangible medium that is capable of storing or encoding a set of instructions for execution by a computer that cause the computer to perform any one or more of the methods described herein.
- the term “computer-readable storage medium” shall include, but not be limited to, solid-state memories, optical media, and magnetic media.
- the methods, components, and features described herein may be implemented by discrete hardware components or may be integrated in the functionality of other hardware components such as ASICS, FPGAs, DSPs or similar devices.
- the methods, components, and features may be implemented by firmware modules or functional circuitry within hardware devices.
- the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in computer programs.
- terms such as “receiving,” “associating,” “deleting,” “initiating,” “marking,” “generating,” “recovering,” “completing,” or the like refer to actions and processes performed or implemented by computer systems that manipulates and transforms data represented as physical (electronic) quantities within the computer system registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
- the terms “first,” “second,” “third,” “fourth,” etc. as used herein are meant as labels to distinguish among different elements and may not have an ordinal meaning according to their numerical designation.
- Examples described herein also relate to an apparatus for performing the methods described herein.
- This apparatus may be specially constructed for performing the methods described herein, or it may comprise a general purpose computer system selectively programmed by a computer program stored in the computer system.
- a computer program may be stored in a computer-readable tangible storage medium.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
- The disclosure is generally related to virtualization systems, and is more specifically related to network updates for virtual machine migration.
- Data centers may include clusters of multiple hosts (e.g., physical servers) in racks. Hypervisors may operate on each host to create and run virtual machines (VMs). VMs emulate computer systems and may be referred to as guest machines. The hosts in the clusters may be connected via one or more wired (e.g., Ethernet) and/or wireless (e.g., WiFi) networks (e.g., the Internet, local area network (LAN)). In some instances, a VM on a source host machine may be migrated to a destination host machine within the cluster. To communicate with the migrated VM on the destination host, various components of the networks may be updated with address updates for the VM.
- The disclosure is illustrated by way of examples, and not by way of limitation, and may be more fully understood with references to the following detailed description when considered in connection with the figures, in which:
-
FIG. 1 depicts a high-level diagram of an example system architecture operating in accordance with one or more aspects of the disclosure; -
FIG. 2 depicts a flow diagram of an example method for network updates for virtual machine migration, in accordance with one or more aspects of the disclosure; -
FIG. 3 depicts a block diagram of an example computer system, in accordance with one or more aspects of the disclosure; -
FIG. 4 depicts a flow diagram of another example method for network updates for virtual machine migration by a notification component, in accordance with one or more aspects of the disclosure; -
FIG. 5 depicts a flow diagram of another example method for network updates virtual machine migration by a notification component, in accordance with one or more aspects of the disclosure; -
FIG. 6 depicts a block diagram of an illustrative computing device operating in accordance with the examples of the disclosure. - Implementations of the disclosure are directed to network updates for virtual machine migration. A virtual machine (VM) may execute via a source hypervisor on a source host in a managed virtualization environment. The VM may be associated with a first address (e.g., a media access control (MAC) address tagged with a virtual local area network (VLAN) address) that corresponds to the source host. The source host may be connected to other hosts via a network. As the VM or the hypervisor sends packets from the source host across the network to other hosts, components (e.g., switches) of the network may inspect the packets and determine that the first address of the VM is coming from a port associated with the source host. This information may be added to a forwarding table maintained by each of the components of the network. As such, when the components receive packets from other hosts that include a destination address set to the address of the VM, the components may forward the packets using the port to the source host.
- In some instances, a topology of the network may change, for example, when a VM executing on the source host is migrated to a destination host. A VM may be migrated from one host to another host for a variety of reasons. For example, the source host may be overloaded from a compute capacity perspective and one or more of the source host's VMs may be migrated to another host that is less overloaded. In another example, a VM may be migrated to another host for network latency efficiencies when a destination host with sufficient capacity is located. Migrating VMs while they execute can raise issues related to minimizing the downtime of the VM during the migration.
- Latency remains one of the notable issues surrounding VM migration. For example, both the time to migrate a VM and the VM response time after migration can be at issue. Post-copy migration is a technique utilized to improved latency during VM migration. Post-copy migration can improve latency by starting the VM on the destination host while maintaining communication with the source host. Provided good connectivity between the source host and the destination host, post-copy migration provides a partial solution to the latency issue.
- Another issue with respect to latency resulting from VM migration is network updates. As part of the migration process of the VM, the network (such as a local area network (LAN)) should consider the destination host as the new location for the VM. Prior to migration, the network had previously observed that packets from the VM were being sent from an address and port corresponding to the source host. However, the VM may now be at a new location in the network (e.g., destination host) and may send packets associated with a different port (of the destination host). Thus, packets from the VM may arrive from a different direction on the different port (e.g., incoming port) at components in the network. Accordingly, in some instances, the destination host may broadcast a notification packet (e.g., reverse address resolution protocol (RARP) packet) using the address of the VM as the source address field in the packet. The notification packet may cause the components of the network to update their forwarding tables with the address and the different port (of the destination host) to enable packets to be sent from and forwarded to the VM at its new location (e.g., on the destination host) in the network.
- In conventional managed virtualization environments, the notification packets are sent periodically for a set number of times at specified intervals. As such, even if the network is temporarily congested, eventually the notification packets should reach all of the network. However, the notification packets may get lost and cause network disruption for the VM. If a post-copy migration strategy is implemented, a significant strain is placed on the network in terms of communication bandwidth, which increases the chances of a notification packet getting lost or delayed. This is especially true in a network where multiple VMs may be migrated at the same time. Such a network address update disruption may be observed as bad performance, which may be hard to distinguish from other downtime sources and may be difficult to debug and recover from efficiently.
- Aspects of the disclosure address the above and other deficiencies by providing technology directed to network updates for VM migration. In an implementation, a VM may be migrated from a source host to a destination host, resulting in the packets from the VM arriving at a different port on the components in the network. When the VM starts successfully on the destination host, the destination host may broadcast a notification packet including a source address field with the address of the VM. Also, when the VM starts successfully on the destination host, an indication of successful migration to the destination host may be sent to the hypervisor on the source host.
- In implementations of the disclosure, a monitoring component (e.g., at the source host, destination host, or a virtualization manager) can keep track of the incoming packets arriving at the source host. The monitoring component may then send a notification to the destination hypervisor over a migration channel established between the source host and the destination host. The notification sent by the monitoring component over the migration channel can be an update notification to notify the destination hypervisor of the incoming packets address to the VM at the source host. In some implementations, the update notification can identify an origin of the incoming packet to the destination hypervisor.
- The destination hypervisor may include a notification component that causes another notification packet for the VM to be re-transmitted out on the network based on the received update notification from the monitoring component. In some implementations, either one or both of the source host and/or the virtualization manager may also include a notification component. The notification packet that is re-transmitted by the notification component may be referred to herein as a retransmitted notification packet. The retransmitted notification packet may be directly sent to the specified origin (e.g., endpoint) of the packet using a unicast address. In some implementations, the retransmitted notification packet may be broadcast to the entire network.
- As such, the techniques disclosed herein may improve performance of the network by reducing communication downtime of a VM after the VM is migrated to a destination host. Downtime may be reduced by quickly detecting that an address of a migrated VM has not been updated at one or more components in a network of the VM, and causing action to be taken to update the pertinent components in the network with the ports associated with the VM at the new location (e.g., destination host). As a result, network components are more efficiently and quickly updated with a new address of a migrated VM, reducing communication disruptions that may result from lost or delayed notification packets.
-
FIG. 1 illustrates anexample system architecture 100 in which implementations of the disclosure may operate. Thesystem architecture 100 may include avirtualization manager 110, a plurality ofhost systems 120A (hereinafter “source host 120A”) and 120B (hereinafter “destination host 120B”), aclient device 130, and one ormore endpoint devices 150 coupled via anetwork 140. Thenetwork 140 may be a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof.Network 140 may include a wireless infrastructure, which may be provided by one or more wireless communications systems, such as a wireless fidelity (WiFi) hotspot connected with thenetwork 140 and/or a wireless carrier system that can be implemented using various data processing equipment, communication towers, etc. Additionally or alternatively,network 140 may include a wired infrastructure (e.g., Ethernet). - The
source host 120A and thedestination host 120B may comprise one or more processors communicatively coupled to memory devices and input/output (I/O) devices. Thesource host 120A and thedestination host 120B may run a plurality of VMs by executing ahypervisor hypervisor 122A may runVM 124. TheVM 124 may execute a guest operating system that may utilize the underlying virtual devices, including virtual processors, virtual memory, and virtual I/O devices. - One or more applications may be running on a VM under the guest operating system. The
hypervisors hypervisors virtualization manager 110 using a Virtual Desktop and Server Management (VDSM) daemon (not shown). The VDSM daemon may include an application programming interface (API) with which thevirtualization manager 110 interfaces. - The
virtualization manager 110 may be hosted by a computer system and include one or more computer programs executed by the computer system for centralized management of thesystem architecture 100. In one implementation, thevirtualization manager 110 may comprise various interfaces, including administrative interface, reporting interface, and/or application programming interface (API) to communicate with theclient device 130, thesource host 120A, and thedestination host 120B ofsystem architecture 100, as well as to user portals, directory servers, and various other components, which are omitted fromFIG. 1 for clarity. -
Virtualization manager 110 may provide VM migration management to migrate existing VMs from one host (e.g.,source host 120A) to a second host (e.g.,destination host 120B). In one example, an administrator may use the client device 130 (e.g., laptop, desktop, tablet, smartphone, server) to request migration of theVM 124 to thedestination host 120B. The migration may include copying various data structures (e.g., disks, volumes, etc.) associated with theVM 124 from thesource host 120A to thedestination host 120B, starting theVM 124 on thedestination host 120B, and/or stopping theVM 124 on thesource host 120A. - The VDSM or any suitable application executing on the
source host 120A and thedestination host 120B may provide status notifications to thevirtualization manager 110 that indicate the operating state of thehypervisors VMs 124. In an example, thehypervisors network 140 and may send notifications back and forth without sending the notifications to thevirtualization manager 110. In one example, thehypervisors migration communication channel 121 established during a migration process of theVM 124. The status notification may be transmitted by the VDSM or other application when theVM 124 is successfully migrated to thedestination host 120B and starts to execute on thedestination host 120B via thehypervisor 122B. - Additionally, the VDSM or any suitable application may provide a notification that indicates that the
hypervisor 122B or theVM 124 sent out a notification packet that includes the VM address in a source address field over the network. Sending the notification packet may cause components (e.g., switches) to identify the address of the VM and update their forwarding tables with the different incoming ports at which the packets from the VM arrived. However, as noted above, the notification packet may not result in the network fully updating (e.g., when a notification packet is lost) and the techniques disclosed herein may be used to detect and resolve such broadcast issues. - In an example, the
source host 120A may include amonitoring component 125 and may be referred to as a monitoring node. Themonitoring component 125 may be implemented as computer instructions stored on one or more memories and executed by one or more processing devices of thesource host 120A. In an example, themonitoring component 125 on thesource host 120A may be part of thehypervisor 122A. In another example, themonitoring component 125 on thesource host 120A may interface with thehypervisor 122A. There may also be one or more additional monitoring nodes (not shown) that include themonitoring component 125 installed throughout thenetwork 140. The monitoring nodes may be various networking components (e.g., host systems in a cluster, switches, relays, etc.) in thenetwork 140. In some implementations, thedestination host 120B may also include a monitoring component 125 (not shown). - In an example, the
destination host 120B may include anotification component 127 and may be referred to as a notification node. Thenotification component 127 may be implemented as computer instructions stored on one or more memories and executed by one or more processing devices of thedesignation host 120B. In an example, thenotification component 127 on thedestination host 120B may be part of thehypervisor 122B. In another example, thenotification component 127 on thedestination host 120B may interface with thehypervisor 122B. In some implementations, thesource host 120A may also include a notification component 127 (not shown). - In some implementations, the
virtualization manager 110 may also include amonitoring component 125 and anotification component 127, which may be the same as themonitoring component 125 andnotification components 127 of thesource host 120A and the destination hot 120B. Themonitoring component 125 and/or thenotification component 127 of thevirtualization manager 110 may be implemented as computer instructions stored on one or more tangible, non-transitory computer-readable media executed by one or more processing devices of the computer system hosting thevirtualization manager 110. - In implementations of the disclosure, the
monitoring component 125 and thenotification component 127 are used to perform one or more actions for network updates for VM migration. Themonitoring component 125 may receive an indication from thevirtualization manager 110 or the hypervisor 122B of thedestination host 120B. This received indication may indicate that theVM 124 migrated successfully and started on thedestination host 120B. The indication may also include the address of theVM 124 that is executing on thedestination host 120B. - Conventionally, the
hypervisor 122A may exit or shut down after receiving the indication that the VM migrated successfully (if there are no other VMs running on thesource host 120A). As such, in conventional migration processes, thesource hypervisor 122A may drop packets (“incoming packets”) received at thesource host 120A that are addressed to theVM 124 that has migrated. However, in implementations of the disclosure, thehypervisor 122A remains active after migration is complete in order to monitor for incoming packets at thesource host 120A with a destination address field set to the address of the VM. The techniques disclosed herein may enable improving performance of thesystem 100 by detecting when a notification packet has not been delivered for theVM 124 that is migrated (and thus, packets destined for the VM were dropped at thesource host 120A) and resolving the notification issue via efficient re-transmission of notification packets toendpoints 150 in thenetwork 140. - In implementations of the disclosure, the
monitoring component 125 can keep track of the incoming packets arriving at thesource host 120A. In response to receiving the indication of successful VM migration, themonitoring component 125 may begin to monitor thenetwork 140 for incoming packets that have a destination address field set to the address of the VM. For example, themonitoring component 125 may monitor a network interface card (NIC) of thesource host 120A for the received incoming packets. In implementations of the disclosure, the incoming packets may be sent from one or more of theendpoint devices 150 of thenetwork 140. Theendpoint devices 150 may be other hosts in thenetwork 140 or other nodes capable of communicating acrossnetwork 140. - In an example, the indication may include an instruction to the
monitoring component 125 to monitor for a specific type of incoming packets (e.g., a reverse address resolution protocol (RARP) packet type). This particular packet type may be used when a change is made to a topology of thenetwork 140, and thus, may enhance robustness of detection at themonitoring components 125 by looking for packets that include the address of the VM and have the packet type indicative of a change in thenetwork 140. In other words, also inspecting for the packet type may help themonitoring components 125 not detect an older message sent by the VM before the network change that resulted from migration. - In response to detection of incoming packets at the
source host 120A having a destination address field having an address of the VM, themonitoring component 125 may send a notification to thedestination hypervisor 122B over themigration channel 121 established between thesource host 120A and thedestination host 120B. As discussed above, themigration channel 121 may be a communication channel established between thesource hypervisor 122A and thedestination hypervisor 122B to communicate with respect to migration. For example, in a post-copy migration approach, themigration channel 121 is used to communicate regarding state of memory pages of theVM 124 during the migration process. Themigration channel 121 may be further utilized in implementations of the disclosure to communicate notifications regarding packets for theVM 124 that are received at thesource host 120A after migration of the VM to thedestination host 120B. - The notification sent by the
monitoring component 125 over themigration channel 121 may be an update notification to notify thedestination hypervisor 122B of the incoming packets at thesource host 120A. In some implementations, the update notification can identify a origin (e.g., endpoint 150) of the incoming packet to thedestination hypervisor 122B. In one implementation, the identification of the origin may be a network address or other identifier of one or more of theendpoint devices 150 ofnetwork 140. In some implementations, the update notification can be a binary flag that indicates that a packet directed to theVM 124 was dropped by thesource hypervisor 122A. In this example, the identification of the origin of the dropped packet is not provided by themonitoring component 125, which can help reduce network bandwidth used by the update notifications sent by themonitoring component 125. - In one implementation, the
destination hypervisor 122B receives the update notification from themonitoring component 125. As discussed above, thedestination hypervisor 122B may include anotification component 127. The notification component can cause another notification packet for theVM 124 to be re-transmitted based on the update notification received from themonitoring component 125. As noted above, thenotification component 127 may also be part of thevirtualization manager 110 and/or thesource host 120A. - The notification packet that is re-transmitted by the
notification component 127 may be referred to herein as a retransmitted notification packet. In some implementations, the retransmitted notification packet can be sent to the specified origin (e.g., endpoint 150) of the packet. In some implementations, the retransmitted notification packet may be broadcast to theentire network 140. - To reduce the amount of communication overhead consumed on the
network 140, thenotification component 127 may detect that a number of addresses corresponding to the dropped packets at thesource host 120A exceeds a specified threshold value (also referred to herein as a dropped packet threshold value). When the dropped packet threshold value is exceeded by the number of dropped packets and/or by the number ofendpoints 150 identified by themonitoring component 125, thenotification component 127 may then notify the originating endpoints 150 (e.g., as a multicast or broadcast communication) via the retransmitted notification packet. This notification can be in lieu of sending individual notifications (via unicast address transmissions) to each originatingendpoint 150. In some implementations, thenotification component 127 may utilize broadcast (or multicast) communication for multiple addresses, and unicast for a single address. - In some implementations, the
notification component 127 may send retransmitted notification packets as a broadcast to theentire network 140 for a first interval of time and then may switch to sending retransmitted notification packets individually to unicast addresses of the identifiedendpoints 150 associated with dropped packets after the first interval of time. In some implementations, thenotification component 127 may utilize a broadcast communication to send the retransmitted notification packet if thenotification component 127 is not able to keep up with a rate of incoming update notifications being received from themonitoring component 125. - In some implementations, the
notification component 127 may send retransmitted notification packets using a multicast address to particular segments or partitions of thenetwork 140. For example, if thenotification component 127 identifies that a particular segment or partition of thenetwork 140 is associated with the dropped packets at thesource host 120A (e.g., due to a faulty network cable or connection), then thenotification component 127 may send the retransmitted notification packet to thoseendpoints 150 associated with the segment/partition using a multicast address corresponding to that segment/partition. -
FIG. 2 depicts a flow diagram of anexample method 200 for network updates for VM migration, in accordance with one or more aspects of the disclosure.Method 200 and each of its individual functions, routines, subroutines, or operations may be performed by one or more processing devices of the computer device executing the method. In certain implementations,method 200 may be performed by a single processing thread. Alternatively,method 200 may be performed by two or more processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In an illustrative example, the processingthreads implementing method 200 may be synchronized (e.g., using semaphores, critical sections, and/or other thread synchronization mechanisms). Alternatively, theprocesses implementing method 200 may be executed asynchronously with respect to each other. - For simplicity of explanation, the methods of this disclosure are depicted and described as a series of acts. However, acts in accordance with this disclosure can occur in various orders and/or concurrently, and with other acts not presented and described herein. Furthermore, not all illustrated acts may be required to implement the methods in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the methods could alternatively be represented as a series of interrelated states via a state diagram or events. Additionally, it should be appreciated that the methods disclosed in this specification are capable of being stored on an article of manufacture to facilitate transporting and transferring such methods to computing devices. The term “article of manufacture,” as used herein, is intended to encompass a computer program accessible from any computer-readable device or storage media. In one implementation,
method 200 may be performed bymonitoring component 125 executed by one or more processing devices of thesource host 120A and/or thevirtualization manager 110. -
Method 200 may begin atblock 202. Atblock 202, a processing device may receive an indication over a network that a VM successfully migrated from a source host to a destination host. The indication may indicate that the VM migrated successfully and started on the destination host. The indication may also include the address of the VM that is executing on the destination host. In an example, the indication may include an instruction to a monitoring component to monitor for a specific type of incoming packets (e.g., a reverse address resolution protocol (RARP) packet type). This particular packet type may be used when a change is made to a topology of the network, and thus, may enhance robustness of detection at the monitoring components by looking for packets that include the address of the VM and have the packet type indicative of a change in the network. - At
block 204, the processing device may, responsive to the indication that the VM successfully migrated from the source host to the destination host, monitor incoming packets to the source host for an incoming packet having a VM address of the VM in a destination address field. A monitoring component may begin to monitor the network for incoming packets that have a destination address field set to the address of the VM. For example, the monitoring component may monitor a NIC of the source host for the received incoming packets. - Lastly, at
block 206, the processing device may, upon determining that one or more of the incoming packets to the source host include the destination address field having the VM address, provide, to the destination host, a notification that the one or more of the incoming packets having the VM address were received at the source host. The notification indicates that an update of the virtual machine address should be performed by one or more endpoints of the network (e.g., by the endpoint(s) from which the above incoming packets have originated or by all endpoints of the network). In one implementation, the destination hypervisor may include a notification component that causes the notification packet for the VM to be re-transmitted based on the received update notification from the monitoring component. The notification packet that is re-transmitted by the notification component may be referred to herein as a retransmitted notification packet. In some implementations, the retransmitted notification packet can be sent to the specified origin (e.g., endpoint) of the packet. In some implementations, the retransmitted notification packet may be broadcast to the entire network. -
FIG. 3 depicts a block diagram of anexample computer system 300, in accordance with one or more aspects of the disclosure.Computer system 300 may be the same or similar to thesource host 120A,destination host 120B, or thevirtualization manager 110 and may include one ormore processing devices 305 and one ormore memory devices 350. In the example shown,computer system 300 may includeindication receiving module 310,monitoring module 320, andnotification sending module 330. Also, as depicted,computer system 300 may be communicatively coupled to thesource host 302, anddestination host 304 via thenetwork 140. Although shown as separate from and connected to the source host 302 (e.g.,source host 120A ofFIG. 1 ), thecomputer system 300 may be part of the source host 302 (e.g.,source host 120A). In addition, although shown as separate from and connected to the destination host 304 (e.g.,destination host 120B ofFIG. 1 ), thecomputer system 300 may be part of the destination host 304 (e.g.,destination host 120B). Furthermore, thecomputer system 300 may be part of the virtualization manager (e.g.,virtualization manager 110 ofFIG. 1 ). - The
indication receiving module 310 may receive an indication over thenetwork 140 that aVM 124 successfully migrated from thesource host 302 to thedestination host 304. Successfully migrating may refer to theVM 124 successfully starting on the destination host. The indication may be received from thevirtualization manager 110 or thedestination host 304. In one implementation, the indication may include a VM address of theVM 124 executing on the destination host. In an example, thehypervisor 122A may remain active on thesource host 120A after receiving the indication that the migration was successful to enable monitoring for incoming packets to thesource host 302. - The
monitoring module 320 may, responsive to the indication that theVM 124 successfully migrated from thesource host 302 to thedestination host 304, start to monitor incoming packets 306 ofsource host 302 for an incoming packet 306 that includes adestination address field 308 having the VM address at thesource host 302. - The
notification sending module 330 may, upon determining that one or more of the incoming packets 306 at thesource host 302 include thedestination address field 308 having the VM address, provide a notification to thedestination host 304 that one or more incoming packets 306 were received at thesource host 302. This notification to thedestination host 304 is to facilitate retransmission of notification packets on the network to cause the VM address to be update to transmit to thedestination host 304. For example, the retransmission of a notification packet by thedestination host 304 may facilitate updating of forwarding tables in switches of thenetwork 140 with at least the new ports at which packets having the VM address arrive (e.g., the forwarding tables updated entries for the VM address with different incoming ports associated with packets received from theVM 124 on the destination host 304). -
FIG. 4 depicts a flow diagram of anexample method 400 for network updates for VM migration at a notification component, in accordance with one or more aspects of the disclosure.Method 400 includes operations performed by thedestination host 120A and/or the virtualization manager. In some implementations,method 400 may be performed by thesource host 120A, as well.Method 400 may be performed in the same or a similar manner as described above in regards tomethod 200.Method 400 may be performed by processing devices of thedestination host 120B executing thehypervisor 122B and thenotification component 127. -
Method 400 may begin atblock 402. Atblock 402, the processing device may receive an update notification from a monitoring component. In one implementation, the update notification can indicate that a packet directed to a migrated VM was received at a source host and dropped by the source host hypervisor. Subsequently, atblock 404, the processing device may determine whether a number of received update notifications at the destination host meets or exceeds a threshold value. - Then, at
block 406, the processing device may broadcast a retransmitted notification packet on the network responsive to the threshold value being met or exceeded. The threshold value may be determined by an administrator of the managed virtualization environment. Utilization of the threshold may provide a baseline where it is more economical (in terms of network bandwidth and/or load) to send a single broadcast message for the notification packet over the network to all components, instead of sending such notification packet to individual components on a one-on-one basis. The utilization of the threshold can help reduce overall network communications, while balancing against the cost of sending an unnecessary notification packet to components that may have already updated the address of the VM. Lastly, atblock 408, the processing device may transmit a retransmitted notification packet to individual endpoints on the network responsive to the threshold value not being exceeded. In one implementation, the update notification from the monitoring component may identify the endpoints associated with the received packets at the source host. The identified endpoints can then be individually contacted with the retransmitted notification packet using a unicast address of the endpoint. -
FIG. 5 depicts a flow diagram of anotherexample method 500 for network updates for VM migration at a notification component, in accordance with one or more aspects of the disclosure.Method 500 includes operations performed by thedestination host 120A and/or the virtualization manager. In some implementations,method 500 may be performed by thesource host 120A, as well.Method 500 may be performed in the same or a similar manner as described above in regards tomethods Method 500 may be performed by processing devices of thedestination host 120B executing thehypervisor 122B and thenotification component 127. -
Method 500 may begin atblock 502. Atblock 502, the processing device may receive an update notification from a monitoring component. In one implementation, the update notification can indicate that a packet directed to a migrated VM was received at a source host and dropped by the source host hypervisor. Atblock 504, the processing device may determine whether a number of received update notifications at the destination host exceeds a threshold rate. The threshold rate may include a ratio of number of update notifications received over a determined period of time. - At
block 406, the processing device may broadcast a retransmitted notification packet on the network responsive to the threshold rate being met or exceeded. The threshold rate may be determined by an administrator of the managed virtualization environment. Lastly, atblock 408, the processing device may transmit a retransmitted notification packet to individual endpoints on the network responsive to the threshold rate not being exceeded. In one implementation, the update notification from the monitoring component may identify the endpoints associated with the received packets at the source host. The identified endpoints can then be individually contacted with the retransmitted notification packet using a unicast address of the endpoint. -
FIG. 6 depicts a block diagram of a computer system operating in accordance with one or more aspects of the disclosure. In various illustrative examples,computer system 600 may correspond to a computing device withinsystem architecture 100 ofFIG. 1 . In one implementation, thecomputer system 600 may be thesource host 120A or thedestination host 120B. In another implementation, thecomputer system 600 may be hosting thevirtualization manager 110. Thecomputer system 600 may be included within a data center that supports virtualization. Virtualization within a data center results in a physical system being virtualized using VMs to consolidate the data center infrastructure and increase operational efficiencies. A VM may be a program-based emulation of computer hardware. For example, the VM may operate based on computer architecture and functions of computer hardware resources associated with hard disks or other such memory. The VM may emulate a physical computing environment, but requests for a hard disk or memory may be managed by a virtualization layer of a host system to translate these requests to the underlying physical computing hardware resources. This type of virtualization results in multiple VMs sharing physical resources. - In certain implementations,
computer system 600 may be connected (e.g., via a network, such as a Local Area Network (LAN), an intranet, an extranet, or the Internet) to other computer systems.Computer system 600 may operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment.Computer system 600 may be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, switch or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device. Further, the term “computer” shall include any collection of computers that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methods described herein. - In a further aspect, the
computer system 600 may include aprocessing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically-erasable programmable ROM (EEPROM)), and adata storage device 616, which may communicate with each other via abus 608. -
Processing device 602 may be provided by one or more processors such as a general purpose processor (such as, for example, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of types of instruction sets) or a specialized processor (such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor). -
Computer system 600 may further include anetwork interface device 622.Computer system 600 also may include a video display unit 610 (e.g., an LCD), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and asignal generation device 620. -
Data storage device 616 may include a non-transitory computer-readable storage medium 624 on which may storeinstructions 626 encoding any one or more of the methods or functions described herein, including instructions implementingmonitoring component 125 and/ornotification component 127 ofFIG. 1 for implementingmethod 200,method 400, andmethod 500. -
Instructions 626 may also reside, completely or partially, withinvolatile memory 604 and/or withinprocessing device 602 during execution thereof bycomputer system 600, hence,volatile memory 604 andprocessing device 602 may also constitute machine-readable storage media. - While computer-
readable storage medium 624 is shown in the illustrative examples as a single medium, the term “computer-readable storage medium” shall include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of executable instructions. The term “computer-readable storage medium” shall also include any tangible medium that is capable of storing or encoding a set of instructions for execution by a computer that cause the computer to perform any one or more of the methods described herein. The term “computer-readable storage medium” shall include, but not be limited to, solid-state memories, optical media, and magnetic media. - The methods, components, and features described herein may be implemented by discrete hardware components or may be integrated in the functionality of other hardware components such as ASICS, FPGAs, DSPs or similar devices. In addition, the methods, components, and features may be implemented by firmware modules or functional circuitry within hardware devices. Further, the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in computer programs.
- Unless specifically stated otherwise, terms such as “receiving,” “associating,” “deleting,” “initiating,” “marking,” “generating,” “recovering,” “completing,” or the like, refer to actions and processes performed or implemented by computer systems that manipulates and transforms data represented as physical (electronic) quantities within the computer system registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices. Also, the terms “first,” “second,” “third,” “fourth,” etc. as used herein are meant as labels to distinguish among different elements and may not have an ordinal meaning according to their numerical designation.
- Examples described herein also relate to an apparatus for performing the methods described herein. This apparatus may be specially constructed for performing the methods described herein, or it may comprise a general purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.
- The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform
methods - The above description is intended to be illustrative, and not restrictive. Although implementations of the disclosure have been described with references to specific illustrative examples and implementations, it will be recognized that the disclosure is not limited to the examples and implementations described. The scope of the disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which the claims are entitled.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/689,320 US11748131B2 (en) | 2019-11-20 | 2019-11-20 | Network updates for virtual machine migration |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/689,320 US11748131B2 (en) | 2019-11-20 | 2019-11-20 | Network updates for virtual machine migration |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210149701A1 true US20210149701A1 (en) | 2021-05-20 |
US11748131B2 US11748131B2 (en) | 2023-09-05 |
Family
ID=75909971
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/689,320 Active 2041-04-16 US11748131B2 (en) | 2019-11-20 | 2019-11-20 | Network updates for virtual machine migration |
Country Status (1)
Country | Link |
---|---|
US (1) | US11748131B2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116319354A (en) * | 2023-01-30 | 2023-06-23 | 杭州优云科技有限公司 | Network topology updating method based on cloud instance migration |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110063998A1 (en) * | 2009-08-20 | 2011-03-17 | Lumexis Corp | Serial networking fiber optic inflight entertainment system network configuration |
US20160119219A1 (en) * | 2014-10-26 | 2016-04-28 | Microsoft Technology Licensing, Llc | Method for reachability management in computer networks |
US20170302686A1 (en) * | 2016-04-14 | 2017-10-19 | Radware, Ltd. | System and method for real-time tuning of inference systems |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9424144B2 (en) | 2011-07-27 | 2016-08-23 | Microsoft Technology Licensing, Llc | Virtual machine migration to minimize packet loss in virtualized network |
US9197489B1 (en) * | 2012-03-30 | 2015-11-24 | Amazon Technologies, Inc. | Live migration of virtual machines in a hybrid network environment |
US9323566B2 (en) * | 2012-08-22 | 2016-04-26 | Hitachi, Ltd. | Virtual computer system for restoring network connection of live-migrated virtual computer |
US10013276B2 (en) | 2014-06-20 | 2018-07-03 | Google Llc | System and method for live migration of a virtualized networking stack |
US10033622B2 (en) * | 2015-08-07 | 2018-07-24 | Lenovo Enterprise Solutions (Singapore) Pte. Ltd. | Controller-based dynamic routing in a software defined network environment |
WO2018049567A1 (en) | 2016-09-13 | 2018-03-22 | 华为技术有限公司 | Application migration method, device, and system |
US10552080B2 (en) | 2017-07-28 | 2020-02-04 | Red Hat, Inc. | Multi-target post-copy guest migration |
US10838752B2 (en) | 2017-08-28 | 2020-11-17 | Red Hat Israel, Ltd. | Network notification loss detection for virtual machine migration |
US10645201B2 (en) * | 2018-07-31 | 2020-05-05 | Vmware, Inc. | Packet handling during service virtualized computing instance migration |
-
2019
- 2019-11-20 US US16/689,320 patent/US11748131B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110063998A1 (en) * | 2009-08-20 | 2011-03-17 | Lumexis Corp | Serial networking fiber optic inflight entertainment system network configuration |
US20160119219A1 (en) * | 2014-10-26 | 2016-04-28 | Microsoft Technology Licensing, Llc | Method for reachability management in computer networks |
US20170302686A1 (en) * | 2016-04-14 | 2017-10-19 | Radware, Ltd. | System and method for real-time tuning of inference systems |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116319354A (en) * | 2023-01-30 | 2023-06-23 | 杭州优云科技有限公司 | Network topology updating method based on cloud instance migration |
Also Published As
Publication number | Publication date |
---|---|
US11748131B2 (en) | 2023-09-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10838752B2 (en) | Network notification loss detection for virtual machine migration | |
US11102059B2 (en) | Virtual network health checker | |
US10225233B2 (en) | Media access control (MAC) address learning in virtualized computing environments | |
US9742726B2 (en) | Distributed dynamic host configuration protocol | |
CN107925633B (en) | Data center resource tracking | |
US8792502B2 (en) | Duplicate MAC address detection | |
US11343184B2 (en) | Methods and apparatus to manage a physical network to reduce network dependencies in a multi-fabric virtual network | |
US20190052520A1 (en) | Cooperative active-standby failover between network systems | |
US11283907B2 (en) | Determining state of virtual router instance | |
US10628198B2 (en) | Hypervisor management of migration notification and response messages for virtual machines | |
US11537422B2 (en) | Virtual machine migration downtime reduction using a multicast address | |
US8533320B2 (en) | Coalescing network notifications for live migration | |
US10397340B2 (en) | Multicast migration | |
US11968078B2 (en) | Maintaining network membership when disconnected from a controller | |
US9372708B2 (en) | Synchronizing multicast groups | |
CN103546556B (en) | One kind online moving method of virtual machine in future network XIA | |
CN113709220B (en) | High-availability implementation method and system of virtual load equalizer and electronic equipment | |
US11070629B2 (en) | Migration notification and response messages for virtual machines | |
US11748131B2 (en) | Network updates for virtual machine migration | |
US10333867B2 (en) | Active-active load-based teaming | |
US10608869B2 (en) | Handling control-plane connectivity loss in virtualized computing environments | |
US11528222B2 (en) | Decentralized control plane | |
Shpiner et al. | SAL: Scaling data centers using smart address learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: RED HAT, INC., NORTH CAROLINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TSIRKIN, MICHAEL;REEL/FRAME:051065/0085 Effective date: 20191119 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |