CN114995959A

CN114995959A - Virtual machine live migration control method and device, storage medium and electronic equipment

Info

Publication number: CN114995959A
Application number: CN202210770923.XA
Authority: CN
Inventors: 林宝洪; 师春雨; 刘艺; 李阳春; 吴林泽
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2022-09-02

Abstract

The present disclosure relates to the field of live migration technologies, and in particular, to a virtual machine live migration control method, a virtual machine live migration control apparatus, a storage medium, and an electronic device. The virtual machine live migration control method comprises the following steps: updating the VF information of the virtual function equipment in response to a live migration request sent by a management node; the live migration request is generated based on the virtual function device VF information of the source virtual machine VM; after the information of the virtual function device VF is updated, responding to an address mapping request sent by the management node, and updating address mapping information from a virtual address of the virtual function device VF to a physical memory address; after the address mapping information is updated, performing live migration on the source virtual machine VM to obtain a target virtual machine VM in response to the incremental state data of the source virtual machine VM, which is sent by the management node. The virtual machine live migration control method can solve the problem of performance loss in SR-IOVVF live migration.

Description

Virtual machine live migration control method and device, storage medium and electronic equipment

Technical Field

The present disclosure relates to the field of live migration technologies, and in particular, to a virtual machine live migration control method, a virtual machine live migration control apparatus, a storage medium, and an electronic device.

Background

In order to ensure high reliability of telecommunication services in an existing network production environment, a VNF is generally required to have a hot-migration capability. When a server where the VNF is located fails or resources are insufficient, virtualization may migrate the VNF to other server nodes meeting resource requirements under the condition that virtual machine startup and network uninterrupted are guaranteed, thereby ensuring that the VNF remains online and provides telecommunication services.

However, the introduction of the SR-IOV technology causes VNF live migration failure using the technology, because the establishment of a data channel of the SR-IOV VF VNIC (virtual network card) of the virtual machine depends on the server IOMMU mapping a virtual memory address of a VM network card to a server memory physical address, and such mapping cannot follow up VM live migration. However, in the existing SR-IOV VF live migration method, either traffic forwarding is always damaged on the normal VNIC layer, or performance loss occurs on the normal VNIC layer during migration, which results in that the VNF cannot fully utilize the hardware forwarding performance of the VF.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

The present disclosure aims to provide a virtual machine live migration control method, a virtual machine live migration control apparatus, a storage medium, and an electronic device, and aims to solve the problem of performance loss in SR-IOV VF live migration.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to an aspect of the embodiments of the present disclosure, a method for controlling virtual machine live migration is provided, including: updating the VF information of the virtual function equipment in response to a live migration request sent by a management node; the live migration request is generated based on the virtual function device VF information of the source virtual machine VM; after the information of the virtual function device VF is updated, responding to an address mapping request sent by the management node, and updating address mapping information from a virtual address of the virtual function device VF to a physical memory address; after the address mapping information is updated, responding to the increment state data of the source virtual machine VM sent by the management node, and performing hot migration on the source virtual machine VM to obtain a target virtual machine VM.

According to some embodiments of the present disclosure, based on the foregoing solution, the updating, in response to the live migration request sent by the management node, virtual function device VF information includes: analyzing the hot migration request to obtain SR-IOV VF updating operation; wherein the SR-IOV VF operation comprises an update operation and a network configuration operation; and calling a standard interface of the SR-IOV PF to execute the SR-IOV VF updating operation to update the VF resource configuration table of the node virtual function equipment so as to update the VF information of the virtual function equipment.

According to some embodiments of the present disclosure, based on the foregoing solution, the method further comprises: and creating a node virtual function device VF resource configuration table based on the node information, the virtual machine VM information, the virtual function device VF information and the network configuration information.

According to some embodiments of the present disclosure, based on the foregoing scheme, the updating address mapping information from a virtual address to a physical memory address of the virtual function device VF includes: resolving the address mapping request to obtain an IOMMU address mapping updating operation; and calling a standard interface of the IOMMU to execute the IOMMU address mapping updating operation to update the address mapping table of the node virtual function equipment VF so as to update the address mapping of the virtual function equipment VF.

According to some embodiments of the present disclosure, based on the foregoing, the method further comprises: and creating a mapping table of the VF address of the node virtual function equipment based on the node information, the VM information of the virtual machine, the VF information of the virtual function equipment, the virtual address information and the physical memory address information.

According to a second aspect of the embodiments of the present disclosure, there is provided a virtual machine live migration control method, including: sending a hot migration request to a target node to enable the target node to update virtual function equipment (VF) information; after the target node updates the information of the virtual function device VF, sending an address mapping request to the target node, so that the target node updates the address mapping information from the virtual address of the virtual function device VF to the physical memory address; after the target node updates the address mapping information, sending the incremental state data of the source virtual machine VM to the target node, so that the target node performs live migration on the source virtual machine VM to obtain a target virtual machine VM.

According to some embodiments of the present disclosure, based on the foregoing solution, the method further comprises: after the target node updates the VF information, updating a global virtual function device VF resource configuration table; and after the target node updates the address mapping information, updating a global virtual function device (VF) address mapping table.

According to some embodiments of the present disclosure, based on the foregoing solution, the method further comprises: reading node virtual function equipment VF resource configuration tables of all nodes to create a global virtual function equipment VF resource configuration table; and reading the node virtual function device VF address mapping tables of all the nodes to create the global virtual function device VF address mapping table.

According to some embodiments of the present disclosure, based on the foregoing solution, the method further comprises: and deleting the VF information of the virtual function equipment of the source virtual machine VM.

According to a third aspect of the embodiments of the present disclosure, there is provided a virtual machine live migration control apparatus, including: the local resource control module is used for responding to the live migration request sent by the management node to update the VF information of the virtual function equipment; the live migration request is generated based on the virtual function device VF information of the source virtual machine VM; the local mapping control module is configured to update, after updating the information of the virtual function device VF, address mapping information from a virtual address of the virtual function device VF to a physical memory address in response to an address mapping request sent by the management node; and the local live migration module is used for responding to the increment state data of the source virtual machine VM sent by the management node after the address mapping information is updated, and carrying out live migration on the source virtual machine VM to obtain a target virtual machine VM.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a virtual machine live migration control apparatus, including: the system comprises a global resource control module, a virtual function device (VF) module and a virtual resource management module, wherein the global resource control module is used for sending a hot migration request to a target node so as to enable the target node to update the VF information of the virtual function device; a global mapping control module, configured to send an address mapping request to the target node after the target node updates the information of the virtual function device VF, so that the target node updates address mapping information from a virtual address of the virtual function device VF to a physical memory address; and the global live migration module is used for sending the incremental state data of the source virtual machine VM to the target node after the target node updates the address mapping information, so that the target node carries out live migration on the source virtual machine VM to obtain a target virtual machine VM.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing a virtual machine live migration control method as in the above embodiments.

According to a sixth aspect of the embodiments of the present disclosure, there is provided an electronic apparatus, comprising: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the virtual machine live migration control method in the above embodiment.

Exemplary embodiments of the present disclosure may have some or all of the following benefits:

in the technical solutions provided in some embodiments of the present disclosure, when a source virtual machine VM corresponding to a source node needs to be migrated, a management node Hypervisor Manager performs global management, and before the source virtual machine VM is migrated, migrates a virtual function device VF of the source virtual machine VM to a target node Hypervisor, and further copies incremental state data related to the source virtual machine VM migration to the target node Hypervisor and implements switching from the source virtual machine VM to the target virtual machine VM. Based on the method, on one hand, the device address mapping migration and the network configuration migration related to the VF thermal migration are realized, so that the actual network card configuration and the state of the VM are unchanged when the VM thermal migration is completed, and the data channel is still in direct hardware communication, thereby fundamentally solving the problem that the VF cannot be thermally migrated and having no energy loss; on the other hand, the virtualization software and the server network card hardware are not changed, so that the existing virtualization product can be directly integrated for use, and the use and maintenance cost is low.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:

fig. 1 schematically illustrates a flowchart of a virtual machine live migration control method in an exemplary embodiment of the present disclosure;

fig. 2 schematically illustrates a structural diagram of a local node Hypervisor in an exemplary embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating another method for controlling live migration of a virtual machine according to an exemplary embodiment of the disclosure;

FIG. 4 is a schematic diagram illustrating a structure of a Hypervisor Manager in an exemplary embodiment of the present disclosure;

FIG. 5 is a system architecture diagram schematically illustrating a virtual machine live migration control method in an exemplary embodiment of the present disclosure;

fig. 6 is a schematic diagram illustrating a composition of a virtual machine live migration control apparatus according to an exemplary embodiment of the present disclosure;

fig. 7 is a schematic diagram illustrating a composition of a virtual machine live migration control apparatus according to an exemplary embodiment of the present disclosure;

FIG. 8 schematically illustrates a schematic diagram of a computer-readable storage medium in an exemplary embodiment of the disclosure;

fig. 9 schematically shows a structural diagram of a computer system of an electronic device in an exemplary embodiment of the disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the disclosure.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

In the background of cloud network convergence, with the continuous advancement of cloud on the network, telecommunication network elements represented by 5GC are gradually deployed on the virtualization of a VNF (virtual machine type network element) in the form of a VNF (virtual machine type network element) on a cloud infrastructure general server, and a high-performance network plane is required to be provided for VNF forwarding plane acceleration by the virtualization.

In order to ensure high reliability of telecommunication services in an existing network production environment, a VNF is generally required to have a hot-migration capability. When a server where the VNF is located fails or resources are insufficient, virtualization may migrate the VNF to other server nodes meeting resource requirements under the condition that virtual machine startup and network uninterrupted are guaranteed, thereby ensuring that the VNF remains online and provides telecommunication services. However, the introduction of the SR-IOV technology causes VNF live migration failure using the technology, because the establishment of a data channel of the SR-IOV VF VNIC (virtual network card) of the virtual machine depends on the server IOMMU mapping a virtual memory address of a VM network card to a server memory physical address, and such mapping cannot follow up VM live migration.

At present, there are two solutions to the above problems: 1. a new layer of normal VNIC (for example, virtio type VNIC) is added between the VM and the VF, and the address mapping after VF migration is changed to the normal VNIC, so that from the viewpoint of the VM, the virtual network card is always normal VNIC, and the network configuration and state after migration are unchanged. 2. And newly adding a normal type VNIC for the VM, making a Bond with the original SR-IOV VF, triggering redundant switching during VM hot migration, carrying VM flow by the newly added VNIC, and switching back to the original VF after the migration is finished.

Both schemes realize SR-IOV VF thermal migration based on an indirect mode, but both have the problem of performance loss: 1. traffic forwarding is always lossy on the normal VNIC layer; 2. there is a performance loss at the normal VNIC layer during migration) resulting in the VNF not being able to fully utilize the hardware forwarding performance of the VF.

Therefore, aiming at the defects of the prior art, the present disclosure provides a virtual machine live migration control method, a global SR-IOV VF live migration controller is constructed in a management node Hypervisor Manager, a local SR-IOV VF live migration adapter is constructed in a local node Hypervisor, and an SR-IOV VF live migration control strategy is designed based on the cooperation of the global SR-IOV VF live migration controller and the local SR-IOV VF live migration adapter, so that the problem that VF cannot be live migrated can be fundamentally solved, energy loss is not generated, and live migration of a virtual machine type telecommunication network element with a high-performance forwarding plane constructed based on an SR-IOV technology can be realized.

Implementation details of the technical solution of the embodiments of the present disclosure are set forth in detail below.

Fig. 1 schematically illustrates a flowchart of a virtual machine live migration control method in an exemplary embodiment of the present disclosure. As shown in fig. 1, the virtual machine live migration control method includes steps S101 to S103:

step S101, updating virtual function equipment (VF) information in response to a live migration request sent by a management node; the live migration request is generated based on the virtual function device VF information of the source virtual machine VM;

step S102, after updating the information of the virtual function device VF, responding to an address mapping request sent by the management node, and updating address mapping information from a virtual address of the virtual function device VF to a physical memory address;

step S103, after updating the address mapping information, performing live migration on the source virtual machine VM to obtain a target virtual machine VM in response to the incremental state data of the source virtual machine VM sent by the management node.

It should be noted that, in the virtual machine live migration control method disclosed by the present disclosure, the Hypervisor Manager cooperates with the plurality of local nodes Hypervisor to implement, and the virtual machine VM is live migrated from the source node Hypervisor to the target node Hypervisor. The above steps S101 to S103 are executed by the target node Hypervisor.

Wherein, for the management node Hypervisor Manager, the global Hypervisor is managed. A global SR-IOV hot migration controller is introduced into a Hypervisor Manager, the controller realizes SR-IOV VF full life cycle management on a global server node, and cooperatively manages and controls VF to follow VM hot migration. And the Hypervisor Manager cooperates with the global SR-IOV VF live migration module to control the increase, deletion, modification and check of global related resources and configuration in the VM live migration process.

And for the local node Hypervisor, receiving and executing a Hypervisor Manager live migration instruction. A local SR-IOV (scheduling object-oriented virtualization) live migration adapter is introduced into a local node Hypervisor, and the adapter realizes entity operation of VF live migration on a source server node and a target server node. And the local node Hypervisor cooperates with the local SR-IOV VF live migration module to control the increase, deletion, modification and check of local related resources and configuration in the VM live migration process.

Hereinafter, each step of the virtual machine live migration control method in the present exemplary embodiment will be described in more detail with reference to the drawings and the embodiments.

In step S101, updating virtual function device VF information in response to a live migration request sent by a management node; wherein the live migration request is generated based on the virtual function device VF information of the source virtual machine VM.

In one embodiment of the present disclosure, step S101 is performed by the target node Hypervisor. The Hypervisor is configured with a local SR-IOV live migration adapter, where the adapter includes a local resource control module, and the module completes local VF lifecycle management and network configuration based on a local VF resource configuration table and a VF control method.

Therefore, step S101 is specifically executed by the local resource control module. Specifically, the updating the virtual function device VF information in response to the live migration request sent by the management node includes: analyzing the hot migration request to obtain SR-IOV VF updating operation; wherein the SR-IOV VF operation comprises an update operation and a network configuration operation; and calling a standard interface of the SR-IOV PF to execute the SR-IOV VF updating operation to update the node virtual function device VF resource configuration table so as to update the virtual function device VF information.

Specifically, after receiving the live migration request, the local resource control module decomposes the live migration request into SR-IOV VF operations, including update operations such as lifecycle-related operations of VF creation, deletion, and modification, and network configuration operations such as configuring network attributes such as MAC and VLAN. These operations may be implemented via Hypervisor invoking the standard interface of the SR-IOV PF.

In one embodiment of the present disclosure, the method further comprises: and creating a node virtual function device VF resource configuration table based on the node information, the virtual machine VM information, the virtual function device VF information and the network configuration information.

Each node maintains a node VF resource configuration table at the node itself, and records VF devices and corresponding network configuration attributes bound by VMs on the Hypervisor node. As shown in table 1:

table 1VF resource allocation table

In step S102, after the information of the virtual function device VF is updated, the address mapping information from the virtual address of the virtual function device VF to the physical memory address is updated in response to the address mapping request sent by the management node.

In one embodiment of the present disclosure, step S102 is also performed by the target node Hypervisor. The target node Hypervisor is configured with a local SR-IOV live migration adapter, wherein the adapter comprises a local mapping control module which completes the establishment and the removal of mapping from the local VF virtual memory address to the server physical memory address before and after VM migration based on a VF address mapping table and an IOMMU control method.

Therefore, step S102 is specifically executed by the local mapping control module. Specifically, the updating of the address mapping information from the virtual address to the physical memory address of the virtual function device VF includes: resolving the address mapping request to obtain an IOMMU address mapping updating operation; and calling a standard interface of the IOMMU to execute the IOMMU address mapping updating operation to update the address mapping table of the node virtual function equipment VF so as to update the address mapping of the virtual function equipment VF.

Specifically, after receiving the address mapping request, the local mapping control module decomposes the operation request into an IOMMU address mapping update operation, which is mainly to establish and remove the mapping from the device virtual address to the physical memory address for the VF according to the VF operation result of the local resource control module, thereby realizing the establishment and removal of the VF to VM direct data link. These operations are implemented by means of the Hypervisor calling the standard interface of the IOMMU. These operations are implemented by means of the Hypervisor calling the standard interface of the IOMMU.

Each node maintains a VF address mapping table of the node virtual function device at the node itself, and records a mapping relationship between a virtual address of a VF device of the VM on the Hypervisor node and a physical memory address. As shown in table 2:

table 2VF address mapping table

In step S103, after the address mapping information is updated, performing live migration on the source virtual machine VM to obtain a target virtual machine VM in response to the incremental state data of the source virtual machine VM sent by the management node.

In an embodiment of the present disclosure, after migrating the source VM VF to the target node, the virtual machine VM is migrated again, that is, incremental state data related to VM live migration is copied to the target virtualization node, and source and target VM switching is performed. The VM live migration technique can be implemented by the prior art, and will not be described in detail herein.

Fig. 2 schematically illustrates a structural diagram of a local node Hypervisor in an exemplary embodiment of the present disclosure. As shown in fig. 2, a local SR-IOV live migration adapter is introduced into the Hypervisor node, and the local SR-IOV live migration adapter includes a local resource control module (i.e., a local VF resource control submodule in the drawing) and a local mapping control module (i.e., a local device address mapping control submodule in the drawing). The adapter realizes entity operation of VF live migration on source and target server nodes, and adapts VF related operation requests issued by the global SR-IOV live migration controller to instructions of local Hypervisor for operating the IOMMU and the SR-IOV network card.

Based on the method, the virtualization management node Hypervisor Manager cooperates with the Hypervisor to copy the incremental state data related to VM live migration to the target virtualization node and migrate the source VM VF to the target node before the source VM and the target VM are switched, so that when the VM live migration is completed, the network card configuration and the state of the actual VM are not changed, and the network is not interrupted. Meanwhile, virtualization software, server IOMMU (input/output unit) and SR-IOV (scheduling request-input/output) network card hardware are not changed during VM (virtual machine) live migration, and the existing virtualization products can be directly integrated for use.

Fig. 3 schematically illustrates a flowchart of another virtual machine live migration control method in an exemplary embodiment of the present disclosure. As shown in fig. 3, the virtual machine live migration control method includes steps S301 to S303:

step S301, sending a live migration request to a target node to enable the target node to update virtual function equipment VF information;

step S302, after the target node updates the information of the virtual function device VF, sending an address mapping request to the target node, so that the target node updates the address mapping information from the virtual address of the virtual function device VF to the physical memory address;

step S303, after the target node updates the address mapping information, sending incremental state data of the source virtual machine VM to the target node, so that the target node performs a live migration on the source virtual machine VM to obtain a target virtual machine VM.

It should be noted that steps S301 to S303 are executed by the Hypervisor Manager. The management node Hypervisor Manager manages the global Hypervisor local node, realizes SR-IOV VF full life cycle management on the global server node, and collaboratively manages and controls VF to follow VM live migration.

In step S301, a live migration request is sent to a target node, so that the target node updates virtual function device VF information.

In one embodiment of the present disclosure, step S301 is performed by the Hypervisor Manager. The Hypervisor Manager is configured with a global SR-IOV hot migration controller, wherein the controller comprises a global resource control module which uniformly controls the VF life cycle and network configuration before and after the global VM migration based on a global VF resource configuration table and a VF resource operation API.

Therefore, step S301 is specifically executed by the global resource control module. Specifically, the global resource control module initiates VF resource adding, deleting, modifying, and checking operations to the specified target node Hypervisor during VM migration by calling the VF resource operation API, and cooperates with the global mapping control module to implement live migration of VF and network configuration. Specifically, the structure of the VF resource operation API is shown in table 3.

TABLE 3VF resource operation API

In step S302, after the target node updates the information of the virtual function device VF, an address mapping request is sent to the target node, so that the target node updates the address mapping information from the virtual address of the virtual function device VF to the physical memory address.

In one embodiment of the present disclosure, step S302 is performed by the Hypervisor Manager. The global SR-IOV hot migration controller in the Hypervisor Manager comprises a global mapping control module, which initiates address mapping increasing, deleting and searching operations of VF equipment to a specified Hypervisor in the VM migration process by calling a VF address mapping operation API, and realizes hot migration of VF address mapping in cooperation with a global resource control module. Specifically, the structure of the VF address mapping operation API is shown in table 4.

TABLE 4VF Address mapping operation API

In step S303, after the target node updates the address mapping information, sending incremental state data of the source virtual machine VM to the target node, so that the target node performs a live migration on the source virtual machine VM to obtain a target virtual machine VM.

In one embodiment of the present disclosure, the method further comprises: after the target node updates the virtual function device VF information, updating a global virtual function device VF resource configuration table; and after the target node updates the address mapping information, updating a global virtual function device (VF) address mapping table.

Specifically, a global SR-IOV hot migration controller of a management node Hypervisor Manager realizes SR-IOV VF full life cycle management on a global server node, and cooperatively manages and controls VF to follow VM hot migration.

Therefore, the global VF resource configuration table and the global VF address mapping table need to be maintained, and after the target node updates the virtual function device VF information and updates the address mapping information, the two tables need to be updated synchronously.

In one embodiment of the present disclosure, the method further comprises: the method further comprises the following steps: reading node virtual function equipment VF resource configuration tables of all nodes to create a global virtual function equipment VF resource configuration table; and reading the node virtual function device VF address mapping tables of all the nodes to create the global virtual function device VF address mapping table.

The global VF resource configuration table is uniformly maintained by the global resource control module in the VM migration process, and records all VF devices bound by the VMs on the Hypervisor and corresponding network configuration attributes.

The global VF address mapping table is uniformly maintained by a global address mapping control module in the VM migration process, and records the mapping relation from the virtual address of the VF equipment of the VM to the physical memory address on all hypervisors.

Specifically, the table structures of the global VF resource configuration table and the global VF address mapping table are similar to the local VF resource configuration table and the local VF address mapping table, and are constructed by the global SR-IOV hot migration controller reading table information of each local node.

In one embodiment of the present disclosure, the method further comprises: after the source virtual machine VM is subjected to the hot migration to obtain a target virtual machine VM, deleting the VF information of the virtual function device of the source virtual machine VM.

Specifically, after the completion of the hot migration, the Hypervisor Manager may recycle the source VM resource. The global SR-IOV VF hot migration controller releases and recovers resources related to a source node VF and network configuration by calling a VF resource operation API and a VF address mapping operation API.

Fig. 4 schematically illustrates a structural diagram of a Hypervisor Manager in an exemplary embodiment of the present disclosure. As shown in fig. 4, a global SR-IOV hot migration controller is introduced into the Hypervisor Manager, and the controller is composed of a global resource control module (i.e., a global VF resource control submodule in the diagram) and a global address mapping module (i.e., a global device address mapping control submodule in the diagram). The controller realizes SR-IOV VF full life cycle management on the global server node, and cooperatively manages and controls VF to follow VM live migration.

Fig. 5 is a system architecture diagram schematically illustrating a virtual machine live migration control method in an exemplary embodiment of the present disclosure. Referring to fig. 5, a global SR-IOV VF live migration controller and a local SR-IOV VF live migration adapter are introduced, and an operation API, a control flow, a data table, and the like of a module are defined.

The system comprises a local SR-IOV thermal migration adapter, a global SR-IOV thermal migration controller, a global virtualization management node Hypervisor Manager, a global SR-IOV thermal migration controller and a local virtualization Hypervisor, wherein the global SR-IOV thermal migration controller is in butt joint with the global virtualization management node Hypervisor Manager, and the local SR-IOV thermal migration adapter is in butt joint with the local virtualization Hypervisor and the global SR-IOV thermal migration controller. Specifically, the virtual machine live migration control method comprises the following steps:

(1) in the initial stage, the local SR-IOV VF thermal migration adapter acquires information such as VM (virtual machine), VM VF (virtual machine) and the like from a local Hypervisor, and the information is analyzed into a Hypervisor ID (identity), a VM ID (identity), a VF ID, VF network attribute configuration and the like and written into a VF resource configuration table and a VF address mapping table by a local address mapping module and a resource control module; the global SR-IOV VF hot migration controller reads VF resource configuration and VF address mapping on all Hypervisor nodes through a VF resource operation API and a VF address mapping operation API so as to form a global VF address mapping table and a global VF resource configuration table;

(2) when VM live migration is initiated from a Hypervisor Manager, a global resource control module initiates a VF resource operation API (network configuration information carrying VF migration) to a live migration target Hypervisor node, and a local resource control module on the target Hypervisor node creates a VF according to a received request, so as to complete copying from a source node VF to a target node;

(3) the method comprises the steps that a global resource control module updates a global VF resource configuration table after receiving a response of newly building a VF by a target Hypervisor node, then a global address mapping module is triggered to initiate a VF address mapping operation API to the hot migration target Hypervisor node, and a local address mapping module on the target Hypervisor node builds mapping from a virtual address of a VM VF device to a physical memory address according to a received request, so that copying from a source node VF data channel to the target node is completed;

(4) after receiving a response of new address mapping of a target Hypervisor node, the global address mapping module updates a global VF address mapping table, so that the target node is ready for VF of VM (virtual machine) live migration, and then the Hypervisor Manager cooperates with the Hypervisor to complete incremental state data copying related to VM live migration and implement VM switching (source VM offline and target VM online);

(5) the Hypervisor Manager recovers source VM resources, wherein the global SR-IOV VF hot migration controller releases and recovers resources related to a source node VF and network configuration by calling a VF resource operation API and a VF address mapping operation API.

Based on the method, the method for realizing SR-IOV VF thermal migration is provided, the method constructs a global SR-IOV VF thermal migration controller and a local SR-IOV VF thermal migration adapter, and designs an SR-IOV VF thermal migration control strategy based on the cooperation of the global SR-IOV VF thermal migration controller and the local SR-IOV VF thermal migration adapter, and realizes device address mapping migration and network configuration migration related to VF thermal migration, so that the following of VM thermal migration VF and network configuration is realized, the problem that VF cannot be thermally migrated is fundamentally solved, and no energy loss is generated.

Fig. 6 schematically illustrates a composition diagram of a virtual machine live migration control apparatus in an exemplary embodiment of the present disclosure, and as shown in fig. 6, the virtual machine live migration control apparatus 600 may include a local resource control module 601, a local mapping control module 602, and a local live migration module 603. Wherein:

the local resource control module 601, in response to the live migration request sent by the management node, updates the information of the virtual function device VF; the live migration request is generated based on the virtual function device VF information of the source virtual machine VM;

a local mapping control module 602, configured to update, after updating the information of the virtual function device VF, address mapping information from a virtual address of the virtual function device VF to a physical memory address in response to an address mapping request sent by the management node;

a local live migration module 603, configured to, after updating the address mapping information, respond to the incremental state data of the source virtual machine VM sent by the management node, perform live migration on the source virtual machine VM to obtain a target virtual machine VM.

According to an exemplary embodiment of the present disclosure, the local resource control module 601 is configured to analyze the live migration request to obtain an SR-IOV VF update operation; wherein the SR-IOV VF operation comprises an update operation and a network configuration operation; and calling a standard interface of the SR-IOV PF to execute the SR-IOV VF updating operation to update the node virtual function device VF resource configuration table so as to update the virtual function device VF information.

According to an exemplary embodiment of the present disclosure, the local resource control module 601 further includes a resource configuration table unit, configured to create the node virtual function device VF resource configuration table based on node information, virtual machine VM information, virtual function device VF information, and network configuration information.

According to an exemplary embodiment of the present disclosure, the local mapping control module 602 is configured to resolve the address mapping request to obtain an IOMMU address mapping update operation; and calling a standard interface of the IOMMU to execute the IOMMU address mapping updating operation to update the address mapping table of the node virtual function equipment VF so as to update the address mapping of the virtual function equipment VF.

According to an exemplary embodiment of the present disclosure, the local mapping control module 602 further includes an address mapping table unit, configured to create the node virtual function device VF address mapping table based on node information, virtual machine VM information, virtual function device VF information, virtual address information, and physical memory address information.

The details of each module in the virtual machine live migration control apparatus 600 are already described in detail in the corresponding virtual machine live migration control method, and therefore are not described herein again.

Fig. 7 schematically illustrates a schematic composition diagram of another virtual machine live migration control apparatus in an exemplary embodiment of the present disclosure, and as shown in fig. 7, the virtual machine live migration control apparatus 700 may include a global resource control module 701, a global mapping control module 702, and a global live migration module 703. Wherein:

a global resource control module 701, configured to send a live migration request to a target node, so that the target node updates information of a virtual function device VF;

a global mapping control module 702, configured to send an address mapping request to the target node after the target node updates the information of the virtual function device VF, so that the target node updates the address mapping information from the virtual address of the virtual function device VF to the physical memory address;

a global live migration module 703 is configured to send, to the target node, incremental state data of the source virtual machine VM after the target node updates the address mapping information, so that the target node performs live migration on the source virtual machine VM to obtain a target virtual machine VM.

According to an exemplary embodiment of the present disclosure, the virtual machine live migration control apparatus 600 further includes an updating module, configured to update a global virtual function device VF resource configuration table after the target node updates the virtual function device VF information; and after the target node updates the address mapping information, updating a global virtual function device (VF) address mapping table.

According to an exemplary embodiment of the present disclosure, the update module further includes a creating module, configured to read node virtual function device VF resource configuration tables of all nodes to create the global virtual function device VF resource configuration table; and reading the node virtual function device VF address mapping tables of all the nodes to create the global virtual function device VF address mapping table.

According to an exemplary embodiment of the present disclosure, the virtual machine live migration control apparatus 600 further includes a releasing module, configured to delete the virtual function device VF information of the source virtual machine VM after the source virtual machine VM is live migrated to obtain the target virtual machine VM.

The details of each module in the virtual machine live migration control apparatus 700 are already described in detail in the corresponding virtual machine live migration control method, and therefore are not described herein again.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

In an exemplary embodiment of the present disclosure, there is also provided a storage medium capable of implementing the above-described method. Fig. 8 schematically illustrates a schematic diagram of a computer-readable storage medium in an exemplary embodiment of the disclosure, and as shown in fig. 7, depicts a program product 800 for implementing the above method according to an embodiment of the disclosure, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a mobile phone. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. Fig. 9 schematically illustrates a structural diagram of a computer system of an electronic device in an exemplary embodiment of the disclosure.

It should be noted that the computer system 900 of the electronic device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiments of the present disclosure.

As shown in fig. 9, a computer system 900 includes a Central Processing Unit (CPU)901 that can perform various appropriate actions and processes in accordance with a program stored in a Read-Only Memory (ROM) 902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for system operation are also stored. The CPU 901, ROM902, and RAM 903 are connected to each other via a bus 904. An Input/Output (I/O) interface 905 is also connected to bus 904.

The following components are connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, and the like; an output section 907 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage portion 908 including a hard disk and the like; and a communication section 909 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read out therefrom is mounted into the storage section 908 as necessary.

In particular, the processes described below with reference to the flowcharts may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 909, and/or installed from the removable medium 911. The computer program executes various functions defined in the system of the present disclosure when executed by a Central Processing Unit (CPU) 901.

It should be noted that the computer readable medium shown in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

As another aspect, the present disclosure also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may be separate and not incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method described in the above embodiments.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A virtual machine live migration control method is characterized by comprising the following steps:

updating the VF information of the virtual function equipment in response to a live migration request sent by a management node; the live migration request is generated based on the virtual function device VF information of the source virtual machine VM;

after the information of the virtual function device VF is updated, responding to an address mapping request sent by the management node, and updating address mapping information from a virtual address of the virtual function device VF to a physical memory address;

after the address mapping information is updated, responding to the increment state data of the source virtual machine VM sent by the management node, and performing hot migration on the source virtual machine VM to obtain a target virtual machine VM.

2. The virtual machine live migration control method according to claim 1, wherein the updating of the virtual function device VF information in response to the live migration request sent by the management node includes:

analyzing the hot migration request to obtain SR-IOV VF updating operation; wherein the SR-IOV VF operation comprises an update operation and a network configuration operation;

and calling a standard interface of the SR-IOV PF to execute the SR-IOV VF updating operation to update the node virtual function device VF resource configuration table so as to update the virtual function device VF information.

3. The virtual machine live migration control method according to claim 2, further comprising:

and creating a node virtual function device VF resource configuration table based on the node information, the virtual machine VM information, the virtual function device VF information and the network configuration information.

4. The method according to claim 1, wherein the updating address mapping information from a virtual address to a physical memory address of the virtual function device VF includes:

resolving the address mapping request to obtain an IOMMU address mapping updating operation;

and calling a standard interface of the IOMMU to execute the IOMMU address mapping updating operation to update the address mapping table of the node virtual function equipment VF so as to update the address mapping of the virtual function equipment VF.

5. The virtual machine live migration control method according to claim 4, further comprising:

and creating a node virtual function device VF address mapping table based on the node information, the virtual machine VM information, the virtual function device VF information, the virtual address information and the physical memory address information.

6. A method for controlling virtual machine live migration is characterized by comprising the following steps:

sending a hot migration request to a target node to enable the target node to update virtual function equipment (VF) information;

after the target node updates the information of the virtual function device VF, sending an address mapping request to the target node, so that the target node updates the address mapping information from the virtual address of the virtual function device VF to the physical memory address;

after the target node updates the address mapping information, sending incremental state data of the source virtual machine VM to the target node, so that the target node performs hot migration on the source virtual machine VM to obtain a target virtual machine VM.

7. The virtual machine live migration control method according to claim 1, further comprising:

after the target node updates the virtual function device VF information, updating a global virtual function device VF resource configuration table; and

and after the target node updates the address mapping information, updating a global virtual function device (VF) address mapping table.

8. The virtual machine live migration control method according to claim 7, further comprising:

reading node virtual function equipment VF resource configuration tables of all nodes to create a global virtual function equipment VF resource configuration table; and

and reading the node virtual function device VF address mapping tables of all the nodes to create the global virtual function device VF address mapping table.

9. The virtual machine live migration control method according to claim 6, further comprising:

after the source virtual machine VM is subjected to the hot migration to obtain a target virtual machine VM, deleting the VF information of the virtual function device of the source virtual machine VM.

10. A virtual machine live migration control apparatus, comprising:

the local resource control module is used for responding to the live migration request sent by the management node to update the VF information of the virtual function equipment; the live migration request is generated based on the virtual function device VF information of the source virtual machine VM;

the local mapping control module is configured to update, after updating the information of the virtual function device VF, address mapping information from a virtual address of the virtual function device VF to a physical memory address in response to an address mapping request sent by the management node;

and the local hot migration module is used for responding to the incremental state data of the source virtual machine VM sent by the management node after the address mapping information is updated, and performing hot migration on the source virtual machine VM to obtain a target virtual machine VM.

11. A virtual machine live migration control apparatus, comprising:

the system comprises a global resource control module, a virtual function device (VF) module and a virtual resource management module, wherein the global resource control module is used for sending a hot migration request to a target node so as to enable the target node to update the VF information of the virtual function device;

a global mapping control module, configured to send an address mapping request to the target node after the target node updates the information of the virtual function device VF, so that the target node updates address mapping information from a virtual address of the virtual function device VF to a physical memory address;

and the global hot migration module is used for sending the incremental state data of the source virtual machine VM to the target node after the target node updates the address mapping information so that the target node carries out hot migration on the source virtual machine VM to obtain a target virtual machine VM.

12. A computer-readable storage medium on which a computer program is stored, which program, when executed by a processor, implements the virtual machine live migration control method according to any one of claims 1 to 9.

13. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the virtual machine live migration control method of any one of claims 1 to 9.