Disclosure of Invention
In view of the foregoing problems, an object of the present invention is to provide a system for improving virtualization performance, which provides a method for greatly reducing vmexit related to device virtualization, which further improves virtualization efficiency and reduces virtualization performance loss, different from a method for device virtualization by using a conventional mmo, pio, and vintr mechanism.
It is a second object of the present invention to provide a physical machine.
The first technical scheme adopted by the invention is as follows: a system for enhancing virtualization performance, comprising a computing device for interacting directly with a virtual machine program over a computer system bus; the direct interaction comprises interaction in a mode of accessing and/or sending a message through a memory address space;
the interaction through the memory address space access specifically includes:
the virtual machine program operates an interface of a protocol by accessing a memory address space on the computer system bus; an interface for the protocol is provided by the computing device;
the interacting in the message sending manner specifically includes:
the computing device sends an interrupt message on the computer system bus and/or sets a signal value in a memory address to inform the virtual machine program so as to realize direct interaction between the computing device and the virtual machine program.
Preferably, a first-level page table or a second-level page table is established in the processor, which satisfies the mapping relationship from the physical address of the virtual machine to the physical address of the physical machine, so that the virtual machine program accesses the physical address of the physical machine where the corresponding resource is located by accessing the physical address of the virtual machine, thereby operating the interface.
Preferably, the computing device writes the data required by the virtual machine program directly into the memory address space of the virtual machine by:
the virtual machine program configures the physical address of the virtual machine written by the computing device, and the kernel VMM _ kernel of the virtual machine monitor translates the physical address of the virtual machine into the physical address of the physical machine, so that the computing device can write the corresponding physical address of the virtual machine by using the physical address of the physical machine in an addressing mode.
Preferably, in the system with the IOMMU, the VMM _ kernel sets the IOMMU based on a relationship between the virtual machine physical address and the physical machine physical address, and performs a translation from the IO virtual address to the physical machine physical address, so as to enable the computing device to write the IO virtual address into the virtual machine memory directly.
The second technical scheme adopted by the invention is as follows: a physical machine comprising a processor, a memory, a virtual machine monitor and a system VMACC for improving virtualisation performance as described in the first aspect; the VMM _ kernel in the virtual machine monitor manages hardware resources and runs a plurality of VMM _ user processes, the VMM _ user processes run a plurality of virtual machines, and the virtual machines run virtual machine programs.
Preferably, the VMM _ kernel sets a parameter of the vmac, so that the vmac directly initiates a memory read-write operation to a virtual machine physical address corresponding to the virtual machine by using the physical machine physical address;
the VMM _ kernel and the virtual machine program negotiate an interrupt vector number, and the VMACC is configured to send an interrupt message to the interrupt vector number.
Preferably, an IOMMU is also included;
the VMM _ kernel configures an interrupt translation table of the IOMMU, maps a physical interrupt vector number of the VMACC to an interrupt vector number set by a virtual machine program, configures a page table of the IOMMU, maps an IO virtual address to a physical machine physical address, the IO virtual address is equal to the virtual machine physical address, and the VMACC directly uses the physical interrupt vector number to deliver interrupt and directly uses a virtual machine physical address configured by the virtual machine program to initiate memory read-write operation to the virtual machine.
Preferably, the VMM _ kernel reads and writes the operating state data of the vmac to implement an online migration operation of the virtual machine using the vmac, specifically:
the VMM _ kernel reads the working state data of the corresponding VMACC on the original physical machine and sends the working state data to the VMM _ kernel of the target physical machine, and the VMM _ kernel of the target physical machine writes the received working state data of the VMACC of the original physical machine so as to realize the online migration of the virtual machine using the VMACC acceleration function.
The beneficial effects of the above technical scheme are that:
(1) the system for improving the virtualization performance disclosed by the invention provides a mode different from the mode of carrying out equipment virtualization by utilizing the traditional mmo, pio and vintr mechanisms, and provides a method for greatly reducing vmexit related to equipment virtualization, thereby further improving the virtualization efficiency and reducing the virtualization performance loss.
(2) The interaction between the VMACC and the virtual machine program can be directly finished without the intervention of a VMM except for the initialization operation, so that when the virtual machine program realizes the corresponding function, vmexit is reduced to a negligible level, a small amount of vmexit can be generated only when VMACC resources are configured, and during the normal operation of the VMACC, the vmexit can be avoided, thereby improving the virtualization efficiency.
(3) The VMACC can realize functions and logic required by the virtual machine, and in the process of accelerating the virtual machine program by the VMACC, the virtual machine program and the VMACC directly finish interaction without vmentry and vmexit, and acceleration can be finished without VMM intervention, so that the performance influence of virtualization equipment on the virtual machine is greatly reduced, and the virtualization efficiency is improved.
(4) The invention can customize the communication data structure and mode between the VMACC and the functional logic of the VMACC according to the specific requirements of the virtual machine program.
(5) The invention uses the online migration function of the virtual machine of the VMACC, expands the flexibility and the maintainability of the VMACC, and in the whole virtualization process, the virtual machine program can directly realize the specific function by using the VMACC without using the resources of the central processing unit, thereby greatly improving the virtualization performance, reducing the virtualization performance loss and expanding the use scene of the virtualization technology.
Detailed Description
The embodiments of the present invention will be described in further detail with reference to the drawings and examples. The following detailed description of the embodiments and the accompanying drawings are provided to illustrate the principles of the invention and are not intended to limit the scope of the invention, which is defined by the claims, i.e., the invention is not limited to the preferred embodiments described.
In the description of the present invention, it is to be noted that, unless otherwise specified, "a plurality" means two or more; the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance; the specific meaning of the above terms in the present invention can be understood as appropriate to those of ordinary skill in the art.
Example 1
As shown in fig. 3, the present embodiment discloses a system (VMACC) for enhancing virtualization performance, which includes a built-in independent memory, a computing device and a network switching device, where the memory is used to store computing data of the computing device, Firmware and data temporarily stored in a virtual machine program, and the memory includes forms of DDR RAM, ROM, NAND Flash, disk storage, etc. (but is not limited thereto); the computing device is used for directly interacting with a virtual machine program running in the central processing unit and running a built-in program of the VMACC to realize functions and logic which need acceleration and are appointed to the virtual machine; the network switching device is used for exchanging data with other devices in the network, and extending the application capability and the scenario of the VMACC, for example, but not limited to, ethernet, Infiniband, fibre channel (fibre channel).
The program running in the VMACC computing device directly interacts with the virtual machine program running in the central processing unit through a computer system bus so as to realize the accelerated virtualization of the virtual machine and realize the functions required by the virtual machine program; computer system buses such as, but not limited to, PCIe bus, internal bus, QPI, front side bus, etc.; the direct interaction comprises the interaction of bus messages such as access through a memory address space, message sending and the like; sending the message includes using an interrupt or setting a signal value at a certain memory address.
(1) The VMACC interacts with the virtual machine program through memory address space access;
the VMACC provides a corresponding interface protocol for the virtual machine program, and a computing device in the VMACC provides an interface of the protocol for the virtual machine program to access through a memory address space on a computer system bus so as to realize direct interaction between the VMACC and the virtual machine program; the VMACC provides the interface of the protocol to the virtual machine program access, e.g., in registers, device memory, etc.
In order to ensure that the virtual machine program correctly accesses the resources corresponding to the VMACC, when the acceleration function provided by the VMACC is initialized, the virtual machine monitor kernel (VMM _ kernel) establishes a primary page table or a secondary page table in the processor (CPU) to satisfy the mapping relationship between the GPA and the HPA, so that the virtual machine program can directly read and write the register and the device memory provided by the VMACC, that is, the virtual machine program can access the HPA where the corresponding resources are located by accessing the GPA, thereby operating (accessing and reading and writing) the interfaces provided by the VMACC, such as the register, the device memory and the like.
The computing device in the VMACC can directly write data required by a virtual machine program into a memory address space of the virtual machine, and the method is specifically realized by the following steps:
the virtual machine program configures a writable virtual machine physical address (GPA) of the computing device in the VMACC, the VMM _ kernel performs address translation, the virtual machine physical address (GPA) is translated into a physical machine physical address (HPA), and the computing device in the VMACC is ensured to be capable of writing the corresponding GPA by using HPA addressing.
Further, in an embodiment, in a system having an IOMMU (input/output memory management unit), the VMM _ kernel can set the IOMMU according to a relationship between the GPA and the HPA when the acceleration function provided by the vmac is initialized, and perform translation from an IO virtual address (IOVA) to an HPA, so that the VMM _ kernel does not need to process the GPA when the virtual machine program configures the GPA, and the computing device in the vmac can directly write the IOVA into the virtual machine memory.
(2) VMACC interacts with virtual machine programs by sending messages
The VMACC can send a message on the computer system bus to inform the virtual machine program to realize direct interaction between the VMACC and the virtual machine program, such as using an interrupt or setting a signal value at a certain memory address; interrupts can be sent, for example, in a direct physical interrupt manner or using an interrupt controller virtualization function to send virtual interrupts for virtual machines (but are not limited to such).
The VMM _ kernel can directly read or write in an interface of a protocol provided by the VMACC, namely resources such as a corresponding register, a device memory and the like, when the VMM initiates the online migration operation of the virtual machine, the VMM _ kernel in the VMM reads the working state data of the VMACC on the original physical machine and sends the working state data to the VMM _ kernel of the target physical machine, and the VMM _ kernel of the target physical machine writes in the received working state data of the VMACC of the original physical machine, so that the online migration of the virtual machine is realized when the VMACC acceleration function is used; the operating state data of the VMACC comprises data in the memory, computing state data in the computing device, and operating state data of the network switching device; the invention uses the online migration function of the virtual machine of the VMACC, so that the VMACC has flexibility and maintainability, and in the whole virtualization process, a virtual machine program can directly realize a specific function by using the VMACC without using the resources of a processor, so that the virtualization performance is greatly improved, the virtualization performance loss is reduced, and the use scene of the virtualization technology is expanded.
The interaction between the VMACC and the virtual machine program can be directly finished without the intervention of a VMM except for the initialization operation, so that when the virtual machine program realizes the corresponding function, vmexit is reduced to a negligible level, a small amount of vmexit can be generated only when VMACC resources are configured, and during the normal operation of the VMACC, the vmexit can be avoided, thereby improving the virtualization efficiency.
The VMACC can realize functions and logic required by the virtual machine, and in the process of accelerating the virtual machine program by the VMACC, the virtual machine program and the VMACC directly finish interaction without vmentry and vmexit, and acceleration can be finished without VMM intervention, so that the performance influence of virtualization equipment on the virtual machine is greatly reduced, and the virtualization efficiency is improved.
The VMACC is, for example, a PCIe device, and directly interacts with a virtual machine program running in the central processor through a PCIe bus; the VMACC is, for example, one or several cores of the central processor, and directly interacts with a virtual machine program running in the central processor through a central processor internal bus, a QPI, or a front-end bus.
The practical effects of the present invention are analyzed below with reference to specific examples:
the VMACC can be used for accelerating functions of a timer, a video codec, an encryption/decryption algorithm, a compression/decompression algorithm and other virtual machine programs needing acceleration;
in one example of the invention, a Timer VMACC _ Timer is realized by using the VMACC, an iperf test tool is used for testing between a physical machine and a virtual machine, and on the premise of achieving the same network throughput bandwidth, compared with the traditional virtual machine, the virtual machine using the VMACC has the advantages that the utilization rate of a CPU in the virtual machine is reduced by 30%, and the delay of receiving and sending network packets is reduced by 7%; in a scenario where timers are frequently used, vmexit of a physical machine in relation to timer virtualization drops from tens of thousands to tens of times per second; the embodiment has great performance improvement aiming at scenes such as IO intensive type, real-time tasks, high-frequency transactions and the like, and the service performance is almost consistent with that of a physical machine; the application scenes of a virtualization technology and a cloud computing architecture are increased; the above example can also be extended to more devices and architectures, further extending the application area of virtualization technology.
The invention can also customize the communication data structure and mode between the VMACC and the functional logic of the VMACC according to the specific requirements of the virtual machine program; for example, in the case of the Timer VMACC _ Timer, a physical machine system with an IOMMU is used, and the process of customizing the data structures, modes, and functional logic of the VMACC and the virtual machine program is as follows:
1) define the data structure, timer _ info, which contains 24 bytes of data: frequency represents timer frequency, 8 bytes, and is read only for the virtual machine program and VMM _ kernel; current _ count represents the counting period value of the timer when counting down currently, 8 bytes are read only for the virtual machine program and the VMM _ kernel; initial _ count represents a count initialization value of start timing set by the virtual machine program, 8 bytes, and can be read and written for the virtual machine program and the VMM _ kernel; wherein frequency is set by VMACC _ Timer, for example, 100 MHz;
2) define data structure, timer _ irq _ info, containing 16 bytes of data: the irq _ vector is used for the VMM _ kernel to configure the VMACC _ Timer to send an interrupted vector number to the central processing unit when the Timer expires, and the vector number is 8 bytes and can be read and written for the VMM _ kernel; the destination _ cpu is used for the VMM _ kernel to configure the VMACC _ Timer to send interrupt to a certain logic core of the central processing unit when the Timer expires, 8 bytes are used for reading and writing the VMM _ kernel;
3) the VMM _ kernel sets the VMACC to provide an acceleration function in a mode of VMACC _ Timer, the VMM _ kernel sets the VMACC to map the Base address of the Timer _ info structure to the HPA address Base _ HAddr1 of the physical machine, and simultaneously define the Base address Base _ GAddr1 of the virtual machine, and establish a page table mapping relationship from Base _ GAddr1 to Base _ HAddr1 in the central processor; the page table mapping relation can be completely established in the initialization process or additionally established when the subsequent page missing exception occurs; through this step, the virtual machine program can directly access the Timer _ info data structure in the VMACC _ Timer;
4) the VMM _ kernel provides a logical data structure VMM _ Timer _ irq _ info for the virtual machine program to configure the virtual CPU and interrupt vector number for which the virtual machine program wants the VMACC _ Timer to send an interrupt, VMM _ Timer _ irq _ info is identical to the structure of the Timer _ irq _ info, except that irq _ vector and destination _ CPU in VMM _ Timer _ irq _ info are readable and writable to the virtual machine program;
5) the VMM _ kernel sets VMM _ timer _ irq _ info initialized according to virtual hardware topology of the virtual machine; set vmm _ timer _ irq _ info _ irq _ vector to VEC _ G, vmm _ timer _ irq _ info _ destination _ cpu to DESTCPU _ G;
the VMM _ kernel sets the timer _ irq _ info according to the physical machine resource planning; setting timer _ irq _ info _ vector to VEC _ H, vmm _ timer _ irq _ info _ destination _ CPU to DESTPUT _ H;
6) the VMM _ kernel configures an IOMMU, maps DESTPUT _ H into DESTPUT _ G, maps VEC _ H into VEC _ G, and when VMACC _ Timer sends an interrupt to DESTPUT _ H by VEC _ H, the VMACC _ Timer is converted into sending the interrupt to DESTPUT _ G of the virtual machine by VEC _ G; the virtual machine program can receive the interrupt of the expiration of the timer; the timer interrupt processing program of the virtual machine program can process the event that the timer expires;
7) VMM _ kernel maps VMM _ timer _ irq _ info to the virtual machine's GPA address Base _ GAddr 2;
optionally, the virtual machine program may access the Base _ GAddr2 for adjusting the virtual CPU and interrupt vector number that the virtual machine program wishes to receive the timer expired interrupt, when the virtual machine program accesses the Base _ GAddr2 and its corresponding GPA address within the VMM _ timer _ irq _ info structure range, a page fault exception occurs, and the VMM _ kernel acquires the VMM _ timer _ irq _ info value that the virtual machine program wishes to configure, for example, VMM _ timer _ irq _ info, irq _ vector is VEC _ G2, VMM _ timer _ irq _ info, destination _ CPU is DEST _ CPUG 2; the VMM _ kernel configures an IOMMU, maps DESTPUT _ H to DESTPUT _ G2, maps VEC _ H to VEC _ G2, and when VMACC _ Timer sends an interrupt to DESTPUT _ H through VEC _ H, the VMACC _ Timer is converted into sending an interrupt to DESTPUT _ G2 of the virtual machine through VEC _ G2;
8) in the timer _ info structure, reading a frequency value by a virtual machine program, and calculating a count value C required to be set according to a required target expiration time T, wherein C = T/frequency; the virtual machine program sets initial _ count as C, and VMACC _ Timer starts to work regularly;
9) the virtual machine program can obtain the current remaining count value of the timer by reading the current _ count value and obtain the last set count initialization value by reading the initial _ count value;
10) after receiving a new initial _ count setting, VMACC _ Timer sets a current _ count value as initial _ count, namely, current _ count is periodically decremented by 1 according to frequency, when current _ count is decremented to 0, timing operation is stopped, namely, current _ count is periodically decremented by 1, and according to the set Timer _ irq _ info, interrupt is sent to a destination _ cpu logic core corresponding to the central processing unit by irq _ vector; according to the setting of the VMM _ kernel to the IOMMU, the virtual machine receives the Timer expiration interrupt sent by the VMACC _ Timer, and the Timer expiration interrupt processing function of the virtual machine program can process the Timer event.
Example 2
As shown in fig. 4, the present invention provides a physical machine, which includes a processor (CPU), a memory, a VMM _ kernel, and the VMACC provided in embodiment 1, where the VMACC can be accessed to a PCIE slot of the physical machine as a PCIE device, for example; processors such as ARM, PowerPC, MIPS, etc.; the VMM _ kernel directly runs in the physical machine, manages all hardware resources, and simultaneously runs a plurality of VMM _ user processes, wherein the VMM _ user processes run virtual machines VM1, VM2 to VMn, and virtual machine programs (Guest) run in the virtual machines; the physical machine adopts a modern computer architecture, such as a PC, a server, a small computer, a notebook computer, a vehicle computer and the like.
Further, in one embodiment, an IOMMU is included in the physical machine for more flexibly configuring memory access and interrupt message delivery for the VMACC.
The VMM _ kernel initializes the setting of the VMACC, establishes a primary page table or a secondary page table in a processor (CPU) and meets the mapping relation from GPA to HPA, so that a virtual machine program can directly read and write a register and a device memory provided by the VMACC, and only a small amount of vmexit caused by page fault exception can appear in the whole reading and writing process.
As shown in fig. 5, in a physical machine hardware environment without an IOMMU or with an IOMMU turned off, the VMM _ kernel sets parameters of the vmac in cooperation with a virtual machine program, so that the vmac directly initiates a memory read-write operation to a GPA memory address corresponding to the virtual machine with an HPA memory address; the VMM _ kernel negotiates an interrupt vector number with the virtual machine program at the same time, and configures the VMACC to send an interrupt message to the corresponding interrupt vector number.
As shown in fig. 6, in the physical machine environment with an IOMMU, the VMM _ kernel configures an interrupt translation table of the IOMMU, maps a physical interrupt vector number of the VMACC to an interrupt vector number set by the virtual machine program, and configures a page table of the IOMMU to map the IOVA to the HPA.
Note 1 in fig. 5 and 6 is that a read/write is initiated on the memory address bus using a GPA address, and the CPU translates the GPA into HPA by setting a page table, which is used for the following functions:
accessing registers and device memory of the VMACC to invoke the VMACC, an acceleration function;
accessing data interacted with the VMACC in the memory; note 2 is a working state request to read and write the VMACC to implement online migration of the virtual machine; and reading and writing the register and the device memory of the VMACC to configure the working state data of the VMACC and the corresponding interrupt vector number and address of the virtual machine.
Aiming at the functions provided by the VMACC, the VMM _ kernel performs online migration operation of the accelerated virtual machine by reading and writing the working state data of the corresponding VMACC, wherein the working state data of the VMACC comprises data in a memory, calculation state data in a calculation device and the working state data of a network exchange device; the read-write mode is, for example, reading through a register and a device memory; when the VMM initiates the online migration operation of the virtual machine, the VMM _ kernel in the VMM reads the working state data of the corresponding VMACC on the original physical machine and sends the working state data to the VMM _ kernel of the target physical machine, and the VMM _ kernel of the target physical machine writes the working state data of the VMACC of the original physical machine, so that the online migration of the virtual machine is realized when the VMACC acceleration function is used.
While the invention has been described with reference to a preferred embodiment, various modifications may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In particular, the technical features mentioned in the embodiments can be combined in any way as long as there is no structural conflict. It is intended that the invention not be limited to the particular embodiments disclosed, but that the invention will include all embodiments falling within the scope of the appended claims. The invention has not been described in detail and is part of the common general knowledge of a person skilled in the art.