WO2017121273A1

WO2017121273A1 - Method and device for processing i/o request under kvm virtualization

Info

Publication number: WO2017121273A1
Application number: PCT/CN2017/070151
Authority: WO
Inventors: 张献涛; 丁圣阁; 田双太
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2016-01-12
Filing date: 2017-01-04
Publication date: 2017-07-20
Also published as: CN106970821A; CN106970821B

Abstract

Provided are a method and device for processing an I/O request under KVM virtualization. The method comprises: intercepting, by means of a kernel-based virtual machine (KVM), an I/O request transmitted by a virtual machine, and determining information of an I/O channel corresponding to the I/O request, wherein the I/O channel has a corresponding buffer zone, and the information of the I/O channel includes a file descriptor for an event notification (eventfd) (101); writing the I/O request into the buffer zone of the corresponding I/O channel (102); using the eventfd to notify the I/O channel of a corresponding user mode process (103); and in the user mode process, monitoring, according to the eventfd, an event of the corresponding I/O channel, acquiring the I/O request from the buffer zone of the corresponding I/O channel, and executing I/O simulation on the I/O request (104), wherein the user mode process includes a first device simulation process and a second device simulation process that is independent from the first device simulation process. The method can realize distribution of I/O requests in a kernel mode, and can support multiple device simulation processes for a single virtual machine, thereby improving an I/O simulation efficiency.

Description

Method and device for processing I/O request under KVM virtualization

The present application claims priority to Chinese Patent Application No. 201610019289.0, entitled "A Method and Apparatus for Processing I/O Requests under KVM Virtualization", filed on January 12, 2016, the entire contents of which are incorporated by reference. In this application.

Technical field

The present application relates to the technical field of KVM virtualization, and in particular to a method for processing an I/O request under KVM virtualization and a device for processing an I/O request under KVM virtualization.

Background technique

Virtual Machine is a complete computer system that is simulated by software and has complete hardware system functions and runs in a completely isolated environment. With virtual machine software, one or more virtual computers can be simulated on a single physical computer. Virtual machines can work like real computers, such as installing an operating system, installing applications, accessing network resources, and more. Usually, the physical computer where the virtual machine is located is called the host, and the virtual machine itself is called the virtual machine (Guest).

KVM (Kernel-based Virtual Machine) is an open source system virtualization module that uses Linux's own scheduler for management. KVM virtualization requires hardware support (such as Intel VT technology or AMD). VT technology).

In the prior art, in the KVM virtualization, when the virtual machine generates I/O operations, it supports VMX (Virtual Machine eXtensions, which is an x86 instruction set virtualization extension implemented by Intel) or SVM (Secure Virtual Machine, which is an amd implementation). The virtualization extension of the x86 instruction set) the extended CPU will stop executing, exit the non-root mode, and enter the root mode to execute the KVM code. KVM knows that the virtual machine needs to perform I/O operations according to the reason of the exit, so it exits from the kernel state to the QEMU process in the user state.

The QEMU process maps a KVM space during initialization. The KVM puts the I/O request in it before exiting. After the QEMU gets the I/O request, it distributes it to the corresponding I/O analog device, I/O. After the analog device performs the I/O simulation, QEMU calls the IOCTL to enter the KVM kernel state. After the KVM is slightly processed, it enters the non-root mode and continues to run the vCPU (virtual processor).

However, the inventor found the following problems in the simulation of implementing the above I/O request:

1. When simulating vCPU, QEMU enters the KVM kernel state through a vCPU IOCTL. When an I/O request occurs, it must exit the kernel state and return to the VCPU thread of the QEMU user state for simulation. Therefore, the I/O simulation must be CPU emulation is in the same process and cannot support multiple device emulation processes.

2, KVM's existing eventfd-based I / O notification mechanism can only be applied to write requests, and does not care about the data written, only concerned with the scene of the written address, such as the Virtio device's queue notification register write access. This mechanism does not pass the actual address and length of the I/O request access, nor can it pass the data from the read operation to the KVM.

3. If you want to implement CPU emulation and device I/O emulation in different threads, there is currently no mechanism to synchronize IO emulation threads and VCPU threads.

Summary of the invention

In view of the above problems, embodiments of the present application have been proposed in order to provide a method for processing I/O requests under KVM virtualization that overcomes the above problems or at least partially solves the above problems, and a corresponding KVM virtualization processing I/ O request device.

In order to solve the above problem, the present application discloses a method for processing an I/O request under KVM virtualization, the method comprising:

The I/O request sent by the virtual machine is intercepted by the kernel virtual machine KVM, and the information of the I/O channel corresponding to the I/O request is determined, wherein the I/O channel has a corresponding buffer, and the I/O The information of the O channel includes an eventfd file descriptor;

Writing the I/O request into a buffer of a corresponding I/O channel;

Notifying the user state process corresponding to the I/O channel by using the eventfd;

In the user state process, according to the eventfd, the event of the corresponding I/O channel is monitored, and an I/O request is obtained from the buffer of the corresponding I/O channel, and the I/O request is executed. An I/O simulation, wherein the user state process includes a first device emulation process and a second device emulation process independent of the first device emulation process.

Preferably, before the step of intercepting the I/O request sent by the virtual machine by the kernel virtual machine KVM and determining the information of the I/O channel corresponding to the I/O request, the method further includes:

Creating a virtual machine by using the first device simulation process to obtain a file descriptor of the virtual machine;

Using the first device simulation process, creating a virtual processor vCPU based on the file descriptor of the virtual machine, and obtaining a file descriptor of the vCPU;

Using the second device simulation process to create an eventfd file descriptor;

Using the second device simulation process to create an I/O channel based on the file descriptor of the virtual machine, and Binding the I/O channel to the eventfd file descriptor to obtain a file descriptor of the I/O channel;

Using the second device simulation process to monitor the eventfd file descriptor;

Using the second device simulation process to map a buffer of the I/O channel for the user state process based on a file descriptor of the I/O channel;

Using the second device emulation process, an I/O address interval is bound to the I/O channel based on a file descriptor of the I/O channel.

Preferably, the I/O request includes an I/O address, and the step of intercepting an I/O request sent by the virtual machine by the kernel virtual machine KVM, and determining information of the I/O channel corresponding to the I/O request include:

Intercepting I/O requests from virtual machine vCPU threads through kernel virtual machine KVM and Intel VT-d hardware technology;

Extracting a corresponding I/O address from the I/O request;

Determining an I/O address interval to which the I/O address belongs;

And determining, by the I/O address interval, a corresponding I/O channel, and acquiring information of the I/O channel corresponding to the I/O channel.

Preferably, the address interval of the I/O channel includes an address subinterval of a plurality of I/O analog devices;

Determining, in the user state process, a corresponding I/O channel according to the eventfd, and acquiring an I/O request from a buffer of the corresponding I/O channel, performing the I/O request The steps of the I/O simulation include:

In the user state process, determining a corresponding I/O channel according to the eventfd;

Obtaining an I/O request from a buffer of the corresponding I/O channel;

Determining a corresponding I/O analog device according to the I/O address in the I/O request;

Distributing the I/O request to the I/O analog device, and performing I/O simulation on the I/O request by the I/O analog device.

Preferably, in the user state process, determining a corresponding I/O channel according to the eventfd, and acquiring an I/O request from the buffer of the corresponding I/O channel, for the I Before /O requests to perform the I/O simulation steps, it also includes:

Suspend the vCPU thread.

Preferably, in the user state process, determining a corresponding I/O channel according to the eventfd, and acquiring an I/O request from the buffer of the corresponding I/O channel, for the I After the /O request to perform the I/O simulation steps, it also includes:

Wake up the suspended vCPU thread.

Preferably, the I/O request includes a read request, and the I/O request is distributed to the I/O analog device, and the I/O request is performed by the I/O analog device. The steps of the /O simulation are:

And distributing the read request to the I/O analog device, and writing, by the I/O analog device, data corresponding to the read request into a buffer of the I/O channel.

Preferably, the method further includes:

If the I/O request is a read request, the data corresponding to the read request is read out from the buffer, and is read into a corresponding member variable in the vCPU structure;

Corresponding simulation processing is performed on the data.

The present application also discloses an apparatus for processing an I/O request under KVM virtualization, the apparatus comprising:

An I/O channel determining module, configured to intercept, by the kernel virtual machine KVM, an I/O request sent by the virtual machine to determine information of an I/O channel corresponding to the I/O request, where the I/O channel has Corresponding buffer, the information of the I/O channel includes an eventfd file descriptor;

a buffer write module, configured to write the I/O request into a buffer of a corresponding I/O channel;

An event notification module, configured to notify the user state process corresponding to the I/O channel by using the eventfd;

An I/O simulation module, configured to: in the user state process, listen to an event of the corresponding I/O channel according to the eventfd, and obtain an I/O request from a buffer of the corresponding I/O channel, Performing an I/O simulation on the I/O request, wherein the user state process includes a first device emulation process and a second device emulation process independent of the first device emulation process.

Preferably, the device further comprises:

a virtual machine creating module, configured to create a virtual machine by using the first device simulation process, and obtain a file descriptor of the virtual machine;

a vCPU creation module, configured to: use the first device simulation process, create a virtual processor vCPU based on a file descriptor of the virtual machine, and obtain a file descriptor of the vCPU;

An eventfd creation module is configured to create an eventfd file descriptor by using the second device simulation process;

An I/O channel creation module, configured to simulate a process by using the second device, create an I/O channel based on a file descriptor of the virtual machine, and bind the I/O channel to the eventfd file descriptor Determining, obtaining a file descriptor of the I/O channel;

a monitoring module, configured to monitor the eventfd file descriptor by using the second device simulation process;

a buffer mapping module, configured to simulate the process by using the second device, based on a file description of the I/O channel a buffer for mapping the I/O channel to the user mode process;

An address interval binding module is configured to bind an I/O address interval to the I/O channel based on a file descriptor of the I/O channel by using the second device simulation process.

Preferably, the I/O request includes an I/O address, and the I/O channel determining module includes:

An I/O request intercepting submodule for intercepting I/O requests from a virtual machine's vCPU thread through a kernel virtual machine KVM and an Intel VT-d hardware technology;

An I/O address extraction submodule, configured to extract a corresponding I/O address from the I/O request;

a first I/O channel determining submodule, configured to determine an I/O address interval to which the I/O address belongs, and determine a corresponding I/O channel by using the I/O address interval, and acquire the I/ Information about the I/O channel corresponding to the O channel.

The I/O simulation module includes:

a second I/O channel determining submodule, configured to determine, according to the eventfd, a corresponding I/O channel in the user state process;

An I/O request reading submodule, configured to obtain an I/O request from a buffer of the corresponding I/O channel;

An I/O analog device determining submodule, configured to determine a corresponding I/O analog device according to an I/O address in the I/O request;

And a distribution submodule, configured to distribute the I/O request to the I/O simulation device, and perform I/O simulation on the I/O request by using the I/O simulation device.

Preferably, the device further comprises:

Suspending a module, for suspending the vCPU thread after the userfest process corresponding to the I/O channel is notified by using the eventfd.

Preferably, the device further comprises:

And a wake-up module, configured to wake up the suspended vCPU thread after performing I/O simulation on the I/O request.

Preferably, the I/O request includes a read request, and the distribution sub-module is further configured to:

Preferably, the device further comprises:

a data reading module, configured to: when the I/O request is a read request, read data corresponding to the read request from a buffer, and read the corresponding member variable into the vCPU structure, The data is subjected to corresponding analog processing.

Embodiments of the present application include the following advantages:

In the embodiment of the present application, the I/O request can be delivered through the I/O channel. After the KVM intercepts the I/O request sent by the virtual machine, the I/O channel corresponding to the I/O request can be determined, and the I/O channel is determined. /O request is written into the buffer of the corresponding I/O channel, and then the user state process corresponding to the I/O channel is notified by eventfd, and the user state process listens to the eventfd event notification, and the I/O channel corresponding to the eventfd Get an I/O request in the buffer and perform an I/O simulation on the I/O request. The application can realize the distribution of I/O requests in the kernel state, and can support multiple device simulation processes for a single virtual machine, thereby improving the efficiency of I/O simulation.

DRAWINGS

1 is a flow chart showing the steps of Embodiment 1 of a method for distributing an I/O request under KVM virtualization according to the present application;

2 is a flow chart of steps of a second embodiment of a method for distributing an I/O request in a KVM virtualization process according to the present application;

3 is a flow chart of steps of a KVM initialization process in Embodiment 2 of a method for distributing an I/O request in a KVM virtualization process according to the present application;

4 is a structural block diagram of an embodiment of an apparatus for processing an I/O request under the KVM virtualization of the present application.

detailed description

The above described objects, features and advantages of the present application will become more apparent and understood.

Referring to FIG. 1 , a flow chart of a first embodiment of a method for distributing an I/O request in a KVM virtualization process of the present application is shown, which may specifically include the following steps:

Step 101: The I/O request sent by the virtual machine is intercepted by the kernel virtual machine KVM, and the information of the I/O channel corresponding to the I/O request is determined, where the I/O channel has a corresponding buffer. The information of the I/O channel includes an event notification to create an eventfd file descriptor.

Step 102: Write the I/O request into a buffer of a corresponding I/O channel.

Step 103: The eventfd is used to notify a user state process corresponding to the I/O channel.

Step 104: In the user state process, according to the eventfd, the event of the corresponding I/O channel is monitored, and an I/O request is obtained from the buffer of the corresponding I/O channel, and the I/O request is performed. O requests to perform an I/O simulation.

The user state process includes a first device simulation process and a second device simulation process that is independent of the first device simulation process.

In the embodiment of the present application, an I/O request can be delivered through an I/O channel, when the KVM intercepts the virtual machine. After the I/O request, the I/O channel corresponding to the I/O request can be determined, and the I/O request is written into the buffer of the corresponding I/O channel, and then the user corresponding to the I/O channel is notified by eventfd. The state process, after the user state process listens to the eventfd event notification, reads the I/O request from the buffer of the I/O channel corresponding to the eventfd, and performs I/O simulation on the I/O request. The application can realize the distribution of I/O requests in the kernel state, and can support multiple device simulation processes for a single virtual machine, thereby improving the efficiency of I/O simulation.

Referring to FIG. 2, a flow chart of steps of a second embodiment of a method for processing an I/O request under KVM virtualization according to the present application is shown. Among them, KVM (Kernel-based Virtual Machine) is an open source virtualization solution based on Linux environment. The idea of KVM is to add a virtual machine management module based on the Linux kernel, and reuse the well-established process scheduling, memory management, I/O management and other parts of the Linux kernel to make it a hypervisor that can support the running of virtual machines. .

KVM is a fully virtualized solution based on hardware-assisted virtualization technology (eg Intel's VT-x or AMD-V). The virtual machine operating system can be run directly in the KVM virtual machine without modification. Each virtual machine Ability to enjoy independent virtual hardware resources: network cards, disks, graphics adapters, and more.

KVM can be compiled into the kernel as a separate module, or it can be loaded as a kernel module after the Linux system is booted.

The KVM API is accessed via the /dev/kvm device. /dev/kvm is a character device that can be manipulated using common system calls such as open, close, IOCTL, and so on. Because the implementation function of the KVM character device does not include write, read, etc., all operations on the KVM are implemented by the IOCTL sending the corresponding control word.

The user space API provided by KVM is functionally divided into three types, namely system instruction, VM instruction, and vCPU instruction. The system instruction is for the global parameter setting of the virtualization system and the control operation for the virtual machine creation; the VM instruction controls the specific VM virtual machine, such as performing memory setting, creating a vCPU, etc.; the vCPU instruction is performed for a specific vCPU. Parameter setting (MRU register read/write, interrupt control, etc.).

In an embodiment of the invention, user space can be controlled by different user mode simulators, so that device I/Os of the same virtual machine can be simulated in different user state processes, and multiple device simulation processes for a single virtual machine can be supported. . The different user mode process may include a first device simulation process and a second device simulation process. As a preferred example of the embodiment of the present application, the first device simulation process may be a QEMU (full name Quick Emulator) process, and the second The device simulation process can simulate a process for an I/O device that is independent of the QEMU process. To facilitate subsequent description, the second device simulation process can be referred to as a DEMU (Dedicated Emulator) process or a DEMU.

Among them, QEMU is an open source simulator and virtual machine monitor (VMM), users can install QEMU through the package manager of different Linux distributions. QEMU simulates the hardware device of the virtual machine in the host user mode. When QEMU is used as the system simulator, it can simulate a virtual machine that can run the operating system independently. Each virtual machine corresponds to a QEMU process in the host (Host), and virtual The vCPU of the machine corresponds to one thread of the QEMU process.

In a specific implementation, the QEMU process and the DEMU process can be created by executing the binary executable files of QEMU and DEMU, and the two processes can be started by a script or a libvirt tool.

Applied to the embodiment of the present application, the initialization process is first performed before the I/O request is processed. In the platform-related KVM module, the module_init macro officially enters the initialization phase of the KVM, and performs related hardware initialization preparation. Enter the kvm_init function in kvm_main.c for formal initialization.

Referring to the flowchart of the steps of the KVM initialization process of FIG. 3, the initialization process may at least include the following steps:

Sub-step S1: Create a virtual machine by using the first device simulation process to obtain a file descriptor of the virtual machine.

Specifically, the interaction interface between the KVM and the user space process is /dev/kvm, and the operation of the KVM API is started by opening the /dev/kvm device file. After using the system call open method, an fd for the kvm subsystem can be obtained. File descriptor. Further operations on the file descriptor are then performed by the IOCTL system instructions.

The IOCTL system instructions can include the KVM_CREATE_VM instruction, which creates a virtual machine VM with the KVM_CREATE_VM instruction. An fd file descriptor can be obtained by calling anon_inode_getfd as a file descriptor of the virtual machine, ie vm_fd, which points to the newly created virtual machine in the kernel space, and then controls the behavior of the virtual machine according to the descriptor.

The vm IOCTL system call implements control of the virtual machine. Most of the parameters of the vm IOCTL control instruction need to be operated from the vm_fd returned by KVM_CREATE_VM. The operations involved are mainly for the virtual machine, such as configuring memory and configuring vCPU.

For the file operation of the VM, two operation functions, IOCTL and mmap, are provided, wherein mmap corresponds to the physical address of the guest operating system GUEST OS, and the address space of the GUEST OS can be read/written directly, and the IOCTL is used to send the KVM. Control word.

In the embodiment of the present application, a corresponding universal unique identifier UUID may be set for the created virtual machine to identify the virtual machine by using the UUID.

The creation process of the virtual machine is essentially the creation and initialization process of the KVM structure. When creating the virtual machine, the kvm_dev_IOCTL_create_vm function creates the KVM structure by calling the kvm_create_vm function.

The KVM structure saves the context and other related states of the virtual machine running. For example, it can include vCPU (virtual processor, CPU in virtual machine), memory, APIC (Advanced Programmable Interrupt Controller), IRQ. The information of the (Interrupt Request), the MMU (Memory Management Unit), and the Event Event Management is applied to the embodiment of the present application. The KVM structure may further include a UUID of the virtual machine.

The initialization process of the KVM structure may include: initializing a KVM memslot structure, a Bus bus structure information, a scru read/write lock information, an eventfd event notification information, an mmu memory management structure information, and the like.

After the virtual machine's KVM structure is created, it can be added to the global linked list vm_list.

In the embodiment of the present application, the creation of a virtual machine may be completed by using a QEMU process.

Step S2: Using the first device simulation process, create a virtual processor vCPU based on the file descriptor of the virtual machine, and obtain a file descriptor of the vCPU.

Specifically, the newly created virtual machine does not have a vCPU and needs to be configured through subsequent IOCTL instructions. After obtaining vm_fd, the KVM_CREATE_vCPU instruction is invoked by the IOCTL, and the vCPU can be created for the virtual machine corresponding to the vm_fd. The entry function address is in the kvm_vm_IOCTL function. After the switch, the program flow is selected to enter the kvm_vm_IOCTL_create_vCPU function for processing, and the vCPU is returned. The file descriptor fd, vCPU_fd, in the KVM virtualization environment, hardware virtualization uses vCPU_fd to describe the vCPU.

In KVM, the data structure corresponding to vCPU is kvm_vCPU. The creation process of vCPU is essentially the creation and initialization process of kvm_vCPU structure. It can include the following process: call kvm_arch_vCPU_create function to create a kvm_vCPU structure; call kvm_arch_vCPU_setup function to data in kvm_vCPU The structure is initialized; the file descriptor vCPU_fd corresponding to the current vCPU is created, and the kvm_vCPU is added to the vCPU array of the KVM.

In the embodiment of the present application, the creation of the vCPU may be completed by using a QEMU process, and each vCPU corresponds to one vCPU thread.

Step S3, using the second device simulation process to create an eventfd file descriptor.

Applied to the embodiment of the present application, the DEMF process can call the standard Linux system API eventfd() to create an eventfd file descriptor. Specifically, eventfd() is the system interface provided by the Linux system. The function prototype is int eventfd (unsigned int initval, int flags). Calling this function user state process can obtain an eventfd file descriptor and create an eventfd in the kernel. The mechanism used by the object to implement wait and notification is mainly used for user state mutual notification and notification of kernel mode to user mode.

The eventfd object can contain a 64-bit integer variable as a counter, the parameters passed in the function prototype. Initval is used to initialize this counter. The eventfd file descriptor is used to reference the eventfd object in user mode. The user state process can perform read, write, poll, and select operations on the eventfd file descriptor.

Step S4, using the second device simulation process, creating an I/O channel based on the virtual machine file descriptor, and binding the I/O channel with the eventfd file descriptor to obtain the I/ The file descriptor of the O channel.

Applied to the embodiment of the present application, the DEMU process may also be used to create one or more I/O channels of the virtual machine.

Specifically, the DEMU process can obtain the UUID of the current virtual machine, and then find the UUID of the virtual machine in the global linked list vm_list. If the search succeeds, the corresponding KVM structure is obtained, thereby obtaining the file descriptor vm_fd of the virtual machine, and then An I/O channel can be created for the virtual machine corresponding to the vm_fd by the new IOCTL instruction (for example, the KVM_CREATE_IOCH instruction). When the I/O channel is created, the binding relationship between the I/O channel and eventfd is generated, and then the file descriptor of the I/O channel is returned, that is, I/O_fd, and the file descriptor of the I/O channel provides the IOCLT instruction to the user. State processes, each vm_fd can mount multiple I / O_fd.

In KVM, the creation process of the I/O channel is essentially the process of creating and initializing the structure of the I/O channel. The structure of the I/O channel may include: an eventfd for notification of the kernel state to the user state; The group stores the buffer of the I/O request and its protection lock; a linked list that holds all the I/O address spaces that this channel is responsible for.

Applicable to the embodiment of the present application, in the virtual environment, the KVM structure corresponding to each virtual machine may further add a linked list for storing all I/O channels registered on the virtual machine.

It should be noted that each I/O simulation process only needs one and corresponds to one I/O channel.

Sub-step S5, monitoring the eventfd file descriptor by using the second device simulation process.

In a specific implementation, the poll or select method may be used on the eventfd file descriptor in the process main loop. Once the eventfd file descriptor is found to have an I/O to be processed, the relevant processing function is called to implement the monitoring of the eventfd.

Step S6: The second device emulation process is used to map a buffer of the I/O channel for the user state process based on a file descriptor of the I/O channel.

In a specific implementation, KVM allocates a buffer to the newly created I/O channel in the kernel state. Subsequently, the user mode program can use the mmap() system call on the file descriptor of the I/O channel to map the I/O. The buffer of the channel, which can be accessed by user-mode processes.

The buffer also has a protection lock to ensure mutual access to the buffer for read and write access.

Step S7, using the second device simulation process, binding an I/O address interval to the I/O channel based on a file descriptor of the I/O channel.

Applying the embodiment of the present application, the DEMU process can also use the file descriptor of the I/O channel to monitor The I/O channel is bound to the I/O address range. Different I/O channels have different I/O address ranges. For example, for I/O address range 1 to 1000, you can bind 1 to 300 to I/. O channel A, binds 301 to 1000 to I/O channel B.

In a specific implementation, the address range of the I/O channel may further include multiple address sub-intervals, each of which corresponds to an I/O analog device (an IO simulation process can simulate multiple IO analog devices), each I/O analog devices only simulate requests that fall within their address range. For example, the address range of I/O channel A is 1 to 300, where 1 to 100 can correspond to I/O analog device 1, 101 to 200 can correspond to I/O analog device 2, and 201 to 300 can correspond to I/O simulation. Device 3.

In the embodiment of the present application, on the basis of the original QEMU process, an independent DEMU process is added, and the two function as a user state simulator can share a part of the underlying data structure of the virtual machine.

After the initialization process is completed, as shown in FIG. 2, the embodiment of the present application may specifically include the following steps:

Step 201: The I/O request sent by the virtual processor vCPU thread of the virtual machine is intercepted by the kernel virtual machine KVM, and the information of the I/O channel corresponding to the I/O request is determined.

The KVM can intercept the virtual machine's I/O operations and determine the corresponding I/O channel information based on the intercepted I/O operations.

In a preferred embodiment of the embodiment of the present application, step 201 may include the following sub-steps:

Sub-step 1: Intercept the I/O request from the virtual machine's virtual processor vCPU thread through the kernel virtual machine KVM and Intel VT-d hardware technology.

In a specific implementation, KVM runs a vCPU thread in kernel space non-root mode. When the vCPU thread initiates an I/O request, the sensitive instruction will trigger the VM exit operation of the Intel VT-d hardware technology, and enter the KVM corresponding to the root mode. Processing function.

Specifically, when the virtual machine is running (the virtual machine is running through the KVM_RUN instruction of the /dev/kvm device IOCTL vCPU interface), the VM Entry is triggered, and the virtual machine enters the guest guest mode, that is, the non-root mode. When a privileged instruction or an external event, such as an I/O access, an operation of a control register, an MSR read/write packet arrives, etc., is executed in a virtual machine (Guest mode), a VM Exit can be triggered (triggered by hardware). The current CPU exits from the guest mode (non-root non-root mode), switches to the root root mode, and the control of the current CPU is transferred to the KVM, which is processed by the KVM.

After the KVM obtains control, the VM_EXIT_REASON field in the VMCS is read to obtain the cause of the VM Exit. The KVM knows that the virtual machine needs to perform I/O operations according to the reason of the exit.

Sub-step 2: Extract the corresponding I/O address from the I/O request.

The I/O request may carry the address information of the virtual machine that issues the I/O request, and after obtaining the I/O request, the corresponding I/O address may be extracted from the I/O request.

Sub-step 3: Determine the I/O address range to which the I/O address belongs.

Sub-step 4: determining a corresponding I/O channel by using the I/O address interval, and acquiring information of an I/O channel corresponding to the I/O channel.

Since the I/O channel has an I/O address interval, it can first determine which address interval the I/O address of the I/O request belongs to, and then determine the corresponding I/O channel according to the matched address interval, and determine the I/. Information about the buffer corresponding to the O channel, the eventfd file descriptor, and so on.

Step 202: Write the I/O request into a buffer of a corresponding I/O channel.

After determining the I/O channel channel corresponding to the I/O request, the KVM can write the I/O request into the buffer of the corresponding I/O channel, thereby completing the distribution of the I/O request in the KVM.

Step 203: The eventfd is used to notify a user state process corresponding to the I/O channel.

In the specific implementation, when the I/O request is written into the buffer of the corresponding I/O channel in the kernel space, the user space does not know about this, so the kernel space needs to send a notification to the user space through eventfd, the user state process. By listening to the eventfd, it is known that an I/O request is written in the buffer of its corresponding I/O channel.

Step 204, suspending the vCPU thread.

Applicable to the embodiment of the present application, since the I/O request is uploaded to the buffer of the I/O channel through the I/O channel, the I/O simulation thread performs I/O simulation, and the I/O simulation thread and the vCPU thread can It is in a different process, so there is no need to exit the vCPU thread when an I/O request occurs. At this time, only the vCPU thread needs to be suspended, and wait for the user state process to complete the I/O request and then wake it up.

Step 205: In the user state process, according to the eventfd, the event of the corresponding I/O channel is monitored, and an I/O request is obtained from the buffer of the corresponding I/O channel, and the I/O request is performed. O requests to perform an I/O simulation.

When the kernel state issues an eventfd event notification to the user state, the user state process can listen to the eventfd event notification, and the corresponding processing function is called to perform I/O simulation through the user state process, wherein the I/O simulation mainly refers to The I/O operation performed by the virtual machine obtains a corresponding processing result through the simulation behavior of the software.

In a preferred embodiment of the embodiment of the present application, step 205 may include the following sub-steps:

Sub-step 1: In the user state process, the corresponding I/O channel is determined according to the eventfd.

Since the eventfd is bound to the I/O channel, after the user state process listens to the function registered by the eventfd process, the I/O channel corresponding to the eventfd can be determined according to the correspondence between the eventfd and the I/O channel.

Sub-step 2: Obtain an I/O request from the buffer of the corresponding I/O channel.

After determining the I/O channel, the I/O request can be further read from the buffer of the I/O channel that has been mapped.

Sub-step 3: Determine the corresponding I/O analog device according to the I/O address in the I/O request.

Sub-step 4: Distributing the I/O request to the I/O analog device, and performing I/O simulation on the I/O request by the I/O analog device.

Since the address space of the I/O channel includes an address subinterval of one or more I/O analog devices, after the user state process obtains the I/O request, it can be based on the I/O address carried in the I/O request. Determining an address subinterval to which the I/O address belongs, and determining a corresponding I/O analog device according to the address subinterval, and distributing the I/O request to the corresponding I/O analog device to complete I/ Further secondary distribution of O requests.

After the I/O analog device obtains the I/O request, it performs I/O simulation on the I/O request.

In an implementation manner, if the I/O request is a read request, the I/O emulation device corresponding to the request may write the data corresponding to the read request into a buffer of the I/O channel, thereby completing the user space. I/O simulation in .

In another implementation manner, if the I/O request is a write request, the I/O emulation device corresponding to the request may write the data corresponding to the write request into a corresponding location, for example, written in the host. An image file that completes the I/O simulation in user space.

Step 206, waking up the suspended vCPU thread.

After the I/O simulation is completed, the user state process calls the new IOCTL instruction in the file descriptor of the I/O channel to enter the KVM kernel state, wake up the suspended vCPU thread, and continue to run the vCPU thread to reach the vCPU thread and I/. O simulates the synchronization of threads, that is, the notification of the user state to the kernel state is achieved by suspending and waking up the vCPU thread.

In an embodiment, if the I/O request is a read request, after the vCPU thread is woken up, the data corresponding to the read request may be read out from the buffer and read into a corresponding member variable in the vCPU structure. In the middle, and through the original I / O simulation code for subsequent simulation processing.

The use of eventfd combined with the use of the buffer, can pass the actual address and length of the I / O request access in the buffer, and pass the data obtained by the read operation to the KVM through the buffer, overcoming the shortcomings of the prior art.

After the processing of the I/O request is completed, the VM Enter can be triggered by a hardware instruction (such as VMLAUNCH), and the guest mode is re-entered to enter the virtual machine running environment to continue running.

In order to enable those skilled in the art to better understand the embodiments of the present application, the following The application examples are exemplified, but it should be understood that the embodiments of the present application are not limited thereto.

While the virtual machine is running, the device driver accesses the MMIO control register of the virtual VGA device to read the state of the analog device. The register-mapped address access triggers an EPT page fault, and the CPU exits the non-root mode into root mode and executes the KVM code.

KVM judges the reason for the exit to be an I/O request, finds the corresponding I/O channel according to the I/O request, copies the I/O request to the corresponding buffer of the I/O channel, and then sends a notification to the vCPU thread with eventfd. Go to sleep.

The QEMU main loop finds the event notification in eventfd, finds the corresponding I/O channel and reads the I/O request from the corresponding buffer, and finds that the request address is in the BAR1 interval of the virtual VGA device in the client physical address layout, so Call the corresponding processing function of the VGA device to simulate the I/O request, write the value of this register back to the buffer, and then wake up the sleeping vCPU thread through the IOCTL instruction.

After the vCPU thread wakes up, it reads the register value data from the buffer and updates it to the corresponding location of the vCPU context, then enters the non-root mode to continue executing the vCPU context.

The read register operation at the driver code completes the simulation.

The embodiment of the present application provides a basis for implementing a device simulation program independent of QEMU under KVM virtualization, and solves the device simulation requirements unique to some vendors.

In addition, the embodiment of the present application implements the distribution of I/O requests in the KVM kernel module, and provides a control interface to the user state, so that the device I/O of the same virtual machine can be simulated in different processes.

It should be noted that, for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the embodiments of the present application are not limited by the described action sequence, because In accordance with embodiments of the present application, certain steps may be performed in other sequences or concurrently. In the following, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required in the embodiments of the present application.

Referring to FIG. 4, a structural block diagram of an apparatus for processing an I/O request in a KVM virtualization process of the present application is shown. Specifically, the following modules may be included:

The I/O channel determining module 401 is configured to intercept, by the kernel virtual machine KVM, an I/O request sent by the virtual machine, and determine information of an I/O channel corresponding to the I/O request, where the I/O channel Having a corresponding buffer, the information of the I/O channel includes an eventfd file descriptor;

a buffer write module 402, configured to write the I/O request into a buffer of a corresponding I/O channel;

The event notification module 403 is configured to notify the user state process corresponding to the I/O channel by using the eventfd;

The I/O simulation module 404 is configured to: in the user state process, listen to an event of the corresponding I/O channel according to the eventfd, and obtain an I/O request from a buffer of the corresponding I/O channel. Performing an I/O simulation on the I/O request.

In a preferred embodiment of the embodiment of the present application, the device may further include:

a monitoring module, configured to monitor the eventfd file descriptor;

a buffer mapping module, configured to: use the second device simulation process to map a buffer of the I/O channel for the user state process based on a file descriptor of the I/O channel;

In a preferred embodiment of the embodiment of the present application, the I/O request includes an I/O address, and the I/O channel determining module 401 may include the following submodules:

In a preferred embodiment of the embodiment of the present application, the address range of the I/O channel includes an address subinterval of a plurality of I/O analog devices;

The I/O simulation module 404 can include the following sub-modules:

In a preferred embodiment of the embodiment of the present application, the device further includes:

In a preferred embodiment of the embodiment of the present application, the I/O request includes a read request, and the distribution sub-module is further configured to:

For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The various embodiments in the present specification are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same similar parts between the various embodiments can be referred to each other.

Those skilled in the art will appreciate that embodiments of the embodiments of the present application can be provided as a method, apparatus, or computer program product. Therefore, the embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, embodiments of the present application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

In a typical configuration, the computer device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory. Memory is an example of a computer readable medium. Computer readable media includes both permanent and non-persistent, removable and non-removable media. Information storage can be implemented by any method or technology. The information can be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape storage or other magnetic storage devices or any other non-transportable media can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-persistent computer readable media, such as modulated data signals and carrier waves.

Embodiments of the present application are described with reference to flowcharts and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing terminal device to produce a machine such that instructions are executed by a processor of a computer or other programmable data processing terminal device Means are provided for implementing the functions specified in one or more of the flow or in one or more blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The instruction device implements the functions specified in one or more blocks of the flowchart or in a flow or block of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device such that a series of operational steps are performed on the computer or other programmable terminal device to produce computer-implemented processing, such that the computer or other programmable terminal device The instructions executed above provide steps for implementing the functions specified in one or more blocks of the flowchart or in a block or blocks of the flowchart.

While a preferred embodiment of the embodiments of the present application has been described, those skilled in the art can make further changes and modifications to the embodiments once they are aware of the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all the modifications and the modifications

Finally, it should also be noted that in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities. There is any such actual relationship or order between operations. Furthermore, the terms "comprises" or "comprising" or "comprising" or any other variations are intended to encompass a non-exclusive inclusion, such that a process, method, article, or terminal device that includes a plurality of elements includes not only those elements but also Other elements that are included, or include elements inherent to such a process, method, article, or terminal device. An element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that comprises the element, without further limitation.

The method for processing I/O requests under KVM virtualization provided by the present application and a device for processing I/O requests under KVM virtualization are described in detail. In this paper, specific examples are applied to the principle of the present application. The description of the above embodiments is only for helping to understand the method of the present application and its core ideas; at the same time, for those skilled in the art, according to the idea of the present application, in the specific implementation and application scope There are variations, and the contents of this specification should not be construed as limiting the application.

Claims

A method for processing an I/O request under KVM virtualization, characterized in that the method comprises:

The I/O request sent by the virtual machine is intercepted by the kernel virtual machine KVM, and the information of the I/O channel corresponding to the I/O request is determined, wherein the I/O channel has a corresponding buffer, and the I/O The information of the O channel includes an eventfd file descriptor;

Writing the I/O request into a buffer of a corresponding I/O channel;

Notifying the user state process corresponding to the I/O channel by using the eventfd;

In the user state process, according to the eventfd, the event of the corresponding I/O channel is monitored, and an I/O request is obtained from the buffer of the corresponding I/O channel, and the I/O request is executed. An I/O simulation, wherein the user state process includes a first device emulation process and a second device emulation process independent of the first device emulation process.
The method according to claim 1, wherein the intercepting the I/O request sent by the virtual machine by the kernel virtual machine KVM determines the information of the I/O channel corresponding to the I/O request. Before the step, it also includes:

Creating a virtual machine by using the first device simulation process to obtain a file descriptor of the virtual machine;

Using the first device simulation process, creating a virtual processor vCPU based on the file descriptor of the virtual machine, and obtaining a file descriptor of the vCPU;

Using the second device simulation process to create an eventfd file descriptor;

Using the second device simulation process, creating an I/O channel based on the file descriptor of the virtual machine, and binding the I/O channel to the eventfd file descriptor to obtain the I/O channel File descriptor

Using the second device simulation process to monitor the eventfd file descriptor;

Using the second device simulation process to map a buffer of the I/O channel for the user state process based on a file descriptor of the I/O channel;

Using the second device emulation process, an I/O address interval is bound to the I/O channel based on a file descriptor of the I/O channel.
The method according to claim 2, wherein the I/O request includes an I/O address, and the I/O request sent by the virtual machine is intercepted by the kernel virtual machine KVM, and the I/O request is determined. The steps of the corresponding I/O channel information include:

Intercepting I/O requests from virtual machine vCPU threads through kernel virtual machine KVM and Intel VT-d hardware technology;

Extracting a corresponding I/O address from the I/O request;

Determining an I/O address interval to which the I/O address belongs;

And determining, by the I/O address interval, a corresponding I/O channel, and acquiring information of the I/O channel corresponding to the I/O channel.
The method according to claim 3, wherein the address interval of the I/O channel comprises an address subinterval of a plurality of I/O analog devices;

Determining, in the user state process, a corresponding I/O channel according to the eventfd, and acquiring an I/O request from a buffer of the corresponding I/O channel, performing the I/O request The steps of the I/O simulation include:

In the user state process, determining a corresponding I/O channel according to the eventfd;

Obtaining an I/O request from a buffer of the corresponding I/O channel;

Determining a corresponding I/O analog device according to the I/O address in the I/O request;

Distributing the I/O request to the I/O analog device, and performing I/O simulation on the I/O request by the I/O analog device.
The method according to claim 3 or 4, wherein in the user state process, a corresponding I/O channel is determined according to the eventfd, and buffering from the corresponding I/O channel Before the step of obtaining an I/O request in the zone and performing an I/O simulation on the I/O request, the method further includes:

Suspend the vCPU thread.
The method according to claim 5, wherein in the user state process, a corresponding I/O channel is determined according to the eventfd, and from a buffer of the corresponding I/O channel After obtaining the I/O request and performing the I/O simulation on the I/O request, the method further includes:

Wake up the suspended vCPU thread.
The method of claim 4 wherein said I/O request comprises a read request, said I/O request being distributed to said I/O analog device, said I/O simulation The steps for the device to perform I/O simulation on the I/O request are:

And distributing the read request to the I/O analog device, and writing, by the I/O analog device, data corresponding to the read request into a buffer of the I/O channel.
The method of claim 7, wherein the method further comprises:

If the I/O request is a read request, the data corresponding to the read request is read out from the buffer, and is read into a corresponding member variable in the vCPU structure;

Corresponding simulation processing is performed on the data.
An apparatus for processing an I/O request under KVM virtualization, characterized in that the apparatus comprises:

An I/O channel determining module, configured to intercept, by the kernel virtual machine KVM, an I/O request sent by the virtual machine to determine information of an I/O channel corresponding to the I/O request, where the I/O channel has Corresponding buffer, the information of the I/O channel includes an eventfd file descriptor;

a buffer write module, configured to write the I/O request into a buffer of a corresponding I/O channel;

An event notification module, configured to notify the user state process corresponding to the I/O channel by using the eventfd;

An I/O simulation module, configured to: in the user state process, listen to an event of the corresponding I/O channel according to the eventfd, and obtain an I/O request from a buffer of the corresponding I/O channel, Performing an I/O simulation on the I/O request, wherein the user state process includes a first device emulation process and a second device emulation process independent of the first device emulation process.
The device according to claim 9, further comprising:

a virtual machine creating module, configured to create a virtual machine by using the first device simulation process, and obtain a file descriptor of the virtual machine;

a vCPU creation module, configured to: use the first device simulation process, create a virtual processor vCPU based on a file descriptor of the virtual machine, and obtain a file descriptor of the vCPU;

An eventfd creation module is configured to create an eventfd file descriptor by using the second device simulation process;

An I/O channel creation module, configured to simulate a process by using the second device, create an I/O channel based on a file descriptor of the virtual machine, and bind the I/O channel to the eventfd file descriptor Determining, obtaining a file descriptor of the I/O channel;

a monitoring module, configured to monitor the eventfd file descriptor by using the second device simulation process;

a buffer mapping module, configured to: use the second device simulation process to map a buffer of the I/O channel for the user state process based on a file descriptor of the I/O channel;

An address interval binding module is configured to bind an I/O address interval to the I/O channel based on a file descriptor of the I/O channel by using the second device simulation process.
The apparatus according to claim 10, wherein the I/O request comprises an I/O address, and the I/O channel determining module comprises:

An I/O request intercepting submodule for intercepting I/O requests from a virtual machine's vCPU thread through a kernel virtual machine KVM and an Intel VT-d hardware technology;

An I/O address extraction submodule, configured to extract a corresponding I/O address from the I/O request;

a first I/O channel determining submodule, configured to determine an I/O address interval to which the I/O address belongs, and determine a corresponding I/O channel by using the I/O address interval, and acquire the I/ Information about the I/O channel corresponding to the O channel.
The apparatus according to claim 11, wherein the address range of the I/O channel comprises an address subinterval of a plurality of I/O analog devices;

The I/O simulation module includes:

a second I/O channel determining submodule, configured to determine, according to the eventfd, a corresponding I/O channel in the user state process;

An I/O request reading submodule, configured to obtain an I/O request from a buffer of the corresponding I/O channel;

An I/O analog device determining submodule, configured to determine a corresponding I/O analog device according to an I/O address in the I/O request;

And a distribution submodule, configured to distribute the I/O request to the I/O simulation device, and perform I/O simulation on the I/O request by using the I/O simulation device.
The device according to claim 11 or 12, further comprising:

Suspending a module, for suspending the vCPU thread after the userfest process corresponding to the I/O channel is notified by using the eventfd.
The device according to claim 13, further comprising:

And a wake-up module, configured to wake up the suspended vCPU thread after performing I/O simulation on the I/O request.
The apparatus according to claim 12, wherein the I/O request comprises a read request, and the distribution sub-module is further configured to:

And distributing the read request to the I/O analog device, and writing, by the I/O analog device, data corresponding to the read request into a buffer of the I/O channel.
The device according to claim 15, wherein the device further comprises:

a data reading module, configured to: when the I/O request is a read request, read data corresponding to the read request from a buffer, and read the corresponding member variable into the vCPU structure, The data is subjected to corresponding analog processing.