CN116954830A

CN116954830A - Method for enabling virtual machine to realize msi/x interrupt under jailhouse

Info

Publication number: CN116954830A
Application number: CN202311199076.7A
Authority: CN
Inventors: 韩乃平; 黄磊; 齐璇
Original assignee: Kirin Software Co Ltd
Current assignee: Kirin Software Co Ltd
Priority date: 2023-09-18
Filing date: 2023-09-18
Publication date: 2023-10-27
Anticipated expiration: 2043-09-18
Also published as: CN116954830B

Abstract

The method for enabling the virtual machine to realize msi/x interrupt under the jailmouse comprises the steps that a vITS structure running on an el2 layer is created for each virtual machine through a virtual ITS driver in the jailmouse and is used as a mapping of physical ITS, so that the virtual machine can send and receive msi/x interrupt; the created vITS structure comprises necessary components simulating ITS, including virtual device table, collection table and related registers, and the vITS command of the virtual machine is inserted into a command queue of the physical ITS through the necessary components simulating the ITS, so that the virtual machine has the capability of sending and receiving msi/x interrupt; the invention solves the problem that Guest OS can not send and receive msi/x interrupt by designing and realizing a vITS mechanism under jailhouse as the mapping of a physical ITS module, thereby dividing the pci/pcie equipment to Guest OS.

Description

Method for enabling virtual machine to realize msi/x interrupt under jailhouse

Technical Field

The invention relates to the technical field of performance optimization of virtual machines under jailmouses, in particular to a method for enabling virtual machines to realize msi/x interrupt under jailmouses.

Background

jailhouse is a lightweight virtualization scheme that is dominated by siemens, paravirtualized. jailhouse focuses on partitioning of hardware resources rather than sharing and virtualization of hardware resources, so overhead and latency introduced by virtualization can be minimized.

According to service requirements, resources such as a CPU, a memory, a device interface, an interrupt and the like are allocated before operation, the safety and the independence of resource allocation are guaranteed in the operation process, the multi-core CPU is divided into a plurality of independent operation environments, and each operation environment can operate different operation systems to deploy different services.

At present, due to the characteristics of an armv8 architecture, a GIC ITS module cannot be shared by a plurality of Guest OSs under the condition of jailhouse, and the Guest OSs cannot send and receive msi/x interrupts.

Disclosure of Invention

Aiming at the problems, the invention provides a method for enabling a virtual machine to realize msi/x interrupt under jailmouse, which solves the problem that Guest OS can not send and receive msi/x interrupt by designing and realizing a vITS mechanism under jailmouse as the mapping of a physical ITS module, thereby dividing pci/pcie equipment to Guest OS.

In order to achieve the above purpose, the invention discloses a method for enabling a virtual machine to realize msi/x interrupt under jailmouse, wherein a vITS structure running in an el2 layer is created for each virtual machine through a virtual ITS driver in the jailmouse and is used as a mapping of a physical ITS, so that the virtual machine can send and receive msi/x interrupt;

the created vITS structure comprises necessary components of the simulation ITS, wherein the necessary components of the simulation ITS comprise virtual device tables, collection tables and related registers, the related registers comprise a GITS_IDREGS register, a GITS_CBASER register, a GITS_CREADR register, a GITS_CWRITER register and a GITS_CTLR register, and the above components are stored in a memory of the jailhouse;

inserting a vITS command of the virtual machine into a command queue of a physical ITS by simulating the necessary components of the ITS, so that the virtual machine has the capability of sending and receiving msi/x interrupts;

wherein:

device table: the mapping relation is used for storing and mapping the deviceID in each virtual machine to the interrupt conversion table;

DeviceID: a deviceID field in the corresponding ITS command representing the corresponding interrupt-generating device number in the command;

collecting table: the Collection table provides a mapping of ICID (interrupt set number Interrupt Collection number) to the base address of the destination redistributor register, which is kept in memory for each v its, wherein the redistributor register is a set of registers for interrupt routing management between gic and cpu;

the gits_idregs register is used to describe the properties of the vGIC itself;

the GITS_CBASER register is used for storing the base address of the command queue;

the GITS_CREADR register is used for recording the offset of the address of the instruction executed by the ITS relative to the base address of the command queue;

the GITS_CWRITER register is used for recording the offset of the address of the software writing instruction relative to the base address of the command queue;

the gits_ctlr register is used to control the operation of ITS, which triggers interrupt translation and begins processing command queues when it writes 1.

Further, creating a vITS structure running in the el2 layer for each virtual machine through a virtual ITS driver in the jailhouse includes the following steps:

step S1: initialization of vITS and realization of a structural body;

step S2: the following data structure is added in the vITS structure: /ITS_CTRL register->Data structure,/>Preserving device table and select table address +.>Data structure,/>Command queue related register and Lock>Data structure,/>Virtual device table and virtual collection table and lock>A data structure;

step S3: analog gits_idregs register, gits_cbaser register, gits_create register, gits_cwreter register, and gits_ctlr register.

Further, in the step S1, the initialization of the v its and the implementation of the structure are implemented by adding a v gic_its_create () function to the jailhouse source code arch_create_cell () function.

Further, in the step S3,

the simulation strategy of the GITS_IDREGS register is to directly transmit the read request of the virtual machine to the ITS;

the simulation strategies of the GITS_CBASER register, the GITS_CREADR register and the GITS_CWRITER register are that the virtual machine is mapped to the physical ITS address by establishing a mapping table established when the virtual machine is established, and the command of the virtual machine is inserted into the command queue tail of the physical ITS;

the simulation strategy of the gits_ctlr register is to put the value of the simulated CTLR register into the struct gic_vits data structure, and write the val value into gic _vits.

Further, in the interrupt receiving process, the virtual machine intercepts msi/x interrupt messages of the pci/pmie device, and routes the interrupt to the core cpu of the corresponding virtual machine through device table and collection table search of the vITS.

The invention solves the problem that Guest OS can not send and receive msi/x interrupt by designing and realizing a vITS mechanism under jailhouse as the mapping of a physical ITS module, thereby dividing the pci/pcie equipment to Guest OS.

Drawings

Fig. 1: the invention discloses an implementation mechanism structure diagram of a method for enabling a virtual machine to realize msi/x interrupt under jailhouse.

Fig. 2: the host and the virtual machine access ITS register logic schematic diagram.

Fig. 3: according to the method for enabling the virtual machine to realize msi/x interrupt under the jailhouse, a register simulation effect diagram is achieved, wherein a gits_cwriter pointer points to the address of a command to be written next in a cmd queue, a gits_creator pointer points to the address of the command next in the cmd queue, and a gits_cbaser variable is used for storing the first address of the command in the cmd queue.

Detailed Description

In order to further understand the technical scheme and beneficial effects of the present invention, the technical scheme and beneficial effects thereof will be described in detail with reference to the accompanying drawings.

In order to solve the problem that Guest OS can not send and receive msi/x interrupt under jailhouse, the general idea of the invention is to realize a vITS mechanism, so that the vITS mechanism operates in an el2 layer and is used as the mapping of a physical ITS module, and the problem that Guest OS can not send and receive msi/x interrupt is solved. A block diagram of an implementation of the implementation mechanism is shown in fig. 1, and will be described in detail below.

1. Physical ITS mechanism implementation

A new interrupt type is defined in GICv3, LPI (locality-specific peripheral interrupts). LPI is a message-based interrupt. The MSI/MSIX in PCI/PCIE protocol is based on LPI type interrupts, which are no longer communicated via interrupt lines.

Gic v3 defines two methods to implement LPI interrupts:

forwarding mode: the peripheral device can directly send the LPI interrupt by accessing the register GICR_SERRLPIR of the redistributor;

using ITS approach: ITS (Interrupt Translation Service) is optional in gic v 3. The ITS is responsible for receiving interrupts from the peripheral devices and converting them into LPI INTIDs for transmission to the corresponding redistributors. In general, comparison recommends using ITS to implement LPI, which can be more efficient in scenarios where interrupt sources are more numerous because ITS provides many features. Thus, the present invention uses ITS to implement LPI interrupts.

1. ITS initialization

The HostOS initializes the ITS when the kernel starts up as long as it lists the item in the device tree. Otherwise, the kernel starts up and reports no ITS. Taking the phytium e2000d as an example, the positions and contents of ITS configuration in dts are as follows:

interrupt-controller@30800000 {

compatible = "arm,gic-v3";

#interrupt-cells = <0x3>;

#address-cells = <0x2>;

#size-cells = <0x2>;

ranges;

interrupt-controller;

reg = <0x0 0x30800000 0x0 0x20000 0x0 0x308c0000 0x0 0x80000 0x0 0x30840000 0x0 0x10000 0x0 0x30850000 0x0 0x10000 0x0 0x30860000 0x0 0x10000>;

interrupts = <0x1 0x9 0x8>;

phandle = <0x1>;

gic-its@30820000 {

compatible = "arm,gic-v3-its";

msi-controller;

reg = <0x0 0x30820000 0x0 0x20000>;

phandle = <0xe>;

};

2. ITS implementation flow

The ITS uses device tables, interrupt translation table, collection table 3 tables to translate and route LPI interrupts, all of which are tables stored in the OS memory where the GIC driver resides. When the ITS receives a write operation at ITS transducer register, it performs the following operations:

(1) First, index the device table using the device id (i.e., the requester id) in the MSIX message to get the base address of interruption translation table;

(2) Each device id corresponds to an interrupt translation table (Interrupt translation table), and after the base address of the interrupt translation table is obtained, an event id (namely data in the MSIX message) index is used for obtaining an INTID (hardware interrupt connection) and a collection id;

(3) The Collection table is a mapping table from Collection id to a redistributor register, and a Collection id is used for indexing the table to obtain a redistributor set, namely a target CPU set to which the interrupt is sent;

(4) The interrupt is forwarded to the selected redistributor register.

3. Implementation of ITS commands

In process 1, three tables related to the ITS implementation flow: the device tables, interrupt translation Table and Collection tables are collectively referred to as ITS tables. The configuration of the ITS tables is configured by core write ITS Command Queue (Command Queue), and Command Queue is stored in memory. After the core writes the Command into the Command Queue, the ITS is notified through the register, the ITS reads the Command from the memory in turn, and then the corresponding ITS Table page Table entry is configured according to the Command content.

The command queue is stored in the memory, and the system designs a section of continuous 4KB with GITS_CBASER. Physical_Address (physical Address) as base AddressThe size space is used to store the ITS Command Queue, where each Entry is 32 bytes (consistent with ITS Command);

the register is used for recording the memory base address, the attribute and the corresponding state of the command queue. The registers are mainly three:

(1) Gits_cbaser: the register stores the base address of Command Queue, the access attribute (cacheable/sharable) of the allocated memory, the size of the allocated memory space (how much 4 KB);

(2) Gits_create: an offset of an address of an instruction for ITS execution relative to a base address of a command queue is recorded;

(3) Gits_cwiriter: for recording the offset of the address of the software write instruction relative to the base address of the command queue.

ITS commands are arranged in a Command Queue in a write order, and ITS reads instructions from the Command Queue in order and executes them. Taking the GITS_CBASER.physical_Address as a base Address, the GITS_CREADR identifies the instruction location of the ITS execution. ITS Command Queue supports executing instructions while writing instructions. Each time a new instruction is written, gits_cwiriter will be offset by 0x20; similarly, each time the ITS executes an instruction, GITS_CREADR will be shifted by +0x20.

2. Virtual vITS implementation mechanism

As shown in fig. 2: the processing of the ITS register is divided into root cell processing and none-root cell processing. The initialization of ITS is performed in root cell linux, and jailhouse hypervisor first maps registers of ITS to EL2 layer, and then performs operations of ITS registers in EL2 layer.

In jailhouse root cell, the HostOS may directly access the physical ITS register to send and receive ITS Command, which is MMIO (Memory mapping I/O), where the jailhouse will pass through MMIO operations; if it is jailhouse none root cell, the ITS Command of the Guest OS is intercepted jailhouse hypervisor and walks into the vITS flow;

as shown in fig. 2, when a non-root cell (i.e., the virtual machine shown in fig. 1) is created, jailhouse virtual ITS driver creates a vITS structure for each Guest OS, where the structure contains necessary components for simulating ITS, including a virtual device table, a Collection table and related registers, and by calculating the above components, the vITS Command of the Guest OS can be inserted into the physical ITS Command Queue, so that the Guest OS has the capability of transceiving msi/x interrupts.

jailhouse virtual ITS driver the process of creating a vITS structure for each Guest OS is as follows:

1. vITS initialization and structure implementation

Adding a vgic_its_create () function in the jailhouse source code arch_create_cell () function;

the rc_create_cell () is a native function, and is used for reflecting the specific position of the newly added function in the original function;

the vciic_its_create () functions as a virtual device table, collection table, cbase/cwriter/create, etc. structure used for initializing the vcis, and a mapping relation is established;

2. new data structure

The original data structure is as follows:

struct cell {

…

/ Architecture-specific fields. />/

struct arch_cell arch;

…

};

the struct_cell search data structure specifically includes:

struct arch_cell {

struct paging_structures mm;

u32 irq_bitmap[1024/32];

struct {

u8 ent_count;

struct pvu_tlb_entry entries;

} iommu_pvu; / < ARM PVU specific fields. />/

after the struct arch_cell arch data structure is added:

/new data structure->/

struct gic_vlits its;

};

Wherein, struct gic_vlisThe its data structure is:

struct gic_vits {

/ITS_CTRL register->/

u32 vits_ctrl;

/Preserving device table and select table address +.>/

u64 baser_device_table;

u64 baser_collection_table;

/Command queue related register and Lock>/

struct spin_lock cmd_lock;

u64 cbaser;

u32 creadr;

u32 cwriter;

/Virtual device table and virtual collection table and lock>/

struct spinlock_t its_lock;

struct device_table_entry device_table_entrys[30];

struct collection_table_entry collection_tables_entrys[30];

};

3. Implementing emulation of corresponding registers

(1) Simulation strategy 1

The Guest OS does not sense vITS, and still initializes vITS with ITS driver in the Guest OS kernel, so that GITS_IDREGS needs to be simulated to configure basic capability of ITS for the Guest OS, and meanwhile, the address mapping relation between vITS and physical ITS of the Guest OS is established by initializing the addresses of the server_device_table and the server_collection_table.

(2) Simulation strategy 2

In the interrupt receiving process, the virtual machine carries out routing according to the established mapping relation in the strategy I: in the sending flow, the jailhouse intercepts msi/x interrupt messages of the pci/pmie device, and routes the interrupt to the core cpu where the corresponding Guest OS is located through device table and collection table search of the vITS.

As described in the first section above, the registers required for physical ITS processing commands relate to gits_cbaser, gits_create, gits_cwriter, so constructing a virtual v ITS structure requires simulating these three registers, while at the same time simulating the registers gits_ctlr that control the operation of ITS.

In the two simulation strategies, the specific simulation method for the four registers is as follows:

(1) Simulation of GITS IDREGS (Identification registers)

Register definition: the register offset 0xFFD0-0xFFFC, which is a read-only register, describes some of the attributes of the vGIC itself.

Simulation strategy: because the set of registers is read-only, there is no write strategy; the read strategy is to pass the Guest OS read request directly to the ITS;

(2) Simulation of GITS CTLR (ITS controller register)

Register definition: register for controlling operation of ITS

Simulation strategy: the GITS CTLR register is a 32-bit register. The value of the emulated CTLR register is placed into the struct gic_vits data structure. Writing the val value into gic _vits.vits_ctrl, the Guest OS will also read and write the GITS_CTLR register value from gic _vits.vits_ctrl;

(3) Simulation of GITS_CBASER/GITS_CWRITER/GITS_CREADER

Simulation strategy: mapping a vITS address in the Guest OS to a physical ITS address by establishing a mapping table established when the Guest OS, and inserting a Command into a physical ITS Command queue queue tail; one instruction is 32 bytes in length and the instruction format of the different instructions is different.

FIG. 3 is a diagram of a register emulation effect in which GITS_CBASER points to a circular queue of Command instructions, cWriter is the head pointer and creater is the tail pointer. As a result, the simulation strategy achieves the effect of directly fetching the contents of the cwriter memory in the Guest OS into the cwriter of the HostOS.

In conclusion, the invention solves the problem that Guest OS can not send and receive msi/x interrupt by designing and realizing a mechanism of vITS under jailhouse as the mapping of a physical ITS module. So that the pci/pcie device can be partitioned to the Guest OS.

Although the present invention has been described with reference to the above preferred embodiments, it should be understood that the present invention is not limited to the above embodiments, and that various changes and modifications can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

A method for enabling a virtual machine to realize msi/x interrupt under jailhouse is characterized by comprising the following steps: creating a vITS structure running in an el2 layer for each virtual machine through a virtual ITS driver in the jailhouse, and enabling the virtual machine to send and receive msi/x interrupts as a mapping of physical ITS;

the created vITS structure comprises necessary components of the simulation ITS, wherein the necessary components of the simulation ITS comprise virtual device tables, collection tables and related registers, the related registers comprise a GITS_IDREGS register, a GITS_CBASER register, a GITS_CREADR register, a GITS_CWRITER register and a GITS_CTLR register, and the above components are stored in a memory of the jailhouse;

inserting a vITS command of the virtual machine into a command queue of a physical ITS by simulating the necessary components of the ITS, so that the virtual machine has the capability of sending and receiving msi/x interrupts;

wherein:

device table: the mapping relation is used for storing and mapping the deviceID in each virtual machine to the interrupt conversion table;

DeviceID: a deviceID field in the corresponding ITS command representing the corresponding interrupt-generating device number in the command;

the Collection table is used for providing a mapping from the interrupt set number Interrupt Collection number to the base address of the destination redistributor register, and only one Collection table is stored in the memory for each vITS;

the gits_idregs register is used to describe the properties of the vGIC itself;

the GITS_CBASER register is used for storing the base address of the command queue;

the GITS_CREADR register is used for recording the offset of the address of the instruction executed by the ITS relative to the base address of the command queue;

the GITS_CWRITER register is used for recording the offset of the address of the software writing instruction relative to the base address of the command queue;

the gits_ctlr register is used to control the operation of ITS, which triggers interrupt translation and begins processing command queues when it writes 1.
2. The method for enabling the virtual machine to realize msi/x interrupt under jailhouse according to claim 1, which is characterized in that: creating a vITS structure running in the el2 layer for each virtual machine through a virtual ITS driver in the jailhouse comprises the following steps:

step S1: initialization of vITS and realization of a structural body;

step S2: the following data structure is added in the vITS structure: /ITS_CTRL register->Data structure,/>Preserving device table and select table address +.>Data structure,/>Command queue related register and lockData structure,/>Virtual device table and virtual collection table and lock>A data structure;

step S3: analog gits_idregs register, gits_cbaser register, gits_create register, gits_cwreter register, and gits_ctlr register.
3. The method for enabling the virtual machine to realize msi/x interrupt under jailhouse according to claim 2, characterized in that: in the step S1, the initialization of the vITS and the implementation of the structure are implemented by adding a vgic_its_create () function to the jailhouse source code arch_create_cell () function.
4. The method for enabling the virtual machine to realize msi/x interrupt under jailhouse according to claim 2, characterized in that: in the step S3 of the above-mentioned process,

the simulation strategy of the GITS_IDREGS register is to directly transmit the read request of the virtual machine to the ITS;

the simulation strategies of the GITS_CBASER register, the GITS_CREADR register and the GITS_CWRITER register are that the virtual machine is mapped to the physical ITS address by establishing a mapping table established when the virtual machine is established, and the command of the virtual machine is inserted into the command queue tail of the physical ITS;

the simulation strategy of the gits_ctlr register is to put the value of the simulated CTLR register into the struct gic_vits data structure, and write the val value into gic _vits.
5. The method for enabling the virtual machine to realize msi/x interrupt under jailhouse according to claim 1, which is characterized in that: in the receiving interrupt flow, the virtual machine intercepts msi/x interrupt information of the pci/pmie device, and routes the interrupt to the core cpu of the corresponding virtual machine through the device table and collection table search of the vITS.