CN114661414A

CN114661414A - Computing system, memory page fault processing method and storage medium

Info

Publication number: CN114661414A
Application number: CN202210209837.1A
Authority: CN
Inventors: 郭凯杰; 罗犇; 彭开桓; 罗宁
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-03-04
Filing date: 2022-03-04
Publication date: 2022-06-24
Also published as: WO2023165400A1

Abstract

The embodiment of the application provides a computing system, a memory page fault processing method and a storage medium. In the embodiment of the application, a missing page request service component is additionally arranged on a side of the direct connection device, a missing page request driver is added on a management node of the virtual machine, the missing page request service component and the missing page request driver are matched with each other, asynchronous recovery of a missing page of a large block IO memory corresponding to large block data is achieved, and the direct connection device can asynchronously access a recovered memory page. Compared with the existing synchronous page fault request recovery mode, the method does not need to wait for all the memory pages corresponding to the large data blocks to be recovered, is beneficial to reducing the IO memory page fault recovery time length, and further reduces the memory access delay of the direct connection equipment.

Description

Computing system, memory page fault processing method and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a computing system, a memory page fault processing method, and a storage medium.

Background

In the field of computers, a Direct Pass-Through technology (Direct Pass-Through) introduces an Input/Output Memory Management Unit (IOMMU), and dedicates a device to a single client by sacrificing the sharing capability of the device, so as to achieve the purpose of realizing all functions and optimal performance of the device. The IOMMU may connect a Direct Memory Access (DMA) I/O bus to the Memory of the host. The IOMMU can convert the virtual address accessed by the direct equipment (device) into a physical address, and the direct equipment can access the memory of the host machine.

In practical applications, a Virtual Machine Manager (VMM) will recycle a physical memory, store data stored in the physical memory in a hard disk, and release a physical memory page for use by other processes. When the process of the pass-through device needs the memory again, an IO memory Page Fault (Page Fault) occurs.

The Page table Request Interface (PRI) is a standard PCI peripheral missing Page Request protocol defined by the PCI-SIG organization. In the PRI specification, a model for providing a PRI service in a Root Complex (Root Complex) is defined. The PRI service provided by Root Complex can recover the IO memory page fault. However, the inventor finds that through practical tests: the time for recovering the missing page of the IO memory by the PRI service provided by the Root Complex is long, which results in a large delay of the direct-through device to access the memory.

Disclosure of Invention

Aspects of the present disclosure provide a computing system, a method for processing a missing page in a memory, and a storage medium, so as to reduce a recovery time of the missing page in the IO memory and reduce a delay of accessing the memory by a pass-through device.

An embodiment of the present application provides a computing system, including: a host machine and a virtual machine management node; the host machine is provided with a virtual machine and is hung with a direct-connection device for the direct connection of the virtual machine; the operating system of the virtual machine comprises a memory management module;

the pass-through device comprises: a missing page request service component; the virtual machine management node includes: a page missing request driver;

the page missing request service component is configured to send a page missing request to the page missing request driver when a memory page accessed by the pass-through device is missing;

the missing page request driver is used for asynchronously requesting the memory management module to recover the missing memory pages according to the missing page request; asynchronously returning the identifier of the recovered memory page to the page missing request service assembly;

and the straight-through equipment asynchronously accesses the recovered memory page according to the identifier of the recovered memory page.

The embodiment of the present application further provides a method for processing a page fault in a memory, including:

acquiring a page missing request sent by a page missing request service component in the direct connection equipment;

according to the page missing request, the asynchronous request memory management module recovers the missing memory pages;

and asynchronously returning the identifier of the recovered memory page to the page missing request service component, so that the straight-through device asynchronously accesses the recovered memory page according to the identifier of the recovered memory page.

Embodiments of the present application further provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the above-mentioned page fault memory processing method.

In the embodiment of the application, a missing page request service component is additionally arranged on a side of the direct connection device, a missing page request driver is added on a management node of the virtual machine, the missing page request service component and the missing page request driver are matched with each other, asynchronous recovery of a missing page of a large block IO memory corresponding to large block data is achieved, and the direct connection device can asynchronously access a recovered memory page. Compared with the existing synchronous page fault request recovery mode, the method does not need to wait for all the memory pages corresponding to the large data blocks to be recovered, is beneficial to reducing the IO memory page fault recovery time length, and further reduces the memory access delay of the direct connection equipment.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic structural diagram of a computing system according to an embodiment of the present application;

FIG. 2 is a schematic diagram illustrating a processing manner of the computing system shown in FIG. 1 for IO memory page faults;

FIG. 3 is a block diagram of another computing system provided in an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating a processing manner of the computing system provided in FIG. 3 for IO memory page faults;

FIG. 5 is a schematic diagram illustrating another processing manner of the computing system of FIG. 3 for IO memory page faults;

fig. 6 is a flowchart illustrating a method for processing a page fault in a memory according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In some embodiments of the present application, a missing page request service component is added on a side of a direct connection device, a missing page request driver is added on a management node of a virtual machine, the missing page request service component and the missing page request driver are matched with each other, so as to implement asynchronous recovery of a missing page of a large block of IO memory corresponding to a large block of data, and the direct connection device 102 can asynchronously access a recovered page memory. Compared with the existing synchronous page fault request recovery mode, the method does not need to wait for all memory pages corresponding to the large data blocks to be recovered, is beneficial to reducing the recovery time of IO memory page fault, and further reduces the memory access delay of the direct connection equipment.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

It should be noted that: like reference numerals refer to like objects in the following figures and embodiments, and thus, once an object is defined in one figure or embodiment, further discussion thereof is not required in subsequent figures and embodiments.

Fig. 1 is a schematic structural diagram of a computing system according to an embodiment of the present application. As shown in fig. 1, a computing system includes: a host Machine 10 and a Virtual Machine management node (VMM) 20.

In this embodiment, the host 10 refers to a computer device having functions of calculation, storage, communication, and the like, and may be, for example, a server device. For example, the host 10 may be a single server device, a cloud server array, or the like. Of course, the host 10 may also be a terminal device such as a mobile phone, a tablet computer, a personal computer, or a wearable device.

In this embodiment, the host 10 is deployed with a Virtual Machine (VM) 101. Among them, the virtual machine 101 may also be referred to as a client (Guest) of the host 10. The VM 101 has a separate CPU, memory, network, disk, and the like. The CPU corresponding to the virtual machine 101 may also be referred to as a virtual machine CPU, i.e., vCPU 101a shown in fig. 1.

In this embodiment, the VMM 20 refers to a logical node for managing a virtual machine, and may be deployed in the host 10, or may be deployed in another physical machine communicating with the host 10. For embodiments in which VMM 20 is deployed in host 10, VMM 20 may run on a processing unit (not shown in fig. 1) of host 10. VMM 20 may perform scheduling tasks, load balancing, status monitoring, etc. on VM 101. The processing unit of the host 10 may be a CPU, GPU, etc.

In Pass-through (Pass through) technology, VM 101 may directly access the I/O hardware mounted on host 10 through VMM 20, so that the I/O operation path of VM 101 is almost the same as the I/O path in a non-virtualized environment. The I/O hardware is a pass-through device 102. The pass-through device 102 refers to an I/O hardware device mounted on the host 10, and may include: network cards, storage media, and the like. The storage medium may be a persistent storage medium such as a magnetic disk or a hard disk. The pass-through device 102 may be communicatively connected to the processing unit of the host 10 via a communication interface. Depending on the form of the communication interface, the pass-through device 102 may be implemented as a PCI device, a PCIe device, or the like.

For the pass-through device 102, the memory of the host 10, i.e. the host memory 103, can be accessed by Direct Memory Access (DMA). For the DMA mode, a data path is provided between the pass-through device 102 and the host memory 103, so as to realize direct data transmission between the pass-through device 102 and the host memory 103. In the data transmission process between the pass-through device 102 and the host memory 103, the CPU of the host 10 is not required to participate.

In virtualization technologies, an operating system (guest OS) running on the VM 101 is generally unaware of the physical memory address of the Host machine it accesses, which may also be referred to as Host Physical Address (HPA), and is simply referred to as physical address. If a DMA operation is to be performed, there is a possibility that the memory will be corrupted. Because the pass-through device 102 can only know the physical memory address of the Virtual machine, which is referred to as a Virtual Address (VA) for short, and may also be referred to as a Guest Physical Address (GPA), the pass-through device 102 does not know the mapping relationship between the GPA and the HPA. In the present embodiment, therefore, the IOMMU104 is introduced in a pass-through technique.

IOMMU104 may be communicatively coupled between pass-through device 102 and host memory 103. Alternatively, IOMMU104 may be communicatively coupled to pass-through device 102 and host memory 103 via a communication interface bus. The serial interface bus may be a PCI interface bus, a PCIe interface bus, or the like. IOMMU104 may ensure that pass-through device 102 can access host memory 103 while performing DMA operations.

For hosts without IOMMU104, pass-through device 102 may access the entire memory address space of host 10 by DMA. For a host provided with IOMMU104, CPU 101a of the virtual machine may allocate a GPA accessible by pass-through device 102 and a corresponding HPA for the GPA by VMM 20. The IOMMU104 may maintain a mapping table between GPAs and HPAs, which may also be referred to as page tables. The page table records the correspondence between GPAs and HPAs. When pass-through device 102 accesses memory of host 10 by way of DMA, VMM 20 may capture DMA requests issued by pass-through device 102 and pass the DMA requests through to IOMMU 104. IOMMU104 may obtain the GPA to be accessed from the DMA request; then, the GPA to be accessed is matched in the page table stored in the IOMMU104 to obtain the HPA corresponding to the GPA to be accessed. For pass-through device 102, the memory space corresponding to host memory 103 may be accessed through the HPA corresponding to the GPA to be accessed.

In practical applications, VMM 20 may recycle the physical memory, store the data stored in the physical memory in the hard disk, and release the physical memory page for use by other processes. When the process of the pass-through device 102 needs the memory again, an IO memory Page Fault (Page Fault) occurs.

PRI is an industry standard PCI bus protocol defined by the PCI-SIG organization for pass-through devices to provide page fault requests to the Operating System (OS) running in the CPU to increase memory usage and flexibility. Under a virtual cloud computing scene, the hardware supporting the PRI capability can improve the utilization rate of the memory resources of the cloud computing to a new height, and the waste of hardware resources and energy is greatly avoided. In the PRI specification, a model for providing PRI services in a Root Complex (Root Complex)30 is defined. The Root Complex may typically be packaged inside a processing unit of host 10.

In a PCI Express (PCIe) system, among other things, Root complexes may connect processing units and memory subsystems to a PCIe switch fabric comprised of one or more switch devices. The Root Complex function may be implemented in a separate device or may be integrated in the processing unit.

In the present embodiment, the IOMMU104 may be located in a Root Complex (Root Complex)30, as shown in FIG. 1. The PRI specification also provides a way to provide IO page fault recovery in Root Complex. Fig. 2 is a schematic process diagram of providing IO page fault recovery by Root Complex according to PRI specification provided in the embodiment of the present application. As shown in FIG. 2, the process of providing IO missing page recovery in Root Complex provided by the PRI specification mainly includes the following steps:

step 1: for a PRI-enabled pass-through device 102 to initiate DMA, an IO Translation Lookaside Buffer (IOTLB) is first queried. In the event that the IOTLB does not cache a physical address corresponding to a Virtual Address (VA) contained by the DMA, pass-through device 102 sends an address translation request to IOMMU104 in root complex 30. The address translation request contains the virtual address contained by the DMA.

Step 2: the IOMMU104 obtains the address translation request and queries the correspondence between the recorded virtual addresses and physical addresses. In the corresponding relation, the virtual address refers to GPA, and the physical address refers to HPA. When the IOMMU104 does not find the physical address corresponding to the virtual address to be translated from the correspondence between the virtual address and the physical address, it determines that the memory is in a page fault state, and returns the address translation result to the pass-through device 102. Wherein, the address translation result in the step 2 reflects the page fault of the memory.

And step 3: the pass-through device 102 initiates a page fault request to the Root Complex 30 in response to the address translation result reflecting the memory page fault. The page fault request includes a virtual address corresponding to the page fault memory and a process space identifier of the pass-through device 102.

And 4, step 4: the IOMMU104 in Root Complex 30 submits a Page fault Request (Page Request) to the operating system via a memory queue (i.e., the Page fault Request queue in FIG. 2), and sends an interrupt to the operating system. The memory queue refers to a message queue in the host memory 103.

And 5: after receiving the interrupt, the operating system software (usually kernel driver) in the CPU reads out the page fault request from the memory queue.

Step 6: operating system software in the CPU restores a single memory page by calling the memory management module. The memory management module refers to a software function module in the operating system.

And 7: operating system software within the CPU sends a Page fault Response (Page Response) to Root Complex 30 via another memory queue, i.e., the Page fault Response queue in fig. 2. The page fault response may include the physical address of the physical memory page allocated for the paged memory.

And 8: the IOMMU104 hardware in Root Complex 30 receives the page fault response and processes the page fault response. Specifically, the IOMMU104 may write a corresponding relationship between a virtual address corresponding to the page-missing memory and a physical address allocated for the memory page into a virtual address space corresponding to the process space identifier corresponding to the pass-through device 102.

And step 9: root Complex 30 sends a page fault response to pass device 102. The page fault response is used to indicate that the page fault memory is restored.

Step 10: the pass-through device 102 receives the page fault response, determines that the page fault is recovered, and continues to perform DMA to acquire the data to be accessed. Specifically, pass-through device 102 sends an address translation request to IOMMU104, which contains the virtual address in the DMA.

Step 11: the IOMMU104 obtains the address translation request, and inquires the corresponding relation between the recorded virtual address and the physical address, to obtain the physical address corresponding to the virtual address in the DMA; and then returns the address translation results to pass-through device 102. The address translation result includes a physical address corresponding to the virtual address in the DMA.

Step 12: the pass-through device 102 directly accesses the memory space corresponding to the physical address based on the physical address corresponding to the virtual address in the DMA, and obtains the data to be accessed.

The above steps 1-12 are the main steps for recovering one memory page for IO missing page provided by Root Complex provided by PRI specification. However, through practical tests, the inventor of the present application finds that when a large number of page faults occur, the performance of IO may be seriously degraded through IO page fault recovery provided by Root Complex, for example, when a user memory with 1GB continuous page faults is processed, the IO page fault recovery mode provided by Root Complex provided by the PRI specification will trigger 262144 page fault interrupts, that is, the above steps 1-12 need to be repeated 262144 times.

The IO page fault recovery provided by Root Complex is 3-80 times slower than the CPU page fault recovery, and when IO page fault occurs, the throughput of the whole IO device (i.e. the pass-through device) is reduced by 30% -40%. Such severe performance degradation occurs because: the processing link for IO page fault recovery provided by Root Complex is long, and one page fault request can only process one memory page in a synchronous manner. The size of one memory page is usually 4KB, which undoubtedly results in long time consuming and high latency for IO missing page recovery provided by Root Complex.

In order to reduce the duration of IO missing page recovery and improve the efficiency of IO memory missing page recovery, embodiments of the present application provide a new computing system architecture, and provide a new IO memory missing page processing mode based on the computing system architecture. The following description will be given with reference to specific examples.

Fig. 3 is a schematic structural diagram of another computing system according to an embodiment of the present application. As shown in fig. 3, the computing system includes: a host Machine 10 and a Virtual Machine management node (VMM) 20. In this embodiment, the host 10 is deployed with a Virtual Machine (VM) 101. For the description of the host 10, the VMM 20 and the VM 101, reference may be made to the relevant contents of the above embodiments, and details are not described herein again.

In the pass-through technique, VM 101 may directly access pass-through device 102 mounted by host 10 through VMM 20. The description of the pass-through device 102 can refer to the related contents of the above embodiments, and will not be described herein.

In this embodiment, in order to reduce the duration of IO missing page recovery and improve the efficiency of IO memory missing page recovery, a missing page request service component 102a is added in the pass-through device 102; and adds the page fault request driver 201 corresponding to the page fault request service component 102a in the VMM 20. The missing page request service component 102a may be implemented as a hardware module, for example, built by using a hardware functional circuit, or may be formed by using a programmable device. The editable device can be an FPGA, an ASIC chip and the like. In the embodiment of the present application, a specific implementation form of the missing page request service component 102a is not limited.

In this embodiment, the missing page request service component 102a can reuse the communication interface of the pass-through device 102 to be communicatively connected to the processing unit (e.g., CPU) of the host 10. VMM 20 may be deployed and run on a processing unit of host 10 to enable communicative coupling of out-of-page request service component 102a with a processing unit of host 10 (not shown in fig. 3).

In this embodiment, VMM 20 may add a page fault request driver 201 corresponding to page fault request service component 102 a. The page fault request driver may be a software function module running on a processing unit (e.g., CPU) of the host 10, and is used for responding to a page fault request of the page fault request service component 102 a.

The following describes, with reference to fig. 4 and fig. 5, an exemplary IO page fault processing method provided in the embodiment of the present application. As shown in fig. 4, the page fault request service component 102a may issue a page fault request to the page fault request driver in case a memory page accessed by the pass-through device 102 is lost. In this embodiment of the present application, the memory page miss accessed by the pass-through device 102 specifically refers to that a physical address corresponding to the virtual address accessed by the pass-through device 102 is not stored in the page table of the IOMMU 104. The missing memory page is a memory page pointed to by a physical address corresponding to the virtual address accessed by the pass-through device 102.

As shown in fig. 4, the pass-through device 102 initiates a memory access request, i.e., a DMA request. The pass-through device 102 first queries an IO Translation Lookaside Buffer (IOTLB). In the event that the IOTLB does not cache a physical address corresponding to a Virtual Address (VA) included in the DMA request, pass-through device 102 sends an address translation request to IOMMU104 in root complex 30. The address translation request contains the virtual address contained by the DMA request. The virtual address included in the memory access request may be a single virtual address, or may be a continuous virtual address space including a plurality of continuous virtual addresses.

The IOMMU104 obtains the address translation request and looks up the correspondence between the virtual address and the physical address recorded by the page table. When the IOMMU104 does not find the physical address corresponding to the virtual address to be translated from the correspondence between the virtual address and the physical address, it determines that the memory is in a page fault state, and returns the address translation result to the pass-through device 102. The address translation result is a memory page fault.

In response to the address translation result of the page fault, the pass-through device 102 determines that the IO page fault is a physical address corresponding to a virtual address included in the memory access request, which is not present in the page table of the IOMMU 104. Further, as shown in fig. 4, the pass-through device 102 may trigger the missing page request service component 102a to issue the missing page request to the missing page request driver 201 when the physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU 104. The physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU104, which indicates that the memory page corresponding to the virtual address included in the memory access request is missing. The page fault request includes the virtual address in the memory access request and the process space identifier of the pass-through device 102. The virtual address in the memory access request included in the page fault request may be all virtual addresses included in the memory access request, or may also be a partial virtual address included in the memory access request, specifically, a virtual address where no corresponding physical address exists in the page table of the IOMMU 104.

In summary, the pass-through device 102 may trigger the missing page request service component 102a to issue the missing page request to the missing page request driver 201 when the page table of the IOMMU104 does not have a physical address corresponding to the virtual address included in the memory access request.

For pass-through device 102, the starting virtual address of the missing memory page, the number of missing memory pages, and the process space identifier may be determined if there is no physical address corresponding to the virtual address included in the memory access request in the page table of IOMMU 104.

Specifically, the pass-through device 102 initiates a memory access request; firstly, inquiring IOTLB; in the event that the IOTLB does not cache a physical address corresponding to a Virtual Address (VA) included in the memory access request, pass-through device 102 sends an address translation request to IOMMU104 in root complex 30. The address translation request includes the virtual address included in the memory access request. For the IOMMU104 to obtain an address translation request, the correspondence between the virtual address and the physical address recorded by the page table may be queried. If the physical address corresponding to the virtual address included in the memory access request is found, an address translation result reflecting successful translation is returned to the pass-through device 102. If the physical address corresponding to the virtual address included in the memory access request is not found in the page table, an address translation result reflecting a memory page fault is returned to the pass-through device 102.

For the pass-through device 102, when receiving the address translation result reflecting the memory page fault, it may determine that there is no physical address corresponding to the virtual address included in the memory access request in the page table of the IOMMU104, and determine the virtual address included in the address translation result as the starting virtual address of the memory page fault. Further, the pass-through device 102 may also determine the number of the missing memory pages according to the virtual address included in the memory access request and the starting virtual address of the missing memory pages. Specifically, the pass-through device 102 may determine, according to the virtual address included in the memory access request, the virtual address of the last memory page included in the memory access request; and determining the starting virtual address of the missing memory page and the virtual address of the last memory page included in the memory access request, and determining the number of the missing memory pages, that is, the number of the memory pages from the starting virtual address of the missing memory page to the virtual address of the last memory page included in the memory access request, that is, the number of the missing memory pages. In the embodiment of the present application, specific values of the number of missing memory pages are not limited, and are specifically determined by actual situations. The number of missing memory pages may be 1 or more. Plural means 2 or more.

Further, the pass-through device 102 may provide the starting virtual address of the missing memory page, the number of missing memory pages, and the process space identification to the missing page request service component 102 a. The page fault request service component 102a may generate a page fault request according to the starting virtual address of the missing memory page, the number of the missing memory pages, and the process space identifier. The page fault request may include a starting virtual address of the missing memory page, a number of the missing memory pages, and a process space identifier. Further, the page fault request service component 102a may provide a page fault request to the page fault request driver 201 (as shown in step 1 of fig. 5). In the embodiment of the present application, the PAGE fault REQUEST may be an asynchronous PAGE fault REQUEST (APR).

In the embodiment of the present application, as shown in fig. 5, the page fault request service component 102a may wake up the page fault request driver 201 by an interrupt (as shown in step 2 in fig. 5). For embodiments in which VMM 20 runs on a processing unit of host 10, the processing unit also running on host 10 is communicatively coupled to page fault request service component 102 a. In particular, the missing page request service component 102a may be communicatively coupled to a processing unit of the host 10 through a communication interface of the pass-through device 102. Accordingly, the page fault request service component 102a may send an interrupt signal to the processing unit over the communication link with the processing unit. In the embodiment of the present application, the implementation form of the interrupt signal is not limited. In some embodiments, the interrupt signal may be a pulse signal. Specifically, the page fault request service component 102a may send an interrupt signal to the processing unit through a communication link with the processing unit in case a memory page accessed by the pass-through device 102 is lost. The processing unit wakes up the page fault request driver 201 in response to the interrupt signal.

The PAGE fault request driver 201 wakes up and returns an asynchronous RESPONSE START message (ASYNC _ PAGE _ RESPONSE _ START, APRS) to the PAGE fault request service component 102a in RESPONSE to the PAGE fault request (step 3 in fig. 5). The APRS indicates asynchronous memory page fault start processing of a page fault request.

The page fault request driver 201 may asynchronously request the memory management module 40 of the Operating System (OS) of the virtual machine 101 to recover the memory page with the page fault according to the page fault request (corresponding to steps 4-x4 in fig. 5). In this embodiment, the operating system of the virtual machine 101 may run on the CPU of the virtual machine. The memory management module 40 refers to a software function module for managing and distributing memory.

In the embodiment of the present application, a specific implementation manner of the page fault request driver 201 asynchronously requesting the memory management module 40 of the Operating System (OS) of the virtual machine 101 to recover the page of the memory that is page fault is not limited. In some embodiments, the missing page request driver 201 may asynchronously request the memory management module 40 to perform batch recovery on the missing memory pages.

Specifically, the missing page request driver 201 may obtain the number of missing memory pages and the process space identifier corresponding to the pass-through device from the missing page request; and the following steps are executed in a circulating manner until the number of the recovered memory pages is equal to the number of the missing memory pages, namely the missing memory pages are recovered completely. Wherein, the step of circulating execution includes:

determining a virtual address to be recovered in a current batch; the asynchronous request memory management module 40 allocates a target memory page to the virtual address to be restored; and writes the correspondence between the physical address of the target memory page and the virtual address to be restored in the current batch into the page table corresponding to the process space identifier in the IOMMU 104. The page table corresponding to the process space identifier is the process space identifier of the pass-through device 102 included in the page fault request.

In some embodiments, the current lot may be the first lot. For the first batch, the missing page request driver 201 may obtain the starting virtual address of the missing memory page from the missing page request; and determining the virtual addresses to be recovered in the first batch according to the initial virtual address. In the embodiment of the present application, the number of memory pages restored per batch is not limited. In some embodiments, if the memory page recovered in each batch is 1 memory page, the starting virtual address of the missing memory page may be determined to be the virtual address to be recovered in the first batch. In other embodiments, the memory pages recovered per batch are M memory pages. Wherein M is not less than 2 and is an integer. Accordingly, the missing page request driver may sequentially select M virtual addresses from the starting virtual address of the missing memory page as the first to-be-recovered virtual addresses. Fig. 5 illustrates only the number of memory pages restored per batch as 1, but is not limited thereto.

In other embodiments, the current lot may be any lot other than the first lot. For any batch except the first batch, the page fault request driver 201 may determine the virtual address to be restored in any batch according to the virtual address corresponding to the memory page restored in the previous batch of any batch. Specifically, the missing page request driver 201 may sequentially select N virtual addresses from the next virtual address of the virtual addresses corresponding to the memory pages recovered from the previous batch of any batch as the virtual addresses to be recovered in the batch. Wherein N is a positive integer. N refers to the number of memory pages recovered per batch.

For any batch, the page fault request driver 201 may asynchronously request the memory management module 40 to allocate a target memory page for the virtual address to be recovered. In the embodiment of the present application, a specific implementation manner of allocating, by the memory management module 40, a target memory page for a virtual address to be recovered is not limited. Alternatively, the memory management module 40 may select the target memory page from the free physical memory pages. And the number of the target memory pages is the same as that of the virtual addresses to be recovered in the current batch. Further, the memory management module 40 may map the target memory page to a virtual address space corresponding to the process space identifier of the pass-through device 102. Specifically, the memory management module 40 may write the correspondence between the physical address of the target memory page and the virtual address to be recovered in the current batch into the page table corresponding to the process space identifier of the pass-through device 102 in the IOMMU104, so as to implement the missing page recovery in the current batch.

The missing page request driver 201 may asynchronously request the memory management module 40 to recover the missing memory page according to the foregoing embodiment until the missing memory page is recovered. Further, as shown in FIG. 4, the page fault request driver 201 can asynchronously return an identification of the recovered memory page to the page fault request service component 102 a. In this way, the pass-through device 102 may asynchronously access the recovered memory pages based on the identity of the recovered memory pages. Specifically, the pass-through device 102 may asynchronously access the recovered memory page and obtain the stored data from the memory page through a DMA (corresponding to steps 7 and 11 … x4 in fig. 5).

In the computing system provided in the embodiment of the present application, the missing page request service component is added on the side of the direct connection device, and the missing page request driver is added on the virtual machine management node, where the missing page request service component and the missing page request driver are mutually matched, so as to implement asynchronous recovery of a missing page in a large block IO memory corresponding to a large block of data, and the direct connection device 102 may asynchronously access a recovered memory page. Compared with the existing synchronous page fault request recovery mode, the method does not need to wait for all the memory pages corresponding to the large data blocks to be recovered, is beneficial to reducing the IO memory page fault recovery time length, and further reduces the memory access delay of the direct connection equipment.

Moreover, the embodiment of the application can realize asynchronous recovery of a plurality of memory pages by one-time interruption, can reduce the number of page-missing interruptions, and is beneficial to improving the IO performance.

For the embodiment that the page-missing request driver 201 asynchronously requests the memory management module 40 to perform batch recovery on the missing memory pages, as shown in fig. 5, the page-missing request driver 201 asynchronously returns, to the page-missing request service component 102a, an identifier of the memory page that is recovered in the current batch under the condition that the memory page corresponding to the virtual address to be recovered in the current batch is completely recovered. The pass-through device 102 may asynchronously access the memory page corresponding to the virtual address to be recovered in the current batch according to the identifier of the memory page recovered in the current batch. Specifically, the pass-through device 102 may asynchronously access the memory page corresponding to the virtual address to be recovered in the current batch through a DMA (corresponding to steps 7 and 11 … x4 in fig. 5). Optionally, the pass-through device 102 may perform DMA access to the memory page corresponding to the virtual address to be recovered in the current batch through the DMA controller 50. The DMA controller 50 is a unique peripheral that transfers data within the system, and can be considered as a controller that can connect internal and external memory to each DMA capable peripheral over a set of dedicated buses.

In the embodiment of the present application, the specific implementation form that the page fault request driver 201 asynchronously returns the identifier of the recovered memory page of the current batch to the page fault request service component 102a is not limited. In some embodiments, if the current batch is not the last batch, the page fault request driver 201 may determine that the virtual address to be recovered in the current batch is the identifier of the memory page recovered in the current batch after the memory page corresponding to the virtual address to be recovered in the current batch is successfully recovered; and asynchronously returns a PARTIAL memory PAGE restore success message (PARTIAL PAGE RESPONSE, PPR) to the PAGE fault request component (corresponding to steps 6, 10, etc. of fig. 5). The partial memory page recovery success message; the partial memory page recovery success message includes: and the virtual address to be recovered in the current batch. Further, the pass-through device 102 may asynchronously access the memory page corresponding to the virtual address to be recovered in the current batch in a DMA manner.

In other embodiments, the current batch is the last batch, and the page fault request driver 201 may determine that the virtual address to be recovered in the last batch is the identifier of the memory page recovered in the current batch after the memory page corresponding to the virtual address to be recovered in the last batch is successfully recovered; and asynchronously returns a full memory PAGE restore success message (PAGE _ RESPONSE _ complete, PRC) to the PAGE fault request service component 102a (corresponding to step x3, etc. of fig. 5). The all memory page recovery success message includes: and determining the virtual address to be recovered in the last batch. The successful recovery message of all memory pages indicates that the recovery of the page fault of the asynchronous large-block memory is completed. Further, the pass-through device 102 may asynchronously access the memory page corresponding to the virtual address to be recovered in the last batch in the DMA manner.

The method can realize asynchronous recovery of the lost memory pages aiming at page missing recovery of the large-block memory, and compared with the existing synchronous page missing request recovery mode, the method can realize asynchronous access to the memory pages recovered in the current batch without waiting for all the memory pages corresponding to the large-block data to be recovered, is beneficial to reducing the page missing recovery time of the IO memory, and further reduces the memory access delay of the direct-connection equipment.

In addition to the computing system provided in the foregoing embodiment, an embodiment of the present application also provides a memory page fault processing method. The following describes an example of a memory page fault processing method provided in the embodiment of the present application.

Fig. 6 is a flowchart illustrating a method for processing a page fault in a memory according to an embodiment of the present disclosure. As shown in fig. 6, the method for processing a page fault in a memory includes:

601. and acquiring the page missing request sent by the page missing request service component in the direct connection equipment.

602. And according to the page missing request, the asynchronous request memory management module recovers the missing memory page.

603. And asynchronously returning the identifier of the recovered memory page to the page fault request service assembly, so that the direct connection device asynchronously accesses the recovered memory page according to the identifier of the recovered memory page.

In this embodiment, in order to reduce the duration of IO missing page recovery and improve the efficiency of IO memory missing page recovery, a missing page request service component 102a is added in the pass-through device; and adding a missing page request driver corresponding to the missing page request service component in the VMM. The present embodiment mainly describes a method for processing a page missing request provided in the present embodiment from the perspective of page missing request driving. In this embodiment, for the descriptions of the missing page request service component and the missing page request driver, reference may be made to the related contents of the above system embodiments, and details are not described herein again.

In step 601, a page fault request issued by a page fault request service component in the pass-through device can be obtained. The page fault request service component can send a page fault request to the page fault request driver under the condition that the memory page accessed by the direct connection device is lost. The page fault request may be an asynchronous page fault request. For a specific implementation of determining that the memory page accessed by the pass-through device is missing, reference may be made to relevant contents of the foregoing system embodiment, and details are not described herein again.

Next, in step 602, the memory management module 40 may asynchronously request the memory page with a missing page to be recovered according to the page missing request. The memory management module may be a memory management module of an Operating System (OS) of the virtual machine, and may manage and allocate the memory.

In some embodiments, the asynchronous request memory management module may perform batch recovery on the missing memory pages according to the page missing request.

Specifically, the number of the missed memory pages and the process space identifier corresponding to the pass-through device can be obtained from the page missing request; and circularly executing the following steps until the number of the recovered memory pages is equal to the number of the missing memory pages, namely the missing memory pages are completely recovered. Wherein, the step of circulating execution includes:

determining a virtual address to be recovered in a current batch; an asynchronous request memory management module allocates a target memory page for a virtual address to be recovered; and writing the corresponding relation between the physical address of the target memory page and the virtual address to be recovered in the current batch into a page table corresponding to the process space identifier in the IOMMU. And the page table corresponding to the process space identifier is the process space identifier of the direct connection device contained in the page missing request.

In some embodiments, the current lot may be the first lot. Aiming at the first batch, the initial virtual address of the missing memory page can be obtained from the missing page request; and determining the virtual addresses to be recovered in the first batch according to the initial virtual address.

In other embodiments, the current lot may be any lot other than the first lot. For any batch except the first batch, the virtual address to be recovered in any batch can be determined according to the virtual address corresponding to the memory page recovered in the previous batch of any batch.

For any batch, the memory management module can asynchronously request to allocate a target memory page for the virtual address to be recovered. For a specific implementation of the memory management module allocating the target memory page for the virtual address to be recovered, reference may be made to the related contents of the above system embodiment, which is not described herein again.

The memory management module may be configured to recover the missing memory page according to the above embodiment, until the recovery of the missing memory page is completed. Further, for the recovered memory page, in step 603, the identifier of the recovered memory page may be asynchronously returned to the missing page request service component. In this way, the pass-through device may asynchronously access the recovered memory pages based on the identity of the recovered memory pages. Specifically, the pass-through device may asynchronously access the recovered memory page in a DMA manner and retrieve the stored data from the memory page.

In this embodiment, a missing page request service component is added on the side of the pass-through device, a missing page request driver is added on the virtual machine management node, and the missing page request service component and the missing page request driver cooperate with each other to implement asynchronous recovery of a missing page of a large block IO memory corresponding to a large block of data, and the pass-through device can asynchronously access a recovered memory page. Compared with the existing synchronous page fault request recovery mode, the method does not need to wait for all memory pages corresponding to the large data blocks to be recovered, is beneficial to reducing the recovery time of IO memory page fault, and further reduces the memory access delay of the direct connection equipment.

For the embodiment in which the asynchronous request memory management module performs batch recovery on the missing memory pages, when the recovery of the memory pages corresponding to the virtual address to be recovered in the current batch is completed, the identifier of the recovered memory pages in the current batch can be asynchronously returned to the missing page request service component. The straight-through device can asynchronously access the memory pages corresponding to the virtual addresses to be restored in the current batch according to the identifier of the memory pages restored in the current batch.

In the embodiment of the present application, a specific implementation form of asynchronously returning the identifier of the recovered memory page of the current batch to the page fault request service component is not limited. In some embodiments, if the current batch is not the last batch, after the memory page corresponding to the virtual address to be recovered in the current batch is successfully recovered, determining that the virtual address to be recovered in the current batch is the identifier of the memory page recovered in the current batch; and asynchronously returns a PARTIAL memory PAGE recovery success message (portion _ PAGE _ RESPONSE, PPR) to the missing PAGE request component. The partial memory page recovery success message; the partial memory page recovery success message includes: and the virtual address to be recovered in the current batch. Further, the pass-through device may asynchronously access the memory page corresponding to the virtual address to be restored in the current batch in a DMA manner.

In other embodiments, the current batch is the last batch, and the virtual address to be recovered in the last batch is determined to be the identifier of the memory page recovered in the current batch after the memory page corresponding to the virtual address to be recovered in the last batch is successfully recovered; and asynchronously returns a full memory PAGE restore success message (PAGE _ RESPONSE _ complete, PRC) to the PAGE fault request service component. The all memory page recovery success message includes: and determining the virtual address to be recovered in the last batch. The successful recovery message of all memory pages indicates that the recovery of the page fault of the asynchronous large-block memory is completed. Further, the pass-through device may asynchronously access the memory page corresponding to the virtual address to be recovered in the last batch in a DMA manner.

It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subject of

steps

601 and 602 may be device a; for another example, the execution subject of step 601 may be device a, and the execution subject of step 602 may be device B; and so on.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations such as 602, 603, etc. are merely used for distinguishing different operations, and the sequence numbers themselves do not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the steps in the above-mentioned memory page fault processing method.

In the embodiments of the present application, the processing unit may be any hardware processing device that can execute the logic of the above method. Alternatively, the processor may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or a Micro Controller Unit (MCU); programmable devices such as Field-Programmable Gate arrays (FPGAs), Programmable Array Logic devices (PALs), General Array Logic devices (GAL), Complex Programmable Logic Devices (CPLDs), etc. may also be used; or Advanced Reduced Instruction Set (RISC) processors (ARM), or System On Chips (SOC), etc., but is not limited thereto.

It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

The storage medium of the computer is a readable storage medium, which may also be referred to as a readable medium. Readable storage media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A computing system, comprising: a host machine and a virtual machine management node; the host machine is provided with a virtual machine and is hung with a direct-connection device for the direct connection of the virtual machine; the operating system of the virtual machine comprises a memory management module;

the missing page request driver is used for asynchronously requesting the memory management module to recover the missing memory pages according to the missing page request; asynchronously returning the identifier of the recovered memory page to the missing page request service assembly;

2. The system according to claim 1, wherein the missing page request driver, when asynchronously requesting the memory management module to recover the missing memory page, is specifically configured to:

and asynchronously requesting the memory management module to recover the missing memory pages in batches according to the missing page request.

3. The system of claim 2, wherein the host further comprises: an IOMMU; the page table stored by the IOMMU records the corresponding relation between the virtual address and the physical address; the physical address in the corresponding relation is the physical address of the memory page of the memory of the host machine; the direct connection equipment accesses the memory of the host machine based on the corresponding relation;

the missing page request driver is specifically configured to, when asynchronously requesting the memory management module to recover the missing memory pages in batches:

acquiring the number of the missed memory pages and a process space identifier corresponding to the pass-through device from the page missing request; and circularly executing the following steps until the number of the recovered memory pages is equal to the number of the missing memory pages; wherein, the step of circulating execution includes:

determining a virtual address to be recovered in a current batch; asynchronously requesting the memory management module to allocate a target memory page to the virtual address to be recovered;

and writing the corresponding relationship between the physical address of the target memory page and the virtual address to be restored in the current batch into a page table corresponding to the process space identifier in the IOMMU.

4. The system of claim 3, wherein the pass-through device initiates a memory access request; and triggering the missing page request service component to send a missing page request to the missing page request driver when the page table does not have a physical address corresponding to the virtual address contained in the memory access request.

5. The system of claim 4, wherein the pass-through device queries the IOTLB based on a virtual address contained in the memory access request; when the IOTLB does not inquire a physical address corresponding to a virtual address contained in the memory access request, the pass-through device sends an address translation request to the IOMMU; the address translation request comprises a virtual address contained in the memory access request;

the IOMMU inquires the corresponding relation between the virtual address and the physical address; under the condition that the physical address corresponding to the virtual address contained in the memory access request is not inquired in the corresponding relation, returning an address translation result reflecting the page fault of the memory to the pass-through equipment;

and the pass-through equipment determines that the page table does not have a physical address corresponding to the virtual address contained in the memory access request based on the address translation result.

6. The system according to any one of claims 1-5, wherein the page fault request drives a processing unit running on the host; the processing unit of the host is in communication connection with the page missing request service component;

the page missing request service component sends an interrupt signal to the processing unit through a communication link between the page missing request service component and the processing unit;

the processing unit responds to the interrupt signal and wakes up the missing page request driver.

7. A method for processing a page fault in a memory is characterized by comprising the following steps:

8. The method according to claim 7, wherein said asynchronously requesting, according to the page fault request, the memory management module to recover the missing memory page comprises:

9. The method according to claim 8, wherein the asynchronously requesting, according to the page fault request, the memory management module to perform batch recovery on the missing memory pages comprises:

and writing the corresponding relation between the physical address of the target memory page and the virtual address to be recovered of the current batch into a page table corresponding to the process space identifier in the IOMMU.

10. The method of claim 9, wherein determining the virtual address of the current batch to be restored comprises:

for a first batch, acquiring a starting virtual address of the missing memory page from the missing page request; determining the starting virtual address as the first batch of virtual addresses to be recovered;

and for any batch except the first batch, determining the virtual address to be recovered of any batch according to the virtual address corresponding to the memory page recovered from the previous batch of any batch.

11. The method according to claim 9, wherein the asynchronously returning the identifier of the recovered memory page to the missing page request service component comprises:

if the current batch is not the last batch, after the memory page corresponding to the virtual address to be recovered in the current batch is successfully recovered, determining that the virtual address to be recovered in the current batch is the identifier of the memory page recovered in the current batch;

asynchronously returning a part of memory page recovery success message to the missing page request component; the partial memory page recovery success message includes: and the virtual address to be restored in the current batch is used for the pass-through device to asynchronously access the memory page corresponding to the virtual address to be restored in the current batch in a direct memory access mode according to the virtual address to be restored in the current batch.

12. The method according to claim 9, wherein the asynchronously returning the identifier of the recovered memory page to the missing page request service component comprises:

if the current batch is the last batch, after the memory page corresponding to the virtual address to be recovered in the last batch is successfully recovered, determining that the virtual address to be recovered in the last batch is the identifier of the memory page recovered in the current batch; asynchronously returning all memory pages to the missing page request service component to recover the success message; the all memory page recovery success message includes: and determining the virtual address to be restored in the last batch, so that the pass-through device asynchronously accesses the memory page corresponding to the virtual address to be restored in the last batch in a direct memory access manner according to the virtual address to be restored in the last batch.

13. A computer-readable storage medium having stored thereon computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the method of any one of claims 6-12.