CN104049905A - Migrating pages of different sizes between heterogeneous processors - Google Patents

Abstract

One embodiment of the invention proposes a computer-implemented method for migrating memory pages from a first memory to a second memory. The method includes: determining a first page size supported by the first memory. The method also includes determining a second page size supported by the second memory. The method further includes determining a usage history of the memory page based on an entry in a page state directory associated with the memory page. The method also includes migrating the memory page between the first memory and the second memory based on the first page size, the second page size, and the usage history.

Description

Migrate pages of different sizes between heterogeneous processors

technical field

The present invention relates generally to computer science, and more particularly to migrating pages of different sizes between heterogeneous processors.

Background technique

A typical computer system includes a central processing unit (CPU) and one or more parallel processing units (GPU). Some advanced computer systems implement a unified virtual memory architecture shared by the CPU and GPU. Among other things, the architecture enables the CPU and GPU to use a common (eg, same) virtual memory address to access a physical memory location, whether that physical memory location is in system memory or GPU-local memory.

In such a unified virtual memory architecture, memory pages may advantageously be sized differently depending on whether the memory page is stored in a memory unit associated with a CPU or a GPU. One disadvantage of having memory pages of different sizes is that it becomes more complicated to migrate memory pages between those different memory cells. For example, one challenge that can arise is migrating large memory pages to memory cells that only store small memory pages. In this case, the unified virtual memory architecture must decide how to tolerate this difference in page size.

As previously stated, what is needed in the art is a more efficient method for migrating memory pages of different sizes in systems implementing a unified virtual memory architecture.

Contents of the invention

One embodiment of the invention proposes a computer-implemented method for migrating memory pages from a first memory to a second memory. The method includes: determining a first page size supported by the first memory. The method also includes determining a second page size supported by the second memory. The method further includes determining a usage history of the memory page based on an entry in a page state directory associated with the memory page. The method also includes migrating the memory page between the first memory and the second memory based on the first page size, the second page size, and the usage history.

One advantage of the disclosed technique is that memory pages of different sizes can be efficiently migrated back and forth between different memory units in a virtual memory architecture. This technology increases the flexibility of the unified virtual memory system by allowing it to work with many different types of memory architectures. Another related advantage is that, by allowing large memory pages to be split into smaller memory pages and allowing small memory pages to be merged into larger memory pages, memory pages having different sizes can be stored in different memories configured to store different memory page sizes. in the unit. This feature allows the unified virtual memory system to group pages where possible in order to reduce the amount of space taken up in page tables and/or translation lookaside buffers (TLBs). This feature also allows memory pages to be split and migrated to different memory units, as long as such splitting will improve memory locality and reduce memory access time.

Description of drawings

So that the above recited features of the present invention can be understood in detail, and a more particular description of the invention, briefly summarized above, may be had by reference to exemplary embodiments, some of which are shown in the accompanying drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may have other equally effective embodiments.

Figure 1 is a block diagram illustrating a computer system configured to implement one or more aspects of the present invention;

Figure 2 is a block diagram illustrating a unified virtual memory system (UVM), according to one embodiment of the present invention;

Figure 3 illustrates operations for transferring a small memory page from system memory to PPU memory, according to one embodiment of the present invention;

Figure 4 illustrates operations for transferring small memory pages and associated "sibling" memory pages from system memory to PPU memory, according to one embodiment of the present invention;

Figure 5 illustrates operations for transferring a small memory page from PPU memory to system memory, according to one embodiment of the present invention;

FIG. 6 illustrates operations for transferring small memory pages and sibling memory pages from PPU memory 204 to system memory 104, according to one embodiment of the invention; and

7 is a flowchart of method steps for migrating memory pages of different sizes between memory units in a virtual memory architecture, according to one embodiment of the present invention.

Detailed ways

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without one or more of these specific details.

System Overview

FIG. 1 is a block diagram illustrating a computer system 100 configured to implement one or more aspects of the present invention. Computer system 100 includes a central processing unit (CPU) 102 and system memory 104 in communication via an interconnection path that may include a memory bridge 105 . The memory bridge 105 may be, for example, a north bridge chip, connected to an I/O (input/output) bridge 107 via a bus or other communication path 106 (eg, a HyperTransport link). I/O bridge 107 , which may be, for example, a south bridge chip, receives user input from one or more user input devices 108 (eg, keyboard, mouse) and forwards the input to CPU 102 via communication path 106 and memory bridge 105 . Parallel processing subsystem 112 is connected to memory bridge 105 via a bus or second communication path 113 such as Peripheral Component Interconnect (PCI) Express, Accelerated Graphics Port, or HyperTransport Link; in one embodiment, parallel processing subsystem 112 is the graphics subsystem that transfers the pixels to the display device 110, which may be any conventional cathode ray tube, liquid crystal display, light emitting diode display, or the like. System disk 114 may also be connected to I/O bridge 107 and may be configured to store applications and data and content used by CPU 102 and parallel processing subsystem 112 . System disk 114 provides non-volatile storage space for applications and data, and may contain fixed or removable hard drives, flash drives, and CD-ROM (Compact Disc Read-Only Memory), DVD-ROM (Digital Versatile Disc -ROM), Blu-ray, HD-DVD (high-resolution DVD), or other magnetic, optical, or solid-state storage devices.

Switch 116 provides connections between I/O bridge 107 and other components such as network adapter 118 and various add-in cards 120 and 121 . Other components (not explicitly shown), including Universal Serial Bus (USB) or other port connections, compact disc (CD) drives, digital versatile disc (DVD) drives, film recording devices, and similar components, may also be connected to the I/O O bridge 107. The various communication paths shown in Figure 1, including specifically named communication paths 106 and 113, may be implemented using any suitable protocol, such as PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol , and as known in the art, connections between different devices may use different protocols.

In one embodiment, parallel processing subsystem 112 includes circuitry optimized for graphics and video processing, including, for example, video output circuitry, and constitutes one or more parallel processing units (PPUs) 202 . In another embodiment, the parallel processing subsystem 112 includes circuitry optimized for general-purpose processing while preserving the underlying computing architecture, as described in greater detail herein. In yet another embodiment, parallel processing subsystem 112 may be integrated with one or more other system elements in a single subsystem, such as combining memory bridge 105, CPU 102, and I/O bridge 107, to form a system-on-chip (SoC) . It is well known that many graphics processing units (GPUs) are designed to perform parallel operations and calculations, and thus are considered a type of parallel processing unit (PPU).

Any number of PPUs 202 may be included in parallel processing subsystem 112 . For example, multiple PPUs 202 may be provided on a single add-in card, or multiple add-in cards may be connected to communication path 113, or one or more PPUs 202 may be integrated into a bridge chip. PPUs 202 in a multiple PPU system may be the same or different from each other. For example, different PPUs 202 may have different numbers of processing cores, different amounts of local parallel processing memory, and so on. Where multiple PPUs 202 are present, those PPUs may be operated in parallel to process data at a higher throughput than is possible with a single PPU 202 . Systems incorporating one or more PPUs 202 can be implemented in a variety of configurations and form factors, including desktop, notebook or handheld personal computers, servers, workstations, game consoles, embedded systems, and the like.

PPU 202 advantageously implements a highly parallel processing architecture. PPU 202 includes a number of general purpose processing clusters (GPCs). Each GPC is capable of concurrently executing a large number (eg, hundreds or thousands) of threads, where each thread is an instance of a program. In some embodiments, single instruction, multiple data (SIMD) instruction issue techniques are used to support parallel execution of a large number of threads without providing multiple independent instruction units. In other embodiments, single instruction, multiple thread (SIMT) technology is used to support parallel execution of a large number of generally simultaneous threads using a common instruction unit configured to issue instructions to a set of processing engines within each of the GPCs 208 . Unlike the SIMD execution mechanism where all processing engines typically execute the same instructions, SIMT execution allows different threads to more easily follow distributed execution paths through a given thread program.

A GPU includes a large number of streaming multiprocessors (SMs), where each SM is configured to process one or more thread groups. A series of instructions delivered to a particular GPC 208 constitutes a thread, and a collection of some number of concurrently executing threads across parallel processing engines (not shown) within an SM is referred to herein as a "warp" or "thread group." ". As used herein, a "thread group" refers to a group of threads that concurrently execute the same program on different input data, one thread of the group being assigned to a different processing engine within the SM. Additionally, multiple related thread groups can be active within an SM concurrently (at different stages of execution). This collection of thread groups is referred to herein as a "cooperative thread array" ("CTA") or "thread array."

In embodiments of the present invention, it may be desirable to use a computing system's PPU 202 or other processor to perform general-purpose computations using an array of threads. Each thread in the thread array is assigned a unique thread identifier ("thread ID") that is accessible to the thread during its execution. Thread IDs, which can be defined as one-dimensional or multi-dimensional values, control aspects of thread processing behavior. For example, the thread ID can be used to determine which portion of an input data set a thread is to process and/or to determine which portion of an output data set a thread is to produce or write.

In operation, CPU 102 is the main processor of computer system 100, controlling and coordinating the operation of other system components. Specifically, CPU 102 issues commands to control the operation of PPU 202 . In one embodiment, communication path 113 is a PCI Express link, as is known in the art, where a dedicated lane is assigned to each PPU 202. Other communication paths may also be used. PPU 202 advantageously implements a highly parallel processing architecture. The PPU 202 can be equipped with any amount of local parallel processing memory (PPU memory).

In some embodiments, system memory 104 includes a unified virtual memory (UVM) driver 101 . The UVM driver 101 includes instructions for performing various tasks related to the unified virtual memory (UVM) shared by the CPU 102 and the PPU 202 . Among other things, the architecture enables CPU 102 and PPU 202 to use common virtual memory addresses to access physical memory locations, whether within system memory 104 or in memory local to PPU 202 (PPU memory).

It should be understood that the systems shown herein are exemplary and that variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of CPUs 102, and the number of parallel processing subsystems 112, can be modified as desired. For example, in some embodiments, system memory 104 is connected directly to CPU 102 rather than through a bridge, and other devices communicate with system memory 104 via memory bridge 105 and CPU 102 . In other alternative topologies, parallel processing subsystem 112 is connected to I/O bridge 107 or directly to CPU 102 instead of memory bridge 105 . Yet in other embodiments, I/O bridge 107 and memory bridge 105 may be integrated onto a single chip rather than exist as one or more discrete devices. Larger embodiments may include two or more CPUs 102 and two or more parallel processing systems 112 . Certain components shown herein are optional; for example, any number of add-in cards or peripherals may be supported. In some embodiments, switch 116 is eliminated and network adapter 118 and add-in cards 120 , 121 are directly connected to I/O bridge 107 .

Unified Virtual Memory System Architecture

FIG. 2 is a block diagram illustrating a unified virtual memory (UVM) system 200 according to one embodiment of the present invention. As shown, the unified virtual memory system 200 includes, without limitation, a CPU 102 , a system memory 104 , and a parallel processing unit (PPU) 202 connected to a parallel processing unit memory (PPU memory) 204 . CPU 102 and system memory 104 are connected to each other and to PPU 202 via memory bridge 105 .

CPU 102 executes threads that may request data stored in system memory 104 or PPU memory 204 via virtual memory addresses. From knowledge of the inner workings of the memory system, virtual memory addresses shield threads that are executing in CPU 102 . Thus, a thread may only know the virtual memory address, and may access data by requesting the data through the virtual memory address.

CPU 102 includes CPU MMU 209, which handles requests from CPU 102 for translating virtual memory addresses into physical memory addresses. Accessing data stored in physical memory units such as system memory 104 and PPU memory 204 requires physical memory addresses. CPU 102 includes CPU fault handler 211 that performs steps to make requested data available to CPU 102 in response to CPUMMU 209 generating a page fault. The CPU fault handling program 211 is generally software that resides in the system memory 104 and executes on the CPU 102 , and is woken up by an interrupt to the CPU 102 .

System memory 104 stores various memory pages (not shown) used by threads executing on CPU 102 or PPU 202 . As shown, system memory 104 stores CPU page tables 206, which contain mappings between virtual memory addresses and physical memory addresses. System memory 104 also stores page state directory 210, which acts as a "home table" for UVM system 200, as discussed in more detail below. System memory 104 stores fault buffer 216 containing entries written by PPU 202 to notify CPU 102P of page faults generated by PU 202 . In some embodiments, system memory 104 includes a unified virtual memory (UVM) driver 101 that contains instructions that, when executed, cause CPU 102 to execute commands for repairing page faults, in addition to. In alternative embodiments, any combination of page state directory 210 and one or more command queues 214 may be stored in PPU memory 204 . Additionally, PPU page tables 208 may be stored in system memory 104 .

In a similar manner to CPU 102 , PPU 202 executes instructions that may request data stored in system memory 104 or PPU memory 204 via virtual memory addresses. PPU 202 includes PPU MMU 213, which handles requests from PPU 202 for translating virtual memory addresses into physical memory addresses. PPU 202 also includes copy engine 212 that executes commands stored in command queue 214 for copying memory pages, modifying data in PPU page table 208, and other commands. PPU fault handler 215 performs steps in response to a page fault on PPU 202 . PPU fault handler 215 may be software running on a processor or a dedicated microcontroller in PPU 202 . Alternatively, PPU fault handler 215 may be a combination of software running on CPU 102 and software running on a dedicated microcontroller in PPU 202 in communication with each other. In some embodiments, CPU fault handler 211 and PPU fault handler 215 may be invoked by a fault on either CPU 102 or PPU 202 . Command queue 214 may be in PPU memory 204 or system memory 104 , but is preferably located in system memory 104 .

In some embodiments, CPU fault handler 211 and UVM driver 101 may be a unified software program. In such cases, the unified software program may be software that resides in system memory 104 and executes on CPU 102 . PPU fault handler 215 may be a separate software program running on a processor or a dedicated microcontroller in PPU 202 , or PPU fault handler 215 may be a separate software program running on CPU 102 .

In other embodiments, PPU fault handler 215 and UVM driver 101 may be a unified software program. In such cases, the unified software program may be software that resides in system memory 104 and executes on CPU 102 . CPU fault handler 211 may be software that resides in system memory 104 and executes on CPU 102 .

In other embodiments, the CPU fault handler 211, the PPU fault handler 215, and the UVM driver 101 may be a unified software program. In such cases, the unified software program may be software that resides in system memory 104 and executes on CPU 102 .

In some embodiments, CPU fault handler 211 , PPU fault handler 215 , and UVM driver 101 may all reside in system memory 104 as described above. As shown in FIG. 2 , UVM driver 101 resides in system memory 104 , while CPU fault handler 211 and PPU fault handler 215 reside in CPU 102 .

CPU fault handler 211 and PPU fault handler 215 may respond to hardware interrupts originating from CPU 102 or PPU 202 , such as interrupts due to page faults. As described further below, UVM driver 101 contains instructions for performing various tasks related to the management of UVM system 200, including without limitation repairing page faults and accessing CPU page table 206, page state directory 210, and/or fault cache device 216.

In some embodiments, CPU page table 206 and PPU page table 208 have different formats and contain different information; for example, PPU page table 208 may contain the following information while CPU page table 206 does not: atomic disable bit (atomic disable bit); compression tag; and memory swizzling type.

In a similar manner to system memory 104 , PPU memory 204 stores various pages (not shown). As shown, PPU memory 204 also includes a PPU page table 208 that contains a mapping between virtual memory addresses and physical memory addresses. Alternatively, PPU page table 208 may be stored in system memory 104 .

translate virtual memory address

When a thread executing in the CPU 102 requests data via a virtual memory address, the CPU 102 requests translation of the virtual memory address into a physical memory address from the CPU memory management unit (CPU MMU) 209 . In response, CPU MMU 209 attempts to translate the virtual memory address into a physical memory address specifying a location in a memory unit, such as system memory 104, where the data requested by CPU 102 is stored.

To translate a virtual memory address to a physical memory address, CPU MMU 209 performs a lookup operation to determine whether CPU page table 206 contains a mapping associated with the virtual memory address. In addition to virtual memory addresses, requests to access data may also specify a virtual memory address space. The unified virtual memory system 200 can implement multiple virtual memory address spaces, each space is assigned one or more threads. Virtual memory addresses are unique within any given virtual memory address space. Furthermore, virtual memory addresses within a given virtual memory address are contiguous across CPU 102 and PPU 202 , allowing the same virtual memory address to point to the same data across CPU 102 and PPU 202 . In some embodiments, two virtual memory addresses may point to the same data, but may not map to the same physical memory address (eg, CPU 102 and PPU 202 may each have a local read-only copy of the data).

For any given virtual memory address, CPU page table 206 may or may not contain a mapping between the virtual memory address and the physical memory address. If CPU page table 206 contains a mapping, CPU MMU 209 reads the mapping to determine the physical memory address associated with the virtual memory address and provides the physical memory address to CPU 102. However, if CPU page table 206 does not contain a mapping associated with the virtual memory address, then CPU MMU 209 cannot translate the virtual memory address into a physical memory address, and CPU MMU 209 generates a page fault. To repair the page fault and make the requested data available to CPU 102, a "page fault sequence" is executed. More specifically, CPU 102 reads PSD 210 to find out the current mapping state of the page and then determines the appropriate page fault sequence. The page fault sequence typically maps the memory page associated with the requested virtual memory address or changes the type of access granted (eg, read access, write access, atomic access). The different types of page fault sequences implemented in UVM system 200 are discussed in more detail below.

Within UVM system 200, data associated with a given virtual memory address may be stored in system memory 104, PPU memory 204, or both system memory 104 and PPU memory 204 as a read-only copy of the same data. Furthermore, for any such data, either or both CPU page table 206 and PPU page table 208 may contain a mapping associated with the data. Note that there is some data for which a mapping exists in one page table but not another. However, PSD 210 contains all mappings stored in PPU page table 208 as well as PPU-related mappings stored in CPU page table 206 . PSD 210 thus serves as the “master” page table for unified virtual memory system 200 . Thus, when CPU MMU 209 does not find a mapping in CPU page table 206 associated with a particular virtual memory address, CPU 102 reads PSD 210 to determine whether PSD 210 contains a mapping associated with that virtual memory address. Various embodiments of PSD 210 may contain different types of information associated with virtual memory addresses in addition to the mappings associated with virtual memory addresses.

When the CPU MMU 209 generates a page fault, the CPU fault handler 211 performs a series of operations for the appropriate page fault sequence to repair the page fault. Also, during the page fault sequence, CPU 102 reads PSD 210 and performs additional operations to change mappings or permissions within CPU page table 206 and PPU page table 208 . Such operations may include reading and/or modifying CPU page tables 206, reading and/or modifying page state directories 210, and/or migrating between memory units (e.g., system memory 104 and PPU memory 204) are referred to as " memory page" block of data.

To determine which operations are to be performed in the page fault sequence, CPU 102 identifies the memory page associated with the virtual memory address. CPU 102 then reads status information about the memory page from PSD 210 associated with the virtual memory address associated with the memory access request that caused the page fault. Such state information may additionally include an ownership state regarding the memory page associated with the virtual memory address. For any given memory page, several ownership states are possible. For example, a memory page may be "CPU-owned", "PPU-owned", or "CPU-shared". A memory page is considered owned by a CPU if the CPU 102 can access the memory page via a virtual address without incurring a page fault, and if the PPU 202 cannot access the memory page via a virtual address without incurring a page fault. Preferably, all pages of the CPU reside in system memory 104 , but may also reside in PPU memory 204 . A memory page is considered owned by the PPU if the PPU 202 can access the page via the virtual address, and if the CPU 102 cannot access the page via the virtual address without incurring a page fault. Preferably, all pages of the PPU reside in PPU memory 204, but may also reside in system memory 104 when migration from system memory 104 to PPU memory 204 is not performed. Finally, memory is considered CPU shared if CPU 102 and PPU 202 can access a memory page via a virtual address without causing a page fault. CPU shared pages may reside in either system memory 104 or PPU memory 204 .

CPU page table 206 may assign an ownership state to a memory page based on various factors, including the usage history of the memory page. The usage history may include information about whether CPU 102 or PPU 202 has recently accessed a page of memory and how many times such access was made. For example, if based on the usage history of a given memory page, the UVM system 200 determines that the memory page is likely to be used primarily or exclusively by the CPU 102, the UVM system 200 can assign the memory page an ownership status of "CPU Owned" and assign the memory page Pages are placed in system memory 104 . Similarly, if based on the usage history of a given memory page, UVM system 200 determines that the memory page is likely to be used primarily or exclusively by PPU 202, UVM system 200 may assign the memory page an ownership status of "PPU Owned" and assign This page is placed in PPU memory 204 . Finally, if based on the usage history of a given memory page, UVM system 200 determines that the memory page is likely to be used by both CPU 102 and PPU 202, and determines that migrating the memory page from system memory 104 to PPU memory 204 and back would be too costly time, the UVM system 200 may assign the memory page an ownership status of "CPU Shared".

As an example, fault handlers 211 and 215 may implement any or all of the following heuristics for migration:

(a) with respect to CPU 102 accesses to unmapped (unmapped) pages that are mapped to PPU 202 and have not been migrated recently, unmap the faulty page from PPU 202, migrate the page to CPU 102, and map the page to CPU 102;

(b) with respect to PPU 202 accesses to unmapped pages that are mapped to CPU 102 and have not been migrated recently, unmap the faulty page from CPU 102, migrate the page to PPU 202, and map the page to PPU 202;

(c) with respect to CPU 102 accesses to the most recently migrated unmapped page mapped to PPU 202 , migrate the faulted page to CPU 102 and map the page on both CPU 102 and PPU 202 ;

(d) for a PPU 202 access to a most recently migrated unmapped page mapped on the CPU 102, map the page to both the CPU 102 and the PPU 202;

(e) with respect to a PPU 202 atomic access to a page mapped to both CPU 102 and PPU 202 but not enabled for atomic operations by PPU 202, unmap the page from CPU 102 and map to PPU 202 with atomic operations enabled;

(f) For PPU 202 write access to a page mapped on CPU 102 and PPU 202 as copy-on-write (COW), copy the page to PPU 202, thereby making an independent copy of the page, set The new page is mapped on the PPU as a read-write (read-write), and the current page is reserved on the CPU102;

(g) For PPU 202 read accesses to pages mapped on CPU 102 and PPU 202 as zero-fill-on-demand (ZFOD), a physical memory page on PPU 202 is allocated and filled with zeros, and the The page is mapped on the PPU, but changed to be unmapped on the CPU102;

(h) with respect to an access by the first PPU 202(1) to an unmapped page that is mapped on the second PPU 202(2) and has not been migrated recently, unmap the failed page from the second PPU 202(2), migrating the page to the first PPU 202(1), and mapping the page to the first PPU 202(1); and

(i) With respect to an access by the first PPU 202(1) to an unmapped page that was mapped on the second PPU 202(2) and was most recently migrated, map the failed page to the first PPU 202(1), and keep The mapping of this page on the second PPU 202(2).

In conclusion, many heuristic rules are possible, and the scope of the invention is not limited to these examples.

Also, any migration heuristic can be "rounded up" to include more pages or larger page sizes, for example:

(j) with respect to CPU 102 accesses to unmapped pages that are mapped to PPU 202 and have not been migrated recently, unmap the failed page plus additional pages adjacent to the failed page in virtual address space from PPU 202, Migrate these pages to CPU 102 and map these pages to CPU 102 (in a more detailed example: for a 4kB fault page, migrate the aligned 64kB region containing the 4kB fault page);

(k) with respect to PPU 202 accesses to unmapped pages that are mapped to CPU 102 and have not been migrated recently, unmap the faulted page plus additional pages adjacent to the faulted page in virtual address space from CPU 102, migrate these pages to PPU 202, and map these pages to PPU 202 (in a more detailed example: for a 4kB fault page, migrate the aligned 64kB region containing the 4kB fault page);

(l) with respect to CPU 102 accesses to unmapped pages that are mapped to PPU 202 and have not been migrated recently, unmap the failed page plus additional pages adjacent to the failed page in virtual address space from PPU 202, Migrate these pages to CPU102, map these pages to CPU102, and treat all migrated pages as one or more larger pages on CPU102 (in a more detailed example: for 4kB faulted pages, migration contains 4kB faults the aligned 64kB region of the page, and treat the aligned 64kB region as a 64kB page);

(m) with respect to PPU 202 accesses to unmapped pages that are mapped to CPU 102 and have not been migrated recently, unmap the faulted page plus additional pages adjacent to the faulted page in virtual address space from CPU 102 , Migrate these pages to PPU202, map these pages to PPU202, and treat all migrated pages as one or more larger pages on PPU202 (in a more detailed example: for 4kB faulted pages, migration contains 4kB faults the aligned 64kB region of the page, and treat the aligned 64kB region as a 64kB page);

(n) For an access by the first PPU 202(1) to an unmapped page that is mapped to the second PPU 202(2) and has not been migrated recently, append the faulted page in the virtual address space with the faulted page unmapping additional pages adjacent to the page from the second PPU 202(2), migrating these pages to the first PPU 202(1), and mapping these pages to the first PPU 202(1); and

(o) For an access by the first PPU 202(1) to a recently migrated unmapped page mapped on the second PPU 202(2), append the faulted page in the virtual address space with the faulted page Additional pages adjacent to the page are mapped to the first PPU 202(1), and the mapping of these pages on the second PPU 202(2) is maintained.

In conclusion, many heuristic rules are possible, and the scope of the invention is not limited to these examples.

In some embodiments, PSD entries may contain transitional state information to ensure proper synchronization between various requests made by units within CPU 102 and PPU 202 . For example, a PSD 210 entry may contain transition state information that a particular page is in the process of transitioning from CPU ownership to PPU ownership. Various units in CPU 102 and PPU 202, such as CPU fault handler 211 and PPU fault handler 215, upon determining that a page is in such a transitional state, may forego a partial page fault sequence to avoid a previous fault to the same virtual memory A step in the page fault sequence triggered by a virtual memory access to an address. As a specific example, if a page fault causes a page to be migrated from system memory 104 to PPU memory 204, a different page fault is detected that would cause the same migration and not cause another page migration. Additionally, various units within CPU 102 and PPU 202 may implement atomic operations for proper sequencing of operations on PSD 210 . For example, with respect to modifications to PSD 210 entries, CPU fault handler 211 or PPU fault handler 215 may issue an atomic compare and swap (swap) operation to modify the page state of a particular entry in PSD 210 . Therefore, the modification can be done without interference from the operation of other units.

Multiple PSDs 210 may be stored in system memory 104 - one for each virtual address space. A memory access request generated by either CPU 102 or PPU 202 may thus contain a virtual memory address and also identify a virtual memory address space associated with the virtual memory address.

Just as CPU 102 may execute memory access requests that include virtual memory addresses (ie, instructions that include requests to access data via virtual memory addresses), PPU 202 may execute similar types of memory access requests. More specifically, PPU 202 includes multiple execution units, such as GPCs and SMs, configured to execute multiple threads and thread groups, as described above in connection with FIG. 1 . In operation, those threads may request data from memory (eg, system memory 104 or PPU memory 204 ) by specifying a virtual memory address. Like CPU 102 and CPU MMU 209 , PPU 202 includes a PPU memory management unit (MMU) 213 . PPU MMU 213 receives requests from PPU 202 for virtual memory address translations and attempts to provide translations from PPU page tables 208 for virtual memory addresses.

Similar to CPU page table 206, PPU page table 208 contains a mapping between virtual memory addresses and physical memory addresses. As is the case with CPU page table 206, for any given virtual address, PPU page table 208 may contain no page table entries that map virtual memory addresses to physical memory addresses. Like the CPU MMU 209, when the PPU MMU 213 requests a translation of a virtual memory address from the PPU page table 208 and there is no mapping in the PPU page table 208 or the access type is not allowed by the PPU page table 208, the PPU MMU 213 generates a page fault . Subsequently, the PPU fault handler 215 triggers a page fault sequence. Also, the different types of page fault sequences implemented in UVM system 200 will be described in more detail below.

During a page fault sequence, CPU 102 or PPU 202 may write commands to command queue 214 for execution by replication engine 212 . This approach frees the CPU 102 or PPU 202 to perform other tasks while the replication engine 212 reads and executes commands stored in the command queue 214, and allows all commands for the fault sequence to be queued at the same time, thereby avoiding the progress monitoring. Commands executed by replication engine 212 may additionally include deleting, creating, or modifying page table entries in PPU page table 208 , reading or writing data from system memory 104 , and reading or writing data to PPU memory 204 .

Fault buffer 216 stores fault buffer entries specifying information related to page faults generated by PPU 202 . The fault buffer entry may include, for example, the type of access attempted (e.g., read, write, or atomic), the virtual memory address for which the attempted access caused the page fault, the virtual address space, and the reference to the page that caused the fault. An indication of the failing apartment or thread. In operation, when PPU 202 causes a page fault, PPU 202 may write a fault buffer entry into fault buffer 216 to notify PPU fault handler 215 of the faulted page and the type of access that caused the fault. PPU fault handler 215 then performs actions to repair the page fault. Because PPU 202 is executing multiple threads, fault buffer 216 may store multiple faults, each of which may cause one or more faults due to the pipelined nature of PPU 202's memory accesses.

page fault sequence

As described above, in response to receiving a request for translation of a virtual memory address, CPU MMU 209 generates a page if CPU page table 206 does not contain a mapping associated with the requested virtual memory address or does not grant the type of access being requested. Fault. Similarly, in response to receiving a request for translation of a virtual memory address, PPU MMU 213 generates a page fault if PPU page table 208 does not contain a mapping associated with the requested virtual memory address or does not grant the type of access being requested . When the CPU MMU 209 or the PPU MMU 213 generates a page fault, the thread that requested the data at the virtual memory address stalls, and the "local fault handler"—the CPU fault handler 211 for the CPU 102 or the PPU for the PPU 202 Fault Handler 215 - Attempts to repair page faults by performing a "page fault sequence". As noted above, a page fault sequence includes a series of operations that enable the faulted unit (ie, the unit that caused the page fault—either CPU 102 or PPU 202 ) to access data associated with a virtual memory address. After the page fault sequence ends, the thread that requested the data via the virtual memory address continues to execute. In some embodiments, fault recovery is simplified by allowing the fault recovery logic to track the failing memory access as opposed to the failing instruction.

If there are any changes in ownership status or changes in access permissions that a memory page associated with a page fault has to undergo, the operations performed during the page fault sequence depend on these changes. A transition from a current ownership state to a new ownership state or a change in access permissions may be part of a page fault sequence. In some examples, migrating memory pages associated with page faults from system memory 104 to PPU memory 204 is also part of the page fault sequence. In other examples, migrating memory pages associated with page faults from PPU memory 204 to system memory 104 is also part of the page fault sequence. Various heuristics described more fully herein may be used to configure UVM system 200 to change memory page ownership states or to migrate memory pages according to various sets of operating conditions and patterns. Described in more detail below is the page fault sequence for the following four memory page ownership state transitions: CPU owned to CPU shared, CPU owned to PPU owned, PPU owned to CPU owned, and PPU owned to CPU shared.

A fault generated by PPU 202 may initiate the transition from CPU ownership to CPU sharing. Prior to this transition, a thread executing in PPU 202 attempted to access data at a virtual memory address that was not mapped in PPU page table 208 . This access attempts to cause a PPU-based page fault, which then causes a fault buffer entry to be written to fault buffer 216 . In response, PPU fault handler 215 reads the PSD 210 entry corresponding to the virtual memory address and identifies the memory page associated with the virtual memory address. After reading PSD 210, PPU fault handler 215 determines that the current ownership state of the memory page associated with the virtual memory address is owned by the CPU. Based on the current ownership state and other factors, such as regarding the usage characteristics of the memory page or the type of memory access, the PPU fault handler 215 determines that the new ownership state for the page should be CPU shared.

To change ownership status, PPU fault handler 215 writes a new entry in PPU page table 208 corresponding to the virtual memory address and associating the virtual memory address with the memory page identified via the PSD 210 entry. PPU fault handler 215 also modifies the PSD 210 entry for the memory page to indicate that the ownership status is CPU shared. In some embodiments, a translation look-aside buffer (TLB) in the PPU 202 is invalidated to account for cases where translations to invalid pages are cached. At this point, the page fault sequence ends. The ownership status of the memory page is CPU shared, meaning that the memory page is accessible to both CPU 102 and PPU 202 . Both CPU page table 206 and PPU page table 208 contain entries that associate virtual memory addresses to memory pages.

A fault generated by PPU 202 may initiate a transition from CPU ownership to PPU ownership. Prior to this transition, an operation being executed in PPU 202 attempted to access data at a virtual memory address that was not mapped in PPU page table 208 . This memory access attempts to cause a PPU-based page fault, which then causes a fault buffer entry to be written to fault buffer 216 . In response, PPU fault handler 215 reads the PSD 210 entry corresponding to the virtual memory address and identifies the memory page associated with the virtual memory address. After reading PSD 210, PPU fault handler 215 determines that the current ownership state of the memory page associated with the virtual memory address is owned by the CPU. Based on the current ownership state and other factors, such as regarding the usage characteristics of the page or the type of memory access, the PPU fault handler 215 determines that the new ownership state for the page should be owned by the PPU.

PPU 202 writes a fault buffer entry into fault buffer 216 indicating that PPU 202 generated a page fault and designating the virtual memory address associated with the page fault. PPU fault handler 215 executing on CPU 102 reads the fault register entry, and in response CPU 102 removes the mapping in CPU page table 206 associated with the virtual memory address that caused the page fault. CPU 102 may flush the cache before and/or after removing the mapping. CPU 102 also writes a command into command queue 214 instructing PPU 202 to copy a page from system memory 104 into PPU memory 204 . Copy engine 212 in PPU 202 reads commands in command queue 214 and copies pages from system memory 104 to PPU memory 204 . PPU 202 writes a page table entry into PPU page table 208 corresponding to the virtual memory address and associates the virtual memory address with the newly copied memory page in PPU memory 204 . Writing to PPU page table 208 may be done via PPU 202 . Alternatively, CPU 102 may update PPU page table 208 . The PPU fault handler 215 also modifies the PSD 210 for the memory page to indicate that the ownership status is owned by the PPU. In some embodiments, entries in the TLB in PPU 202 or CPU 102 may be invalidated to account for cases where translations are cached. At this point, the page fault sequence ends. The ownership status of the memory page is owned by the PPU, meaning that the memory page is only accessible to the PPU 202 . Only PPU page table 208 contains an entry associating a virtual memory address with that memory page.

A fault generated by CPU 102 may initiate a transition from PPU ownership to CPU ownership. Prior to this transition, an operation being executed in CPU 102 attempted to access data at a virtual memory address that was not mapped in CPU page table 206, causing a CPU-based page fault. CPU fault handler 211 reads the PSD 210 entry corresponding to the virtual memory address and identifies the memory page associated with the virtual memory address. After reading PSD 210, CPU fault handler 211 determines that the current ownership status of the memory page associated with the virtual memory address is owned by the PPU. Based on the current ownership state and other factors, such as regarding the usage characteristics of the page or the type of access, the CPU fault handler 211 determines that the new ownership state of the page is owned by the CPU.

CPU fault handler 211 changes the ownership state associated with the memory page to CPU owned. CPU fault handler 211 writes a command into command queue 214 to cause replication engine 212 to remove from PPU page table 208 an entry associating a virtual memory address with the memory page. Various TLB entries can be invalidated. CPU fault handler 211 also copies memory pages from PPU memory 204 into system memory 104 , which may be done via command queue 214 and copy engine 212 . CPU fault handler 211 writes a page table entry in CPU page table 206 that associates a virtual memory address with a memory page copied into system memory 104 . CPU fault handler 211 also updates PSD 210 to associate virtual memory addresses with newly copied memory pages. At this point, the page fault sequence ends. The ownership status of the memory page is CPU, meaning that the memory page is only accessible to the CPU 102 . Only CPU page table 206 contains an entry associating a virtual memory address with that memory page.

A fault generated by CPU 102 may initiate the transition from PPU ownership to CPU sharing. Prior to this transition, an operation being executed in CPU 102 attempted to access data at a virtual memory address that was not mapped in CPU page table 206, causing a CPU-based page fault. CPU fault handler 211 reads the PSD 210 entry corresponding to the virtual memory address and identifies the memory page associated with the virtual memory address. After reading PSD 210, CPU fault handler 211 determines that the current ownership status of the memory page associated with the virtual memory address is owned by the PPU. Based on the current ownership state and other factors, such as regarding the usage characteristics of the page, the CPU fault handler 211 determines that the new ownership state of the page is CPU shared.

The CPU fault handler 211 changes the ownership state associated with the memory page to CPU shared. CPU fault handler 211 writes a command into command queue 214 to cause replication engine 212 to remove from PPU page table 208 an entry associating a virtual memory address with the memory page. Various TLB entries can be invalidated. CPU fault handler 211 also copies memory pages from PPU memory 204 into system memory 104 . This copy operation can be accomplished via command queue 214 and copy engine 212 . CPU fault handler 211 then writes a command into command queue 214 to cause replication engine 212 to change an entry in PPU page table 208 so that a virtual memory address is associated with a memory page in system memory 104 . CPU fault handler 211 writes a page table entry to CPU page table 206 to associate a virtual memory address with a memory page in system memory 104 . CPU fault handler 211 also updates PSD 210 to associate virtual memory addresses with memory pages in system memory 104 . At this point, the page fault sequence ends. The ownership status of the page is CPU shared, and the memory page has been copied into system memory 104 . Because CPU page table 206 contains an entry associating a virtual memory address with a memory page in system memory 104 , the page is accessible to CPU 102 . Since PPU page table 208 contains an entry associating a virtual memory address with a memory page in system memory 104 , that page is also accessible to PPU 202 .

Detailed example of a page fault sequence

In this context, a detailed description of the page fault sequence performed by the PPU fault handler 215 in the event of a transition from CPPU-owned to CPU-shared is now provided to show how atomic operations and transition states are used to manage the sequence more efficiently. The page fault sequence is triggered by a PPU 202 thread attempting to access a virtual address for which no associated mapping exists in the PPU page table 208 . When a thread attempts to access data via a virtual memory address, the PPU 202 (specifically, a user-level thread) requests a translation from the PPU page table 208 . In response, a PPU page fault occurs because the PPU page table 208 does not contain a mapping associated with the requested virtual memory address.

After a page fault occurs, the aforementioned threads are trapped, stalled, and the PPU fault handler 215 executes the page fault sequence. PPU fault handler 215 reads PSD 210 to determine which memory page is associated with the virtual memory address and to determine the status of the virtual memory address. PPU fault handler 215 determines from PSD 210 that the ownership status of the memory page is CPU owned. Thus, data requested by PPU 202 is accessible to PPU 202 via the virtual memory address. The status information for the memory page also indicates that the requested data cannot be migrated to PPU memory 204 .

Based on the state information obtained from PSD 210, PPU fault handler 215 determines that the new state of the memory page should be CPU shared. PPU Fault Handler 215 changes state to "Transition to CPU Sharing". This status indicates that the page is currently in the process of transitioning to CPU sharing. When the PPU fault handler 215 is running on the microcontroller in the memory management unit, then the two processors will asynchronously update the PSD 210, using an atomic compare-swap (“CAS”) operation on the PSD 210 to change state to “Transition to GPU visible (visible)" (CPU sharing).

PPU 202 updates PPU page table 208 to associate virtual memory addresses with memory pages. PPU 202 also invalidates TLB cache entries. Next, PPU 202 performs another atomic compare-swap on PSD 210 to change the ownership state associated with the memory page to CPU shared. Finally, the page fault sequence terminates, and the thread that requested data via the virtual memory address continues to execute.

Variation of UVM system architecture

Various modifications to unified virtual memory system 200 are possible. For example, in some embodiments, upon writing a fault register entry in fault register 216, PPU 202 may trigger a CPU interrupt to cause CPU 102 to read the fault register entry in fault register 216 and respond to the fault register entry and perform any appropriate action. In other embodiments, the CPU 102 may periodically poll the fault register 216 . If CPU 102 finds a faulty register entry in faulty register 216, CPU 102 performs a series of operations in response to the faulty register entry.

In some embodiments, system memory 104 , rather than PPU memory 204 , stores PPU page table 208 . In other embodiments, a single-level or multi-level cache hierarchy, such as a single-level or multi-level translation lookaside buffer (TLB) hierarchy (not shown), may be implemented for the CPU page table 206 or the PPU Page table 208 caches virtual address translations.

In still other embodiments, PPU 202 may take one or more actions in the event a thread executing in PPU 202 causes a PPU fault ("faulted thread"). These actions include stalling the entire PPU 202, stalling the SM executing the failing thread, stalling the PPU MMU 213, stalling only the failing thread, or stalling one or more levels of the TLB. In some embodiments, after a PPU page fault occurs, and the unified virtual memory system 200 has executed the page fault sequence, the faulted thread continues to execute, and the faulted thread retries the memory access request that caused the page fault. In some embodiments, stalls at the TLB are performed in such a way that they appear as long-latency memory accesses to the faulting SM or thread, thereby requiring the SM to do nothing in response to the fault. special operations.

Finally, in other alternative embodiments, UVM driver 101 may contain instructions that cause CPU 102 to perform one or more operations for managing UVM system 200 and repairing page faults, such as accessing CPU page tables 206, PSD 210, and/or fault buffers 216. In other embodiments, an operating system kernel (not shown) may be configured to manage UVM system 200 and repair page faults by accessing CPU page table 206 , PSD 210 , and/or fault buffer 216 . In yet other embodiments, an operating system kernel may operate in conjunction with UVM driver 101 to manage UVM system 200 and repair page faults by accessing CPU page table 206 , PSD 210 and/or fault buffer 216 .

Migrate memory pages of different sizes

Memory pages stored in system memory 104 are permitted to have a different size than memory pages stored in PPU memory 204 . For example, a memory page stored in system memory 104 may have a size of 4KB, while a memory page stored in PPU memory 204 may have a size of 128KB. As another example, memory pages stored in system memory 104 may have a size of 4KB, while memory pages stored in PPU memory 204 may have a mix of 4KB pages and 128KB pages. As yet another example, memory pages stored in system memory 104 may have a mix of 4KB and 1MB pages, while memory pages stored in PPU memory 204 may have a mix of 4KB pages and 128KB pages. During a page fault sequence, UVM system 200 may transfer a memory page from one memory unit to another (eg, from PPU memory 204 to system memory 104 ). To accommodate differences in the size of memory pages, UVM system 200 may split large memory pages or combine multiple small memory pages when UVM system 200 transfers memory pages. UVM system 200 may also transfer one or more additional "slibling" memory pages with the memory page being transferred. In some embodiments, a sibling memory page is a memory page with a smaller size (eg, a memory page has a 4KB size in a system that stores both 4KB and 128KB memory pages) that can accommodate the alignment of the larger page. within the address span (span). An aligned address span refers to the address range from start to end of a larger sized memory page. Smaller pages that lie within such an address span are considered sibling memory pages.

Several operations associated with partitioning a large memory page into multiple smaller memory pages or combining multiple small memory pages into larger memory pages and transferring these memory pages between memory cells are described below with reference to FIGS. 3-7. For example, operations are described that occur when a small memory page is transferred from system memory 104 to PPU memory 204 (FIG. 3); a small memory page and a sibling memory page are transferred from system memory 104 to a large memory pages (FIG. 4); splitting large memory pages in PPU memory 204 into small memory pages and transferring one of these small memory pages from PPU memory 204 to system memory 104 (FIG. 5); and The small memory page and its sibling memory pages are transferred into system memory 104 ( FIG. 6 ).

FIG. 3 illustrates operations for transferring a small memory page from system memory 104 to PPU memory 204, according to one embodiment of the invention. In this operation, system memory 104 stores small memory page 302 and PPU memory 204 stores both small memory page 304 and large memory page 306 . From this operation, UVM driver 101 determines that a particular small memory page 302 ( 0 ) in system memory 104 is to be migrated to PPU memory 204 . Because PPU memory 204 is capable of storing memory pages of small size or large size, the migrated memory page is stored as small memory page 304(1). The migrated small memory page 304 ( 1 ) may be stored next to other small memory pages 304 in PPU memory 204 . Large memory pages 306 may also be stored in PPU memory 204 . After migration, the space in system memory occupied by the migrated small memory pages is deallocated and made available for future allocations.

The UVM driver 101 alters the PSD entry to perform the operation for the associated memory page. Altering the PSD entry may include setting the PSD entry for the associated memory page to indicate an intermediate and/or locked state. More specifically, UVM driver 101 sets the PSD entry associated with memory page 302(0) to indicate that memory page 302(0) is in transfer and can only be read. Subsequently, the UVM driver 101 sets the PSD entry associated with the target large memory page 306 to indicate that the large memory page 306 is in transfer and cannot be accessed. UVM driver 101 copies small memory page 302(0) to target large memory page 306 . Subsequently, the UVM driver 101 sets the PSD entry to indicate that the target large memory page 306 is accessible (read and write).

As described above with reference to FIG. 2, particular memory pages may be migrated for a variety of reasons. When a single small memory page 302 stored in system memory 104 is required in PPU memory 204, but a "sibling" memory page is also required in system memory 104 surrounding the single required memory page, the implementation shown in FIG. operate. This situation arises when a single small memory page 302 is frequently accessed by, for example, PPU 202 , while one or more of the sibling memory pages are frequently accessed by, for example, CPU 102 . Generally, when a memory page group containing a single small memory page and sibling memory pages is "strongly asserted," meaning that the various memory pages in the memory page group are accessed by different processing units, such as the CPU 102 and the PPU 202, the UVM driver 101 The group of memory pages can be partitioned.

FIG. 4 illustrates operations for transferring a small memory page and associated "sibling" memory pages from system memory 104 to PPU memory 204, according to one embodiment of the invention. In this operation, system memory 104 stores small memory page 402 and PPU memory 204 stores both small memory page 404 and large memory page 406 as in the operation described with reference to FIG. 3 . According to the operations shown in FIG. 4, UVM driver 101 determines that a particular small memory page 402(1), as well as sibling memory pages 402(0), sibling memory pages 402(2), and sibling memory pages 402(3) are to be removed from the system Memory 104 is migrated to PPU memory 204 . UVM driver 101 causes these memory pages to be migrated from system memory 104 to PPU memory 204 . Because the PPU memory 204 typically works more efficiently with larger memory pages, the UVM driver 101 can coalesce the small memory pages 402 copied from the system memory 104 into one large coalesce containing all the data from the small memory pages Memory page 405.

In addition, as described above with reference to FIGS. 2 and 3 , reasons for migrating a particular memory page can be various, such as usage history. The operations shown in FIG. 4 may be performed when a particular memory page is needed in PPU memory 204 and moving a sibling memory page is deemed advantageous. In one example, such pages are merged based on a least-recently-used tracking scheme. Small pages that are accessed infrequently are merged together.

As described above, UVM driver 101 alters PSD entries to perform operations for associated memory pages. Altering the PSD entry may include setting the PSD entry for the associated memory page to indicate an intermediate and/or locked state. More specifically, UVM driver 101 sets the PSD entries associated with memory page 420(0), memory page 420(1), memory page 420(2), and memory page 420(3) to indicate that these memory pages are in transfer and can only be read. Subsequently, the UVM driver 101 sets the PSD entry associated with the target large memory page 406 to indicate that the large memory page 406 is in transfer and cannot be accessed. UVM driver 101 then copies these small memory pages to target large memory pages 406 . Subsequently, the UVM driver 101 sets the PSD entry to indicate that the target large memory page 406 is accessible (read and write).

Figure 5 illustrates operations for transferring a small memory page from PPU memory 204 to system memory 104, according to one embodiment of the present invention. In this operation, system memory 104 stores small memory page 502 and PPU memory 204 stores both large memory page 504 and small memory page 506 as described above with reference to FIGS. 3 and 4 . From this operation, UVM driver 101 determines that a particular portion of a large memory page stored in PPU memory 204 is to be migrated to system memory 104 . The UVM driver 101 causes the large memory page to break up into small memory pages, and then causes the small memory pages associated with the aforementioned parts to be migrated to the system memory 104 . Additionally, as noted above, the reasons for migrating a particular page of memory can be various, including usage history.

As described above, "strongly asserted" memory pages may be partitioned. In other words, for a particular large memory page, if the small memory pages within the large memory page are frequently accessed by multiple different processing units, such as CPU 102 or PPU 202 , the large memory page may be split. This analysis is one type of analysis based on usage history.

As described above, the UVM driver 101 modifies the PSD entry for the associated memory page to perform the operations shown in FIG. 5 . More specifically, the UVM driver 101 sets the PSD entry associated with the large memory page to be split to indicate that the memory page is in transfer and can only be read. Subsequently, the UVM driver 101 sets the PSD entry associated with the target small memory page 502 to indicate that the small memory page 502 is in transfer and cannot be accessed. The UVM driver 101 then copies the aforementioned portion of the large memory page 506 to the target target small memory page 502 . Subsequently, the UVM driver 101 sets the PSD entry to indicate that the target target small memory page 502 is accessible (read and write).

FIG. 6 illustrates operations for transferring small memory pages and sibling memory pages from PPU memory 204 to system memory 104, according to one embodiment of the invention. In this operation, system memory 104 stores small memory page 602 and PPU memory 204 stores both large memory page 604 and small memory page 606 as described above with reference to FIGS. 3-5 . From this operation, UVM driver 101 determines that a particular portion of large memory page 606 is to be migrated from PPU memory 204 to system memory 104 . UVM driver 101 has UVM driver 101 split into small memory pages 604, and has small memory page 604(1) and sibling memory pages—small memory page 604(0), small memory page 604(2), and small memory page 604 (3)——Migrate to system memory 104 . As with Figures 3-5, certain memory pages, such as small memory page 604(1), may migrate from PPU memory 204 to system memory 104 for a variety of reasons, as described above with reference to Figures 1 and 2 .

Also as described above, the UVM driver 101 alters the PSD entry to perform the operation for the associated memory page. More specifically, the UVM driver 101 sets the PSD entry associated with the large memory page to be split to indicate that the memory page is in transfer and can only be read. Subsequently, the UVM driver 101 sets the PSD entry associated with the target small memory page 602 to indicate that the small memory page 602 is in transfer and cannot be accessed. The UVM driver 101 then copies the large memory page 606 , now broken into small memory pages, to the target small memory page 602 . Subsequently, the UVM driver 101 sets the PSD entry to indicate that the target small memory page 602 is accessible (read and write).

7 is a flowchart of method steps for migrating memory pages of different sizes between memory units in a virtual memory architecture, according to one embodiment of the present invention. Although these method steps are described in connection with Figures 1-6, those skilled in the art will understand that any system configured to perform these methods, in any order, is within the scope of the present invention.

As shown, method 700 begins at step 702, where UVM driver 101 determines that a memory page is to be migrated. In step 704, UVM driver 101 determines whether the memory page is a large memory page. If the memory page is a large memory page, method 700 proceeds to step 706 . In step 706, the UVM driver 101 determines whether to split the large memory page. If the UVM driver 101 determines that the large memory page should be split, the method proceeds to step 708 . In step 708, the UVM driver 101 splits the large memory page and copies the small memory pages from the large memory page from one memory unit to another memory unit. If at step 708 the UVM driver 101 determines not to split the large memory page, then the method proceeds to step 710 . At step 710, the UVM driver 101 copies a large memory page from one memory unit to another memory unit.

Returning to step 704 , if the UVM driver 101 determines that the memory page is not a large memory page, then the memory page is a small memory page and the method proceeds to step 712 . In step 712, the UVM driver 101 determines whether to merge the small memory page and the sibling memory page. If the UVM driver 101 determines that the small memory page should be merged, the UVM driver 101 proceeds to step 714 . In step 714, the UVM driver 101 copies the small memory page and peer memory from one memory unit to another memory unit. If at step 712 the UVM driver 101 determines not to merge the memory page and sibling memory pages, then the method proceeds to step 716 . In step 716, the UVM driver 101 copies the small memory page from one memory unit to another memory unit.

In summary, a method is provided by which memory pages residing in memory cells storing memory pages of different sizes can be migrated between different memory cells. The UVM driver 101 determines which memories are to be migrated. If the memory page is a small memory page, the UVM driver 101 determines whether to migrate a sibling memory page as well. If the memory page is a large memory page, UVM driver 101 determines whether the large memory page is to be split, or whether the entire memory page is to be migrated. During migration, the UVM driver 101 blocks access to memory pages involved in the migration.

One advantage of the disclosed technique is that memory pages of different sizes can be efficiently migrated back and forth between different memory units in a virtual memory architecture. This technology increases the flexibility of the unified virtual memory system by allowing it to work with many different types of memory architectures. Another related advantage is that, by allowing large memory pages to be split into smaller memory pages and allowing small memory pages to be merged into larger memory pages, memory pages having different sizes can be stored in different memories configured to store different memory page sizes. in the unit. This feature allows the unified virtual memory system to group pages where possible in order to reduce the amount of space taken up in page tables and/or translation lookaside caches. This feature also allows memory pages to be split and migrated to different memory cells, as long as such splitting improves memory locality and reduces memory access time.

While the foregoing is directed to embodiments of the invention, other and further embodiments of the invention may be devised without departing from the essential scope thereof. For example, aspects of the present invention can be implemented in hardware or software or a combination of hardware and software. One embodiment of the invention can be implemented as a program product for use with a computer system. The program(s) of the program product define functions of the embodiments (including the methods described herein) and can be contained on a variety of computer-readable storage media. Exemplary computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer, such as compact disc read-only memory (CD-ROM) disks that can be read by a CD-ROM drive , flash memory, read-only memory (ROM) chips, or any type of solid-state non-volatile semiconductor memory) on which to store persistent information; and (ii) writable storage media (for example, a floppy disk or any type of solid-state random-access semiconductor memory) on which information can be changed. Such computer-readable storage media, when carrying computer-readable instructions for the functions of the present invention, are embodiments of the present invention.

The invention has been described above with reference to specific embodiments. Those of ordinary skill in the art will appreciate, however, that various modifications and changes can be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. Accordingly, the foregoing description and drawings are to be regarded in an illustrative rather than a restrictive sense.

Accordingly, the scope of the invention is defined by the appended claims.

Claims (10)

1. A computer-implemented method for migrating a memory page from a first memory to a second memory, the method comprising: determining a first page size supported by the first memory; determining a second page size supported by the second memory; determining a usage history of the memory page based on an entry in a page state directory (PSD) associated with the memory page; and The memory page is migrated between the first memory and the second memory based on the first page size, the second page size, and the usage history. 2. The method of claim 1 , wherein the first page size is smaller than the second page size, and migrating the memory page comprises transferring the memory page from system memory to a parallel processing unit ( PPU) local memory. 3. The method of claim 2, further comprising: transferring at least one peer memory page from the system memory to a memory local to the PPU, wherein the at least one peer memory page is to be shared with the memory page combining to generate at least a portion of a larger memory page in memory local to the PPU. 4. The method of claim 1, wherein the first size is larger than the second size, and migrating the memory page comprises transferring the first memory page from memory local to a parallel processing unit (PPU) to system memory. 5. The method of claim 4, wherein migrating the memory page further comprises: dividing the memory page in the PPU memory into a plurality of smaller memory pages including a second memory page, and dividing the second memory page into Two memory pages are transferred from the PPU local memory to the system memory. 6. A computing device for migrating memory pages, the computing device comprising: first storage; second memory; Page State Directory (PSD); and Unified Virtual Memory (UVM) drivers configured to: determining a first page size supported by the first memory; determining a second page size supported by the second memory; determining a usage history of the memory page based on an entry in a page state directory associated with the memory page; and The memory is migrated between the first memory and the second memory based on the first page size, the second page size, and the usage history. 7. The computing device of claim 6, wherein the first page size is smaller than the second page size, and migrating the memory page comprises transferring the memory page from system memory to a parallel processing unit (PPU) local memory. 8. The computing device of claim 7, wherein the UVM driver is further configured to: transfer at least one peer memory page from the system memory to the PPU-local memory, wherein the at least one A sibling memory page is to be combined with the memory page to generate at least a portion of a larger memory page in memory local to the PPU. 9. The computing device of claim 6, wherein the first size is larger than the second size, and migrating the memory page comprises: moving the first memory page from memory local to a parallel processing unit (PPU) Transfer to system memory. 10. The computing device of claim 9 , wherein migrating the memory page further comprises: dividing the memory page in the PPU memory into a plurality of smaller memory pages including a second memory page, and dividing the A second memory page is transferred from memory local to the PPU to the system memory.