DE102013022168B4 - Unified Virtual Storage System Migration Scheme - Google Patents

Abstract

A computer-implemented method for accessing data in a virtual memory subsystem, the method comprising: issuing a page fault in response to a memory access request, wherein a local page table (208) does not contain an entry corresponding to a virtual memory address contained in the memory access request ;responsive to the page fault, performing a page fault sequence that includes:locating an entry in a page state table (210) that corresponds to a memory page associated with the virtual memory address;reading an ownership state associated with the memory page from the entry in the page state directory (210), the state of ownership being determined based on a usage history of the memory page; and updating the local page table (208) to include an entry corresponding to the virtual memory address and linking the virtual memory address to the memory page.

Description

BACKGROUND OF THE INVENTION

field of invention

Embodiments of the present invention relate generally to virtual memory, and more particularly to a migration scheme for a unified virtual memory system.

Description of the prior art

Most modern computer systems typically implement some type of virtual memory architecture. Among other things, the virtual memory architecture makes it possible to access memory using virtual memory addresses rather than physical memory addresses. By providing this virtual storage layer between physical storage and an application software, the user-level software is shielded from the details of physical storage management that are left to a dedicated storage management system.

A typical computer system implementing a virtual memory architecture includes a central processing unit (CPU) and one or more parallel processing units (GPU). In operation, a software process running on a CPU or a GPU may request data through a virtual memory address. In many common architectures, the virtual memory systems that handle requests for data via virtual memory addresses for a CPU and a GPU are independent. In particular, a separate CPU memory management system and a separate GPU memory management system can handle requests for data from the CPU and GPU accordingly.

There are some disadvantages associated with such independent memory management systems. For example, each independent storage management system does not necessarily have knowledge of the contents of the storage units belonging to the other storage management system. Therefore, the storage management systems may not cooperate with each other to provide certain efficiencies, such as determining where data should be stored for improved access processing time. Furthermore, since the memory management systems are independent, pointers for one such system are not necessarily compatible with the other system. Therefore, a programmer of an application must monitor the two different types of pointers. The U.S. 2010/0 321 397 A1 describes a system having a first memory and a host memory, both memories having page tables for assigning virtual to physical memory addresses.

As the foregoing indicates, what is needed in this field of technology is a more efficient way to manage virtual memory in a system with heterogeneous processors, such as a CPU and a GPU.

OVERVIEW OF THE INVENTION

An embodiment of the present invention provides a system according to claim 10 for managing mappings from virtual memory to physical memory using a page state map. The system includes a first processing unit configured to perform a first operation that references a first virtual memory address. The system further includes a first memory management unit (MMU) coupled to the first processing unit and configured to generate a first memory error upon determining that a first page table stored in a first memory unit coupled to the first processing unit does not Allocation corresponding to the first virtual memory address contains. The system further comprises a first copying unit connected to the data processing unit. The first copy unit is configured to read a first command queue to determine a first mapping associated with the first virtual memory address and contained in a first page state table. The first copy engine is further configured to update the first page table to include the first mapping.

An advantage of the disclosed approach is that user-level applications do not need to keep track of multiple pointers depending on where a particular data item is stored. Another advantage is that memory pages are migrated between memory devices based on usage, which allows memory pages to be localized to the devices that access the memory pages more frequently. Another advantage is that an error buffer is provided that allows errors generated by the PPU to be merged for efficient execution.

character list

• 1 ist eine Blockansicht, die ein Computersystem darstellt, das ausgebildet ist, einen oder mehrere Aspekte der vorliegenden Erfindung zu realisieren;
• 2 ist eine Blockansicht, das ein vereinheitlichtes bzw. vereinigtes virtuelles Speichersystem gemäß einer Ausführungsform der vorliegenden Erfindung zeigt;
• 3 ist eine Konzeptdarstellung eines Systems 300 zur Verfolgung des Zustands von Speicherseiten gemäß einer Ausführungsform der vorliegenden Erfindung;
• 4 ist eine Konzeptdarstellung eines Systems, das eine Migrationsoperation gemäß einer Ausführungsform der vorliegenden Erfindung realisiert;
• 5 zeigt ein virtuelles Speichersystem zur Speicherung von Fehlern in einem Fehlerpuffer gemäß einer Ausführungsform der vorliegenden Erfindung;
• 6 zeigt ein virtuelles Speichersystem zur Auflösung von Seitenfehlern, die von der GPU erzeugt werden, gemäß einer Ausführungsform der vorliegenden Erfindung;
• 7 gibt ein Flussdiagramm von Verfahrensschritten zur Verwaltung von Zuordnungen vom virtuellen Speicher zum physikalischen Speicher mittels eines Seitenzustandsverzeichnis gemäß einer Ausführungsform der vorliegenden Erfindung an;
• 8 gibt ein Flussdiagramm von Verfahrensschritten zur Verfolgung von Seitenfehlern gemäß einer Ausführungsform der vorliegenden Erfindung an;
• 9 gibt ein Flussdiagramm von Verfahrensschritten zur Auflösung von Seitenfehlern in einem Fehlerpuffer gemäß einer Ausführungsform der vorliegenden Erfindung an;
• 10 gibt ein Flussdiagramm von Verfahrensschritten zur Erzeugung und zur Verwaltung gemeinsamer Zeiger in einer virtuellen Speicherarchitektur gemäß einer Ausführungsform der vorliegenden Erfindung an; und
• 11 zeigt ein Flussdiagramm von Verfahrensschritten zur Verwaltung des Besitzzustands in einem virtuellen Speichersubsystem gemäß einer Ausführungsform der vorliegenden Erfindung.

In order to indicate the manner in which the above features of the present invention can be understood in detail, a more specific description of the invention, briefly summarized above, is given by reference to embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the attached drawings show only typical embodiments of this invention and therefore should not be considered as limiting its scope, since the invention admits of other equally effective embodiments.

• 1 Figure 12 is a block diagram illustrating a computer system configured to implement one or more aspects of the present invention;
• 2 12 is a block diagram showing a unified virtual memory system according to an embodiment of the present invention;
• 3 Figure 300 is a conceptual representation of a system 300 for tracking the state of memory pages according to an embodiment of the present invention;
• 4 Fig. 12 is a conceptual diagram of a system realizing a migration operation according to an embodiment of the present invention;
• 5 Figure 12 shows a virtual memory system for storing errors in an error buffer according to an embodiment of the present invention;
• 6 Figure 12 shows a virtual memory system for resolving page faults generated by the GPU according to an embodiment of the present invention;
• 7 Figure 12 provides a flow chart of method steps for managing virtual memory to physical memory mappings using a page state table in accordance with an embodiment of the present invention;
• 8th Figure 12 provides a flowchart of method steps for tracking page faults according to an embodiment of the present invention;
• 9 Figure 12 provides a flowchart of method steps for resolving page faults in a fault buffer according to an embodiment of the present invention;
• 10 Figure 12 provides a flowchart of method steps for creating and managing shared pointers in a virtual memory architecture according to an embodiment of the present invention; and
• 11 12 shows a flow diagram of method steps for managing ownership in a virtual storage subsystem according to an embodiment of the present invention.

DETAILED DESCRIPTION

system overview

1 1 is a block diagram showing a computer system 100 configured to implement one or more aspects of the present invention. The computer system 100 includes a central processing unit (CPU) 102 and a system memory 104 that communicate over an interconnect path that may include a memory bridge 105 . The memory bridge 105, which may be, for example, a Northbridge chip, is connected to an I/O (input/output) bridge 107 via a bus or other communication path 106 (e.g., a HyperTransport connection). The I/O bridge 107, which may be, for example, a south bridge chip, receives user input from one or more user input devices 108 (e.g. keyboard, mouse) and routes the input to the CPU 102 via the communication path 106 and the memory bridge 105 further. A parallel processing subsystem 112 is connected to the memory bridge 105 via a bus or second communication path 113 (e.g connected via a Peripheral Component Interconnect (PCI) Express, Accelerated Graphics Port, or HyperTransport connection); In one embodiment, parallel processing subsystem 112 is a graphics subsystem that provides pixels to a display device 110, which may be a conventional cathode ray tube, liquid crystal display, light emitting diode display, or the like. A system disk 114 is also connected to I/O bridge 107 and is configured to store content and applications and data for use by CPU 102 and parallel processing subsystem 112 . The system disk 114 provides non-volatile storage space for applications and data and can be fixed or removable hard drives, flash memory devices, and CD (Compact Disk Read Only Memory), DVD (Digital Versatility Disk ROM), Blu-ray, HD-DVD (high definition DVD) or other magnetic, optical or semiconductor storage devices.

A switch 116 provides connections between the I/O bridge 107 and other components such as a network adapter 118 and various add-in cards 120 and 121 . Other components (not explicitly shown), including a universal serial bus (USB) or other port connections, compact disk (CD) drives, digital versatile disk (DVD) drives, movie recorders, and the like may also be connected to the I/O bridge 107 . In the 1 The various communication paths shown, including specifically designated communication paths 106 and 113, may be implemented using any suitable protocol, such as PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or one or more other bus protocols or point-to-point communication protocols , and connections between different devices may use different protocols, as is known in the art.

In one embodiment, parallel processing subsystem 112 includes circuitry optimized for graphics and video processing, including, for example, video output circuitry, and forms one or more parallel processing units (PPU) 202. In another embodiment, parallel processing subsystem 112 includes circuitry designed for general purpose processing is optimized while maintaining the underlying computational architecture as described in more detail below. In yet another embodiment, parallel processing subsystem 112 may be integrated with one or more other system elements into a single subsystem, such as by unifying memory bridge 105, CPU 102, and I/O bridge 107 to form a system on a chip (SoC ) to build. As is well known, many graphics processing units (GPU) are designed to perform operations and computations in parallel and can therefore be viewed as a class of parallel processing units (PPU).

Any number of PPUs 202 can be included in a parallel processing subsystem 112 . For example, multiple PPUs 202 can be provided in a single option card, or multiple option cards can be connected to the communication path 113, or one or more PPUs 202 can be integrated in a bridge chip. The PPUs 202 in a multi-PPU system may be identical to or different from one another. For example, different PPUs 202 may have different numbers of processing cores, different amounts of local parallel processing memory, and so on. If multiple PPUs 202 are present, these PPUs can operate in parallel to process data at a higher throughput than would be possible with a single PPU 202. Systems incorporating one or more PPUs 202 may be implemented in a variety of configurations and form factors, including desktop, mobile or handheld personal computers, service providers, workstations, game consoles, embedded systems, and the like.

The PPU 202 advantageously implements a highly parallel processing architecture. The PPU 202 contains a number of general processing clusters (GPC). Each GPC is capable of executing a large number (e.g., hundreds or thousands) of threads simultaneously, with each thread being an instance of a program. In some embodiments, single-instruction-multiple-data (SIMD) instruction issue techniques are employed to handle parallel execution of a large number of threads without providing multiple independent instruction units. In other embodiments, single instruction multiple thread (SIMT) techniques are employed to handle parallel execution of a large number of commonly synchronized threads. Unlike a SIMD execution regime, in which all processing units typically execute identical instructions, SIMT execution allows different threads to better follow divergent execution paths through a given thread program.

GPCs contain a number of stream multiprocessors (SM), each SM configured to process one or more thread groups. The set of instructions dispatched to a particular GPC forms a thread, as previously defined herein, and the aggregation of a certain number of concurrently executing threads across the parallel processing units within an SM is referred to herein as a "chain" or " strand group”. As used herein, a "thread group" means a group of threads that are concurrently executing the same program on different input data, with a single thread of the thread group being dedicated to a different processing unit within an SM. Furthermore, several related thread groups can be active simultaneously (in different phases of execution) within an SM. This collection of strand groups is referred to herein as a "cooperative strand array"("CTA") or "strand array".

In embodiments of the present invention, it is desirable to use the PPU 202 or one or more other processors of a computing system to perform general purpose computations using thread arrays. Each thread in the thread array has an associated unique thread identifier identifier ("thread ID") that the thread can access during execution of the thread. The thread ID, which can be defined as a one-dimensional or multi-dimensional numeric value, controls various aspects of the processing behavior of the thread. For example, a thread ID may be used to determine which portion of the input data set a thread is to process and/or to determine which portion of an output data set a thread is to create or write.

During operation, CPU 102 is the master processor of computer system 100 to control and coordinate the operation of other system components. In particular, the CPU 102 issues commands that control the operation of the PPUs 202. In one embodiment, communication path 113 is a PCI Express connection in which specific lanes are assigned to each PPU 202, as is known in the art. Other communication paths can also be used. The PPU 202 advantageously implements a highly parallel processing architecture. A PPU 202 can be provided with any amount of local parallel processing (PPU) memory.

In some embodiments, system memory 104 includes a unified virtual memory (UVM) driver 101. UVM driver 101 contains instructions for performing various tasks related to management of a unified virtual memory (UVM) system common to the CPU 102 and the PPUs 202 is concerned. Among other things, the architecture allows the CPU 102 and the PPU 202 to access a physical memory location using a common virtual memory address, regardless of whether the physical memory location is within system memory 104 or in memory dedicated to the PPU 202 is local.

It should be noted that the system shown herein is illustrative in nature and that variations and modifications are possible. The interconnect topology, including the number and placement of bridges, the number of CPUs 102, and the number of parallel processing subsystems 112 can be modified as needed. For example, in some embodiments, system memory 104 is connected to CPU 102 directly rather than through a bridge, and other devices communicate with system memory 104 through memory bridge 105 and CPU 102. In other alternative topologies, parallel processing subsystem 112 is coupled to the I/O Bridge 107 or connected directly to CPU 102 instead of memory bridge 105. In still other embodiments, I/O bridge 107 and memory bridge 105 may be integrated into a single chip rather than existing as one or more discrete devices. Large embodiments may have two or more CPUs 102 and two or more parallel processing subsystems 112 . The specific components shown herein are optional; for example, any number of add-on cards or peripheral devices can be handled. In some embodiments, switch 116 is omitted and network adapter 118 and add-in cards 120, 121 connect directly to I/O bridge 107.

Unified virtual storage system architecture

2 12 is a block diagram showing a unified virtual memory (UVM) system 200 according to an embodiment of the present invention. As shown, the unified virtual memory system 200 includes, without limitation, the CPU 102, the system memory 104, and the parallel processing unit (PPU) 202 coupled to parallel processing unit (PPU) memory 204 memory. The CPU 102 and the system memory 104 are connected to each other and to the PPU 202 via the memory bridge 105 .

The CPU 102 executes threads that can request data stored in the system memory 104 or in the PPU memory 204 via a virtual memory address. Virtual memory addresses prevent threads executing on CPU 102 from having to be aware of the internal workings of a memory system. Therefore, a thread may only have knowledge of virtual memory addresses, and may access data by requesting data through a virtual memory address.

The CPU 102 includes a CPU MMU 209 that processes requests from the CPU 102 to translate virtual memory addresses to physical memory addresses. The physical memory addresses are required to access data stored in a physical memory unit such as system memory 104 and PPU memory 204 . The CPU 102 includes a CPU error handler 211 that performs steps in response to the CPU MMU 209 generating a page fault to make requested data available to the CPU 102. The CPU error handler 211 is generally software residing in the system memory 104 and executing on the CPU 102, the software being invoked by a CPU 102 interrupt.

System memory 104 stores various memory pages (not shown) containing data for use by threads executing in CPU 102 or PPU 202 . As shown, system memory 104 stores a CPU page table 206 containing mappings between virtual memory addresses and physical memory addresses. The system memory 104 also stores a page state table 210 that functions as a "master page table" for the UVM system 200, as discussed in more detail below. The system memory 104 includes a fault buffer 216 which contains entries written by the PPU 202 to notify the CPU 102 of a page fault generated by the PPU 202. In some embodiments, system memory 104 includes unified virtual memory (UVM) driver 101, which includes instructions that, when executed, cause CPU 102 to execute instructions for, among other things, page fault recovery. In alternative embodiments, any combination of the page state table 210, the error buffer 216, and one or more command queues 214 may be stored in the PPU memory 204. A PPU page table 208 may also be stored in system memory 104 .

Similar to CPU 102, PPU 202 executes instructions that may request data stored in system memory 104 or PPU memory 204 via a virtual memory address. The PPU 202 includes a PPU MMU 213 that processes requests from the PPU 202 to translate virtual memory addresses to physical memory addresses. The PPU 202 includes a copy unit 212 that executes commands stored in the command queue 214 to copy memory pages, modify data in the PPU memory table 208, and also executes other commands. A PPU error handler 215 performs steps in response to a page fault in the PPU 202. The PPU error handler 215 may be software running in a processor or dedicated microcontroller in the PPU 202, or the PPU error handler 215 may be software running in the CPU 102, the latter being the preferred choice is. In some embodiments, CPU error handler 211 and PPU error handler 215 are a unified software program that is invoked by an error in CPU 102 or PPU 202 . The command queue 214 may reside in the PPU memory 204 or in the system memory 104, but is preferably located in the system memory 104.

In some embodiments, the CPU error handler 211 and the UVM driver 101 are a unified software program. In such cases, the unified software program may be software residing in system memory 104 and executing on CPU 102 . The PPU error handler 215 may be a separate software program running on a processor or dedicated microcontroller in the PPU 202, or the PPU error handler 215 may be a separate software program running on the CPU 102.

In other embodiments, the PPU error handler 215 and the UVM driver 101 are a unified software program. In such cases, the unified software program may be software residing in system memory 104 and executing on CPU 102 . CPU error handling unit 211 may be a separate software program residing in system memory 104 and executing on CPU 102 .

In other embodiments, the CPU error handler 211, the PPU error handler 215, and the UVM driver 101 are a unified software program. In such cases the unified software program may be software residing in system memory 104 and executing on CPU 102 .

In some embodiments, CPU error handler 211, PPU error handler 215, and UVM driver 101 may each reside in system memory 104, as previously described. As in 2 As shown, the UVM driver 101 resides in the system memory 104, while the CPU error handler unit 211 and the PPU error handler unit 215 reside in the CPU 102.

CPU error handler 211 and PPU error handler 215 respond to hardware interrupts originating from CPU 102 or PPU 202, such as interrupts resulting from a page fault. As described below, the UVM driver 101 contains instructions for performing various tasks related to the administration of the UVM system 200, including, without limitation, recovering a page fault and accessing the CPU page table 206, the page state table 210, the command queue 214 and/or the error buffer 216.

In some embodiments, CPU page table 206 and PPU page table 208 have different formats and contain different information; for example, PPU page table 208 may include the following while CPU page table 206 does not: an atomic disable bit; compression marks; and memory exchange type.

Similar to system memory 104, PPU memory 204 includes various pages of memory (not shown). As shown, PPU memory 204 may also include PPU page table 208, which contains mappings between virtual memory addresses and physical memory addresses. Alternatively, PPU page table 208 may be stored in system memory 104 .

Page State Directory

3 Figure 300 is a conceptual view of a system 300 for tracking the state of memory pages according to an embodiment of the present invention. As shown, the system 301 includes a page state table 210 coupled to a CPU page table 206 and a PPU page table 208 .

The page state table 210 is a data structure that stores allocations associated with each memory page in a specific virtual memory address space. To obtain a physical address from the PSD 210, a requestor provides a requested virtual address to the PSD 210, which performs a lookup operation based on the virtual address. In some embodiments, PSD 210 is capable of tracking memory pages of different sizes. To accomplish this, PSD 210 includes several arrays. For example, a first array manages CPU-sized pages and a second array manages PPU-sized pages.

Although page state table 210 may be implemented in any technically feasible manner, in one embodiment page state table 210 includes a multi-level table. Each non-leaf level contains an array of pointers to entries in the next level. Pointers may point to entries in PPU memory 204 or system memory 104.

The CPU 102 or the PPU 202 updates the PSD 210. Updates to a page of the PSD 210 in the system memory 104 can be accomplished using atomic compare-and-swap operations over the PCI-E bus. Updates to the PSD 210 side in PPU memory 204 may be accomplished by placing an update request in a PSD update ring buffer contained in system memory. An agent checks the ring buffer to apply updates to the PSD 210 before a read operation.

As described below, multiple virtual memory address spaces can exist. Two different processes executing on CPU 102 may be associated with two different virtual memory address spaces. Some processes can share address spaces. There is a PSD 210 for each virtual memory address space.

Different PSD 210 may each contain a mapping for the same memory location in system memory 104 or PPU memory 204 . In such a case, a single process can be designated as the owner of the memory location. The PSD 210 corresponding to this single process is considered the "owner PSD". The owner PSD contains a mapping to the store memory space, and the PSD 210 for all other processes that have allocations for that memory location, contain a link to the allocation in the owning PSD.

When a process associated with a particular PSD 210 no longer requires a particular mapping associated with a particular memory location, the process causes the mapping to be removed from the PSD 210 associated with that process. The assignment is arranged in a final list. At this point, other PSDs 210 may still contain mappings to the storage location. Those PSD 210 will continue to contain that mapping until the processes associated with that PSD 210 determine that the mapping is no longer needed. If no PSD 210 contains assignments associated with the location, then the assignment is removed from the final list.

Entries in the PSD 210 contain a mapping between a virtual memory address and a physical memory address. The entries also contain state information for the memory page associated with the entry. The following list includes some example states that may be included in the PSD entries in various embodiments of the invention. "Exclusive" - a memory page may be considered "exclusive" meaning that the memory page is non-duplicated and visible to a PPU 202 or a CPU 102 or both. The "exclusive" state is similar to the "PPU native" or "CPU native" state discussed below. "Shared-uncached" - a memory page can be considered "shared-uncached", meaning that the memory page is not duplicated, but that it is visible to one or more PPUs 202 and/or one or more CPUs 102. The "shared-uncached" state is similar to the "CPU-shared" state discussed below, with the additional property "uncached" meaning "not duplicated". A memory page may reside in more than one memory unit (e.g., both system memory 104 and PPU memory 204) and thus may be "duplicated". "Read-duplicated" - a memory page is considered "read-duplicated", meaning that more than one copy of the memory page exists and that at least one of the copies is local to a CPU 102 or PPU 202 and for read-through only that CPU 102 or PPU 202 is available. "Migrate or move read-only" - a memory page is considered "migrate or move read-only", meaning that the memory page is in the process of being moved. For example, the UVM system 200 is in the process of moving the memory page from the PPU memory 204 to the system memory 104. Since the memory page is considered to be "migrating-read-only", the memory page can only be read but not written when she is in this state. "Migrate-invisible" - a memory page can be considered "migrate-invisible", meaning the memory page is in the process of being moved, but the memory page is "invisible", meaning no process can read or write to the memory page can. "Entry-Forward-Entry" - a particular entry in a PSD 210 is considered an "Entry-Forward-Entry", meaning that the entry contains a link to another entry in the PSD 210 that contains an association, associated with the memory page.

The UVM system 200 may store pages of memory in a backing store, such as a location on a hard drive. The UVM driver 101 or the operating system monitors the memory pages stored in the background memory. If a lookup operation performed on the PSD 210 indicates that a memory page is stored in the backing memory, then the UVM driver 101 moves the memory page from the backing memory to the system memory 104 or the PPU memory 204. After the memory page from the background memory is copied, the UVM driver 101 repeats the lookup at the PSD 210.

The table below shows examples of PSD entries. Each row shows a different illustrative entry. Condition Data there are other pages to the site address forwarding Invalid entry No Move (read only) No move (invisible) No Exclusive entry Allowed access physical address Shared non-cached entry physical address Allowed access Read duplicate entry Bloom filter for which the processor has this page physical address

Translation of virtual memory addresses

It's back up 2 referred; when a thread executing in the CPU 102 requests data via a virtual memory address, then the CPU 102 requests the translation of the virtual memory address to a physical memory address from the CPU memory management unit (CPU MMU) 209 . The CPU MMU 209 then attempts to translate the virtual memory address to a physical memory address that specifies a location in a memory unit, such as system memory 104, that stores the data requested by the CPU 102.

To translate a virtual memory address to a physical memory address, the CPU MMU 209 performs a lookup operation to determine if the CPU page table 206 contains a mapping associated with the virtual memory address. In addition to a virtual memory address, a request to access data may also specify a virtual memory address space. The unified virtual memory system 200 may implement multiple virtual memory address spaces, each associated with one or more threads. Virtual memory addresses are unique within a given virtual memory address space. Furthermore, within a given virtual memory address space, virtual memory addresses are consistent between CPU 102 and PPU 202, so it is possible for the same virtual address to identify the same CPU 102 and PPU 202 data. In some embodiments, two virtual memory addresses in the same virtual address space may refer to the same data, but generally do not map to the same physical memory address (e.g., CPU 102 and PPU 202 each have a local read-only copy of the data.)

For a given virtual memory address, the CPU page table 206 may or may not contain a mapping between the virtual memory address and a physical memory address. If the CPU page table 206 contains a mapping, then the CPU MMU 290 reads that mapping to determine a physical memory address associated with the virtual memory address and provides that physical memory address to the CPU 102. However, if the CPU If the page table 206 does not contain a mapping associated with the virtual memory address, then the CPU MMU 209 is unable to translate the virtual memory address to the physical memory address and the CPU MMU 209 generates a page fault. In order to recover from a page fault and make the requested data available to the CPU 102, a "page fault sequence" is executed. Specifically, the CPU 102 reads the PSD 210 to determine the current allocation state of the page and then determines the appropriate page fault sequence. The page fault sequence generally allocates the memory page associated with the requested virtual memory address or changes the type of accesses that are allowed (e.g., read, write, atomic, or atomic, respectively) unless a fatal error has occurred. The different types of page fault sequences implemented in the UVM system 200 are explained in more detail below.

In the UVM system 200, data associated with a given virtual memory address may be stored in system memory 104, in PPU memory 204, or in both system memory 104 and PPU memory 204 as read-only copies of the same data access are saved. Further, for such data, the CPU page table 206 and/or the PPU page table 208 may contain a mapping associated with that data. Note that there may be some data that is mapped in one side table but not in the other. However, the PSD 210 contains all mappings stored in the PPU page table 208 and the PPU relevant mappings stored in the CPU page table 206. The PSD 210 thus serves as a "master" page table for the unified virtual memory system 200. Therefore, if the CPU MMU 209 does not find a mapping in the CPU page table 206 associated with a particular virtual memory address, the CPU 102 reads that PSD 210 to determine if PSD 210 contains a mapping associated with that particular virtual memory address. Various embodiments of PSD 210 may include different types of information associated with virtual memory addresses, in addition to mappings associated with the virtual memory address.

When the CPU MMU 209 generates a page fault, the CPU fault handler unit 211 performs a sequence of operations on the appropriate page fault sequence to recover from the page fault. Again, during a page fault sequence, the CPU 102 reads the PSD 210 and performs other operations to change the CPU page table 206 and PPU page table 208 allocations or deallocations. Such operations may include reading and/or modifying CPU page table 206, reading and/or modifying page state table 210 entries, and/or migrating blocks of data, referred to as "memory pages", between memory devices (e.g., the system memory 104 and the PPU memory 204).

4 Figure 4 is a conceptual representation of a system 400 for implementing a relocation operation according to an embodiment of the present invention. As shown, the system 401 includes a page state table 210, system memory 104, and PPU memory 204.

As previously explained, the page state directory 210 stores PSD entries 401 specifying all or part of a virtual memory address 402, all or part of a physical memory address 404, and state information 406. The PSD entries 401 thus map virtual memory addresses 402 to physical memory addresses 404 .

In response to a page fault, UVM driver 101 may determine that a memory page, such as memory page 408, is to be migrated or moved from one storage device to another storage device to recover from the page fault. For example, UVM driver 101 may determine that memory page 408 is to be moved from system memory 104 to PPU memory 204 . In response to this determination, UVM driver 101 performs a series of operations, hereinafter referred to as a page fault sequence, to cause memory page 408 to be moved. Furthermore, the page fault sequence may alter a portion of the page state table 401 entry associated with the memory page 408. In particular, the page fault sequence may update the physical memory address 404 to the physical location of the memory page 408 after the memory page has been moved. However, the virtual memory address 402 in the page state table entry 401 remains unchanged, causing pointers in application programs to remain the same and point to memory pages 408 regardless of where the memory pages 408 are stored.

To determine which operations to perform in a page fault sequence, CPU 102 determines the memory page associated with the virtual memory address. The CPU 102 then reads memory page state information from the PSD 210 pertaining to the virtual memory address associated with the memory access request that caused a page fault. Such state information may include, among other things, the state of ownership for the memory page associated with the virtual memory address. Multiple ownership states are possible for a given memory page. For example, a memory page can be CPU-native, PPU-native, or CPU-shared. A memory page is considered CPU owned if the CPU 102 can access the memory page via a virtual address and the PPU 202 cannot access the memory page via a virtual address without incurring a page fault. Preferably, a CPU-owned page resides in system memory 104, but may reside in PPU memory 204 as well. A memory page is considered PPU owned if the PPU 202 can access the page via a virtual address and if the CPU 102 cannot access the memory page via a virtual address without causing a page fault. Preferably, a PPU owned page resides in PPU memory 204, but may reside in system memory 104 if a move from system memory 104 to PPU memory 204 is generally not performed due to the short-term nature of PPU ownership. A memory page is considered CPU-shared if the memory page is stored in system memory 104 and there is a mapping to the memory page in PPU page table 208 that allows the PPU 202 to access the memory page in system memory 104 using a virtual memory address access.

The UVM system 200 may assign ownership states to memory pages based on a variety of factors, including usage history of the memory page, where usage history is optionally stored in entries in the PSD 210 . The usage history may include information regarding whether the CPU 102 or the PPU 202 recently accessed the memory page and how often such accesses were made. For example, the UVM system 200 may assign an ownership state of "CPU Own" for the given memory page and place the page in the system memory 104 if, based on the usage history of the memory page, the UVM system 200 determines that the memory page is likely to be primarily or exclusively owned by of the CPU 102 is used. Similarly, if, based on the usage history of the memory page, the UVM system 200 determines that the memory page is likely to be used mostly or exclusively by the PPU 202. Finally, the UVM system 200 may assign the ownership state to "CPU-shared" for a given memory page if, based on the usage history of the memory page, the UVM system 200 determines that the memory page is likely shared by both the CPU 102 and the PPU 202 is used and that moving the memory page between the system memory 104 and the PPU memory 204 would take too much time.

• (a) bei Zugriff der CPU 102 auf eine nicht zugeordnete Seite, die auf die PPU 202 abgebildet ist, die noch nicht vor kurzem verschoben wurde, Aufheben der Zuordnung der fehlererzeugenden Seite für die PPU 202, Verschieben der Seite zu der CPU 102 und Zuordnen der Seite zu der CPU 102;
• (b) bei Zugriff der PPU 202 auf nicht zugeordnete Seite, die der CPU 102 zugeordnet ist, die nicht vor kurzem verschoben worden ist, Aufheben der Zuordnung der fehlererzeugenden Seite zu der CPU 102, Verschieben der Seite zu der PPU 202 und Zuordnen der Seite zu der PPU 202;
• c) bei Zugriff der CPU 102 auf nicht zugeordnete Seite, die der PPU 202 zugeordnet ist und die vor kurzem verschoben wurde, Verschieben der fehlererzeugenden Seite zu der CPU 102 und Zuordnen der Seite zu der CPU 102 und der PPU 202;
• (d) bei Zugriff der PPU 202 auf nicht zugeordnete Seite, die der CPU 102 zugeordnet ist, die vor kurzem verschoben worden ist, Zuordnen der Seite der CPU 102 und der PPU 202;
• (d) bei atomaren Zugriff der PPU 102 auf die Seite, die der CPU 102 und der PPU 202 zugeordnet ist, die aber für atomare Operationen durch die PPU 202 nicht freigegeben ist, Aufheben der Zuordnung der Seite zu der CPU 102 und Zuordnen zu der PPU 202, wobei atomare Operationen freigegeben sind;
• (f) bei Schreibzugriff der PPU 202 auf Seite, die der CPU 102 und der PPU 202 als Kopie-bei-Schreiben (COW) zugeordnet ist, Kopieren der Seite in die PPU 202, wodurch unabhängige Kopien der Seite erzeugt werden, Zuordnen der neuen Seite als Lesen-Schreiben in der PPU und Belassen der aktuellen Seite als der CPU 102 zugeordnet;
• (d) bei Lesezugriff der PPU 202 auf Seite, die der CPU 102 und der PPU 202 mit Nullen-Auffüllung-auf-Verlangen (ZFOD) zugeordnet ist, Zuweisen einer Seite an physikalischem Speicher in der PPU 202 und Füllen dieser Seite mit Nullen, und Zuordnen dieser Seite der PPU, aber Ändern auf nicht zugeordnet in der CPU 102.
• (h) bei Zugriff durch eine erste PPU 202(1) auf eine nicht zugeordnete Seite, die einer zweiten PPU 202(2) zugeordnet ist, die nicht vor kurzem verschoben worden ist, Aufheben der Zuordnung der fehlererzeugenden Seite zu der zweiten PPU 202(2), Verschieben der Seite zu der ersten PPU 202(1) und Zuordnen der Seite zu der ersten PPU 202 (1); und
• (i) bei Zugriff durch eine erste PPU 202(1) auf eine nicht zugeordnete Seite, die einer zweiten PPU 202(2) zugeordnet ist und die nicht vor kurzem verschoben worden ist, Zuordnen der fehlererzeugenden Seite zu der ersten PPU 202(1) und Beibehalten der Zuordnung der Seite zu der zweiten PPU 202(2).

As examples, error handlers 211 and 215 may perform some or all of the following heuristic actions for migration and relocation, respectively:

• (a) when the CPU 102 accesses an unallocated page mapped to the PPU 202 that has not been recently moved, deallocating the error-producing page for the PPU 202, moving the page to the CPU 102, and allocating it the side to the CPU 102;
• (b) if the PPU 202 accesses an unallocated page allocated to the CPU 102 that has not recently been moved, deallocating the error-producing page to the CPU 102, moving the page to the PPU 202, and allocating the page to the PPU 202;
• c) if the CPU 102 accesses an unallocated page that is allocated to the PPU 202 and that has recently been moved, moving the error-producing page to the CPU 102 and allocating the page to the CPU 102 and the PPU 202;
• (d) when the PPU 202 accesses an unallocated page allocated to the CPU 102 that has recently been moved, allocating the page to the CPU 102 and the PPU 202;
• (d) if the PPU 102 atomically accesses the page that is allocated to the CPU 102 and the PPU 202 but is not enabled for atomic operations by the PPU 202, deallocating the page to the CPU 102 and allocating it to the PPU 202 with atomic operations enabled;
• (f) upon PPU 202 write access to page allocated to CPU 102 and PPU 202 as copy-on-write (COW), copying the page to PPU 202, thereby creating independent copies of the page, allocating the new one page allocated as read-write in the PPU and leaving the current page as the CPU 102;
• (d) upon PPU 202 read access to page allocated to CPU 102 and PPU 202 with zero-pad-on-demand (ZFOD), allocating a page of physical memory in PPU 202 and zero-padding that page, and allocating that side of the PPU but changing it to unallocated in the CPU 102.
• (h) upon access by a first PPU 202(1) to an unallocated page allocated to a second PPU 202(2) that has not recently been relocated, deallocating the offending page from the second PPU 202( 2) move the page to the first PPU 202(1) and allocate the page to the first PPU 202(1); and
• (i) upon access by a first PPU 202(1) to an unallocated page which is allocated to a second PPU 202(2) and which has not recently been moved, allocating the error-producing page to the first PPU 202(1) and retaining the page's association with the second PPU 202(2).

Overall, many heuristic rules are possible and the scope of the present invention is not limited to these examples.

• (j) bei Zugriff der CPU 102 auf eine nicht zugeordnete Seite, die der PPU 202 zugeordnet ist und die nicht vor kurzem verschoben worden ist, Afheben der Zuordnung der fehlerhaften Seite, plus weiterer Seiten, die benachbart zu der fehlerhaften Seite in dem virtuellen Adressenraum liegen, zu der PPU 202, Verschieben der Seiten zu der CPU 102 und Zuordnen der Seiten zu der CPU 102 (in einem detaillierteren Beispiel: für eine 4 kB fehlerhafte Seite, Verschieben des dazu ausgerichteten 64 kB-Gebiets, das die für 4 kB fehlerhafte Seite enthält);
• (k) bei Zugriff der PPU 202 auf nicht zugeordnete Seite, die der CPU 102 zugeordnet ist und nicht vor kurzem verschoben worden ist, Aufheben der Zuordnung der fehlerhaften Seite plus weiterer Seiten, die benachbart zu der fehlerhaften Seite in dem virtuellen Adressenraum liegen, zu der CPU 102, Verschieben der Seiten zu der PPU 202 und Zuordnen der Seiten zu der PPU 202 (in einem detaillierteren Beispiel: für eine 4 KB fehlerbehaftete Seite, Verschieben des dazugehörenden 64 kB-Gebiets, das die 4 kB fehlerhafte Seite enthält);
• (l) bei Zugriff der CPU 102 auf nicht zugeordnete Seite, die der PPU 202 zugeordnet ist und die nicht kürzlich verschoben worden ist, Aufheben der Zuordnung der fehlererzeugenden Seite plus weiterer Seiten, die benachbart zu der fehlererzeugenden Seite in dem virtuellen Adressenraum liegen, zu der PPU 202, Verschieben der Seiten zu der CPU 102, Zuordnen der Seiten zu der CPU 102 und Behandeln aller verschobener Seiten als eine oder mehrere größere Seiten in der CPU 102 (in einem detaillierteren Beispiel: für eine 4 kB fehlerhafte Seite, Verschieben des dazu ausgerichteten 64 kB-Gebiets, das die 4 kB fehlerbehaftete Seite enthält, und Behandeln des ausgerichteten 64 kB-Gebiets als eine 64 kB-Seite);
• (m) bei Zugriff der PPU 202 auf nicht zugeordnete Seite, die der CPU 102 zugeordnet ist, und die nicht kürzlich verschoben worden ist, Aufheben der Zuordnung der fehlererzeugenden Seite plus weiterer Seiten, die benachbart zu der fehlererzeugenden Seite in dem virtuellen Adressenraum liegen, zu der CPU 102, Verschieben der Seiten zu der PPU 202, Zuordnen der Seiten zu der PPU 202 und Behandeln aller verschobener Seiten als eine oder mehrere größere Seiten in der PPU 202 (in einem detaillierteren Beispiel: für eine 4 kB-fehlerbehaftete Seite, Verschieben des dazu ausgerichteten 64 kB-Gebiets, das die 4 kB fehlerbehaftete Seite enthält, und Behandeln des ausgerichteten 64 kB-Gebiets als eine 64 kB-Seite);
• (n) bei Zugriff durch eine erste PPU 202(1) auf eine nicht zugeordnete Seite, die einer zweiten PPU 202(2) zugeordnet ist und die nicht vor kurzem verschoben worden ist, Aufheben der Zuordnung der fehlererzeugenden Seite plus weiterer Seiten, die benachbart zu der fehlererzeugenden Seite in dem virtuellen Adressenraum liegen, zu der zweiten PPU 202(2), Verschieben der Seiten zu der ersten PPU 202(1) und Zuordnen der Seiten zu der ersten PPU 202(1); und
• (o) bei Zugriff durch eine erste PPU 202(1) auf eine nicht zugeordnete Seite, die einer zweiten PPU 202(2) zugeordnet ist und vor kurzem verschoben worden ist, Zuordnen der fehlererzeugenden Seite plus weiterer Seiten, die benachbart zu der fehlererzeugenden Seite in dem virtuellen Adressenraum liegen, zu der ersten PPU 202(1), und Beibehalten der Zuordnung der Seite zu der zweiten PPU 202(2).

Furthermore, any heuristic concepts for migration or movement can be "extended" to include more pages or a larger page size, for example:

• (j) upon CPU 102 accessing an unallocated page allocated to PPU 202 that has not recently been relocated, deallocating the faulty page, plus other pages adjacent to the faulty page in the virtual address space lying, to the PPU 202, shifting the pages to the CPU 102 and allocating the pages to the CPU 102 (in a more detailed example: for a 4 KB faulty page, shift the aligned 64 KB region containing the 4 KB faulty page contains);
• (k) upon PPU 202 access to an unallocated page that is allocated to CPU 102 and has not recently been moved, deallocating the faulty page plus other pages adjacent to the faulty page in the virtual address space the CPU 102, moving the pages to the PPU 202 and allocating the pages to the PPU 202 (in a more detailed example: for a 4 KB bad page, move the associated 64 KB region containing the 4 KB bad page);
• (l) upon CPU 102 access to an unallocated page that is allocated to PPU 202 and has not recently been relocated, deallocating the faulting page plus other pages adjacent to the faulting page in the virtual address space of the PPU 202, moving the pages to the CPU 102, allocating the pages to the CPU 102, and treating all the moved pages as one or more larger pages in the CPU 102 (in a more detailed example: for a 4 kB bad page, moving the to aligned 64 KB area containing the 4 KB errored page and treating the aligned 64 KB area as a 64 KB page);
• (m) when the PPU 202 accesses an unallocated page that is allocated to the CPU 102 and has not recently been moved, deallocating the fault-producing page plus other pages that are adjacent to the fault-producing page in the virtual address space, to the CPU 102, shifting the pages to the PPU 202, allocating the pages to the PPU 202, and treating all shifted pages as one or more larger pages in the PPU 202 (in a more detailed example: for a 4 KB bad page, shift the aligned 64 KB region thereto containing the 4 KB errored page and treating the aligned 64 KB region as a 64 KB page);
• (n) upon access by a first PPU 202(1) to an unallocated page which is allocated to a second PPU 202(2) and which has not recently been moved, deallocating the fault-producing page plus other pages that are adjacent to the faulting page in the virtual address space, to the second PPU 202(2), moving the pages to the first PPU 202(1), and allocating the pages to the first PPU 202(1); and
• (o) upon access by a first PPU 202(1) to an unallocated page that is allocated to a second PPU 202(2) and has recently been moved, allocating the faulting page plus other pages adjacent to the faulting page reside in the virtual address space, to the first PPU 202(1), and maintaining the association of the page with the second PPU 202(2).

Overall, many heuristic rules that include "expansion" are possible, and the scope of the present invention is not limited to these examples.

In some embodiments, the PSD entries may contain transition state information to ensure proper synchronization between various requests executed by entities within CPU 102 and PPU 202 . For example, a PSD 210 entry may contain a transition state indicating that a particular page is in the process of transitioning from CPU owned to PPU owned. Various units in the CPU 102 and the PPU 202, such as the PPU error handling unit 211 and the PPU error handling unit 215, upon detecting that a page is in such a transitional state, may refrain from executing portions of a page fault sequence to steps in the page fault sequence to avoid being triggered by a previous virtual memory access to the same virtual memory address. As a specific example, if a page fault occurs in a page that is being moved from system memory 104 to PPU memory 204, another page fault that would cause the same move will not cause another page move. Furthermore, when there is more than one writer to the PSD 210, various units in the CPU 102 and the PPU 202 can implement atomic operations for the correct ordering of operations on the PSD 210. For example, for modifications to PSD 210 entries, the CPU error handler 211 or the PPU error handler 215 may issue an atomic compare-and-swap operation to modify the page state of a particular entry in the PSD 210. Therefore, the modification is carried out without being disturbed by operations of other units.

Multiple PSDs 210 may be stored in system memory 104 - one for each virtual memory address space. A memory access request generated by CPU 102 or PPU 202 may therefore include a virtual memory address and may also specify the virtual memory address space associated with that virtual memory address. Further details regarding page state dictionaries are provided below with reference to 3 specified.

Just as CPU 102 can perform memory access requests that include virtual memory addresses (ie, instructions that include requests to access data through a virtual memory address), PPU 202 can also perform similar types of memory access requests. In particular, PPU 202 includes multiple execution units, such as GPC and SM, previously described in connection with 1 are described which are designed to carry out multiple strands and strand groups. During operation, these threads can request data from memory (e.g., system memory 104 or PPU memory 204) by specifying a virtual memory address. As with the CPU 102 and the CPU MMU 209, the PPU 202 contains the PPU memory management unit (MMU) 213. The PPU MMU 213 receives requests for translation of virtual memory addresses from the PPU 202 and attempts a translation from the PPU page table 208 for the virtual to provide memory addresses. Similar to CPU page table 206, PPU page table 208 contains mappings between virtual memory addresses and physical memory addresses. As with the CPU page table 206, for a given virtual address, the PPU memory table 208 may not contain a page table entry that maps the virtual memory address to the physical memory addresses. As with the CPU MMU 209, the PPU MMU 213 generates a page fault when the PPU MMU 213 requests a translation for a virtual memory address from the PPU page table 208 and there is no mapping in the PPU page table 208 or the type of access by the PPU -Page table 208 is not allowed. Again, different types of page fault sequences implemented in the SM system 200 are described in more detail below.

As previously indicated, in response to receiving a request for a virtual memory address translation, the CPU MMU 209 generates a page fault if the CPU page table 206 does not contain an allocation associated with the requested virtual memory address or if the type of the requested access is not allowed. Similarly, in response to receiving a request for translation of a virtual memory address, PPU MMU 213 generates a page fault if PPU page table 208 does not contain an allocation associated with the requested virtual memory address or if that type of requested access is not allowed. When the CPU MMU 209 or the PPU MMU 213 generate a page fault, the thread that requested data from the virtual memory address is stopped, and a "local error handler" - the CPU 211 error handler for the CPU 102 or the PPU error handler 215 for the PPU 202 - will fix the page fault by running a "page fault sequence". As previously indicated, a page fault sequence includes a series of operations that allow the faulting entity (i.e., the entity - the CPU 102 or the PPU 202 - that generated the page fault) to access the data associated with the virtual memory address are. After the page fault sequence completes, the thread that requested the virtual memory address data resumes execution. In some embodiments, error recovery is simplified by allowing error recovery logic to track error-producing memory accesses as opposed to error-producing instructions.

The operations performed during a page fault sequence depend on the change in ownership or a change in the access enables, if any, that the memory page associated with the page fault has to perform. Transitioning from a current ownership state to a new ownership state or changing access permissions may be part of the page fault sequence. In some cases, moving the memory page associated with the page fault from system memory 104 to PPU memory 204 is also part of the page fault sequence. In other cases, moving the memory page associated with the page fault from PPU memory 204 to system memory 104 is also part of the page fault sequence. Various heuristics are described in more detail that may be used to configure the UVM system 200, to change a memory page ownership state, or to move memory pages under various sets of operating states and patterns. Described below in more detail are memory error sequences for the following four memory page ownership transitions: CPU-native to CPU-shared, CPU-native to PPU-native, PPU-native to CPU-native, and PPU-native to PPU-shared.

A fault by the PPU 202 can cause a transition from CPU native to CPU common. Prior to such a transition, a thread executing in the PPU 202 attempts to access data at a virtual memory address that is not mapped in the PPU page table 208. This access attempt causes a PPU-based page fault, which then causes an error buffer entry in error buffer 216 to be written to. The PPU error handler 215 then reads the entry in the PSD 210 that corresponds to the virtual memory address and determines the memory page that associated with the virtual memory address. After reading the PSD 210, the PPU error handler 215 determines that the current state of ownership for the memory page associated with the virtual memory address is CPU owned. Based on the current ownership state, as well as other factors, such as usage characteristics for the memory page or the type of memory access, the PPU error handler 215 determines that a new ownership state for the page should be CPU-shared.

To change ownership, the PPU error handler 215 writes a new entry in the PPU page table 208 that corresponds to the virtual memory address and relates the virtual memory address to the memory page identified by the PSD 210 entry. The PPU error handler 215 also modifies the PSD 210 entry for that memory page to indicate that the ownership state is CPU-shared. In some embodiments, an entry in a translation preview buffer (TLB) in PPU 202 is invalidated to account for the case where the translation is stored in an illegal page. At this point the page fault sequence is complete. The ownership state for the memory page is CPU-shared, which means that the memory page is accessible to both the CPU 102 and the PPU 202. Both the CPU page table 206 and the PPU page table 208 contain entries that associate the virtual memory address with the memory page.

A fault by the PPU 202 can cause a transition from CPU own to PPU own. Prior to such a transition, an operation executing in the PPU 202 attempts to access memory at a virtual memory address that is not mapped in the PPU page table 208. This memory access attempt causes a PPU-based page fault, which then causes a fault buffer entry in fault buffer 216 to be written. In response, the PPU error handler 215 reads the PSD 210 entry corresponding to the virtual memory address and determines the memory page associated with the virtual memory address. After reading the PSD 210, the PPU error handler 215 determines that the current state of ownership for the memory page associated with the virtual memory address is CPU owned. Based on the current ownership state, as well as other factors, such as usage characteristics for the page or the type of memory access, the PPU error handler 215 determines that a new ownership state for the page is PPU native.

The PPUs 202 writes a fault buffer entry to the fault buffer 216 indicating that the PPU 202 generated a page fault and indicating the virtual memory address associated with the page fault. The PPU fault handler 215 executing on the CPU 102 reads the fault buffer entry and in response the CPU 102 removes the allocation in the CPU page table 206 associated with the virtual memory address that caused the page fault. CPU 102 may flush caches before and/or after removing the association. CPU 102 may also write commands to command queue 214 that instruct PPU 202 to copy the memory page from system memory 104 to PPU memory 204 . The copy unit 212 in the PPU 202 reads the commands in the command queue 214 and copies the page from the system memory 104 to the PPU memory 204. The PPU 202 writes a page table entry in the PPU page table 208 corresponding to the virtual memory address and the associates the virtual memory address with the newly copied memory page in PPU memory 204. Writing to the PPU page table 208 can be done by the copy unit 212 . Alternatively, the CPU 102 can update the PPU page table 208. The PPU error handler 215 also modifies the PSD 210 entry for this memory page to indicate that the ownership state is PPU-native. In some embodiments, the entries in TLBs in PPU 202 or CPU 102 may be invalidated to account for the case where the translations have been cached. At this point the page fault sequence is complete. The ownership state for the memory page is PPU Own, which means that the memory page can only be accessed by the PPU 202. Only the PPU page table 208 contains an entry linking the virtual memory address to the memory page.

A fault by the CPU 102 can initiate a transition from PPU native to CPU native. Prior to such a transition, an operation executing in the CPU 102 attempts a memory access to a virtual memory address that is not mapped in the CPU page table 206, causing a CPU-based page fault. The CPU error handler 211 reads the PSD 210 entry corresponding to the virtual memory address and determines the memory page associated with the virtual memory address. After reading the PSD 210, the CPU error handler 211 determines that the current state of ownership for the memory page associated with the virtual memory address is PPU owned. Based on current ownership status and other factors, such as usage characteristics for the page or the type of access, the CPU error handler 211 determines that a new ownership state for the page is CPU native.

The CPU error handling unit 211 changes the ownership state associated with the memory page to CPU own. The CPU error handler unit 211 writes an instruction to the instruction queue 214 to cause the copy unit 212 to remove the entry from the PPU page table 208 that links the virtual memory address to the memory page. Various TLB entries can be invalidated. The CPU error handler unit 211 also copies the memory page from the PPU memory 204 to the system memory 104, which can be accomplished by means of the command queue 214 and the copy unit 212. The CPU error handler 211 writes a page table entry into the CPU page table 206 that associates the virtual memory address with the memory page being copied into the system memory 104. The CPU error handler 211 also updates the PSD 210 to associate the virtual memory address with the newly copied memory page. At this point the page fault sequence is complete. The ownership state for the memory page is CPU owned, which means that only the CPU 102 can access the memory page. Only the CPU page table 206 contains an entry linking the virtual memory address to the memory page.

A fault by the CPU 102 can cause a transition from PPU native to CPU common. Prior to such a transition, an operation executing in the CPU 102 attempts to access memory at a virtual memory address not mapped in the CPU page table 206, causing a CPU-based page fault. The CPU error handler 211 reads the PSD 210 entry corresponding to the virtual memory address and determines the memory page associated with the virtual memory address. After reading the PSD 210, the CPU error handler 211 determines that the current state of ownership for the memory page associated with the virtual memory address is PPU owned. Based on the current ownership state or type of access, as well as other factors such as usage characteristics for the page, the CPU error handler 211 determines that a new ownership state for the memory page is CPU-common.

The CPU error handler 211 changes the ownership state associated with the memory page to CPU shared. The CPU error handler unit 211 writes an instruction to the instruction queue 214 to cause the copy unit 212 to remove the entry from the PPU page table 208 that links the virtual memory address to the memory page. Various TLB entries can be set as invalid. The CPU error handling unit 211 also copies the memory page from the PPU memory 204 to the system memory 104. This copy operation can be performed by means of the command queue 214 and the copy unit 212. The CPU error handler unit 211 then writes an instruction to the instruction queue 214 to cause the copy unit 212 to change the entry in the PPU page table 208 so that the virtual memory address is linked to the memory page in the system memory 104. Various TLB entries can be invalidated. The CPU error handler 211 writes a page table entry into the CPU page table 206 to associate the virtual memory address with the memory page in the system memory 104. The CPU error handler 211 also updates the PSD 210 to associate the virtual memory address with the memory page in the system memory 104. At this point the page fault sequence is complete. Ownership for the page is CPU shared and the memory page has been copied to system memory 104 . The CPU 102 can access the page because the CPU page table 206 contains an entry that links the virtual memory address to the memory page in system memory 104 . The page is also accessible to the PPU 202 because the PPU page table 208 contains an entry that links the virtual memory address to the memory page in the system memory 104.

Detailed example of a page fault sequence

In this regard, a detailed description of a page fault sequence performed by the PPU fault handler 215 in the event of a transition from CPU native to CPU common is now provided to show how atomic operations and transition states are used can be used to more effectively manage a page fault sequence. The page fault sequence is triggered by a thread in the PPU 202 attempting to access a virtual address for which a mapping in the PPU page table 208 does not exist. When a thread attempts to access data via a virtual memory address, the PPU 202 (particularly a user-level thread) requests a translation from the PPU page table 208 . In response, a PPU page fault occurs because the PPU page table 208 does not contain an allocation associated with the requested virtual memory address.

After the page fault occurs, the thread enters a trap state, stops, and the PPU fault handler 215 executes a page fault sequence. The PPU error handler 215 reads the PSD 210 to determine which memory page is associated with the virtual memory address and to determine the state for the virtual memory address. The PPU error handler 215 determines from the PSD 210 that the ownership state for this memory page is CPU native. Consequently, the data requested by the PPU 202 cannot be accessed by the CPU 202 via a virtual memory address. State information for the memory page also indicates that the requested data cannot be moved to the PPU memory 204 .

Based on the state information obtained from the PSD 210, the PPU error handler 215 determines that a new state for the memory page should be CPU common. The PPU error handler 215 changes the state to transition to CPU shared. This state indicates that the page is currently in the process of transitioning to CPU shared. When the PPU error handler 215 executes in a microcontroller in the memory management unit, two processors update the PSD 210 in an asynchronous manner, applying atomic compare-and-swap ("CAS") operations to the PSD 210 to update the state Change "GPU-visible transition" (CPU-shared).

The PPU 202 updates the PPU page table 208 to associate the virtual address with the memory page. The PPU 202 also invalidates the TLB cache entries. Next, the PPU 202 performs another atomic compare-and-swap operation on the PSD 210 to set the ownership state associated with the memory page to CPU-shared. Eventually, the page fault frequency ends and the thread that requested the virtual memory address data resumes processing.

error buffer

Page fault recovery generated by the CPU 102 does not involve the fault buffer 216. However, page fault recovery generated by the PPU MMU 213 does involve the fault buffer 216. The role of the fault buffer 216 in page fault recovery, generated by the PPU MMU 213 is described in more detail below with reference to the 5 and 6 described.

5 FIG. 5 shows a virtual memory system 500 for storing errors in an error buffer according to an embodiment of the present invention. As shown, the virtual memory system 500 includes a PPU error handler 215, an error buffer 216, and a PPU 202 with multiple data stream multiprocessors 504.

The fault buffer 216 stores fault buffer entries 502 that indicate information related to page faults generated by the PPU 202. FIG. The error buffer entries 502 may include, for example, the type of access attempted (e.g., read, write, or atomic), the virtual memory address for which an attempted access caused a page fault, the virtual address space, and an indication of a thread halt that caused a page fault did contain. During operation, if the PPU 202 generates a page fault, the PPU 202 may write an error buffer entry 502 to the error buffer 216 to inform the PPU error handler 215 of the memory page generating the error and the type of access that generated the error. The PPU error handler 215 then takes actions to recover from the page fault. The error buffer 216 may store multiple errors as the PPU 202 executes multiple threads, where each thread may generate one or more errors due to the pipelined nature of the PPU 202 memory accesses. Each of the error buffer entries 502 may be generated by one or more data stream multiprocessors 504 included in the PPU 202.

6 FIG. 6 shows a virtual memory system 600 for recovering from page faults generated by the PPU 202 according to an embodiment of the present invention. As shown, the virtual memory system 600 includes a PPU error handling unit 215, an error buffer 216, a system memory 104 with a command queue 214, and a PPU 202 with a copy unit 212.

The PPU error handler 215 reads error buffer entries 502 stored in the error buffer 216 to determine how to resolve the page faults associated with the error buffer entries 502. To recover from the page faults, the PPU fault handler 215 executes a page fault sequence to, among other things, remove a PSD entry associated with memory pages that corresponding to the error buffer entries 502, and/or to move memory pages associated with the error buffer entries 502. During a page fault sequence, the CPU 102 or the PPU 202 may write instructions to the instruction queue 214 for the copy unit 212 to execute. Such an approach frees the CPU 102 or PPU 202 to perform other tasks while the copy unit 212 reads and executes the instructions stored in the instruction queue 214, and allows all instructions for an error sequence to be queued at the same time which avoids monitoring the progress of the error sequence. Instructions executed by copy unit 212 may include, but are not limited to, deleting, creating or modifying page table entries in PPU page table 208, reading or writing data from system memory 104, and reading or writing data to the PPU -include memory 204.

CPU 102 and PPU 202 may separately perform context switching. In other words, a PPU 202 may write an error buffer entry to the error buffer 216 in response to an error being detected. This error buffer may not be processed immediately by the PPU error handling unit 215 in the CPU. Instead, the CPU 102 can perform other processing tasks and then handle the PPU errors. Therefore, the CPU 102 and the PPU 202 may not necessarily be operating concurrently in the same context. In other words, the CPU 102 can execute a process that is different from the process that generated the work currently executing in the PPU 202 . To inform the PPU error handler 215 which process belongs to the work of the PPU 202 that created an error buffer entry 502, the PPU 202 provides an instance pointer with the error buffer entry to inform the CPU 102 of the address space in which the PPU 202 generated the error. Fault buffer 216 may contain multiple page fault entries associated with the same memory page, since multiple stream multiprocessors 504 are executing in parallel and may generate page faults affecting the same memory page. The PPU error handler 215 examines the error buffer 216 to determine which errors to fix.

Variations in UVM system architecture

Various modifications to the unified virtual memory system 200 are possible. For example, in some embodiments, after writing an error buffer entry to error buffer 216, PPU 202 may issue a CPU interrupt to cause CPU 102 to read error buffer entries in error buffer 216 and perform any operations appropriate in response to the error buffer entry. In other embodiments, CPU 102 may poll error buffer 216 periodically. In the event CPU 102 finds an error buffer entry in error buffer 216, CPU 102 performs a series of operations in response to the error buffer entry.

In some embodiments, system memory 104 stores PPU page table 208 instead of PPU memory 204. In other embodiments, a single or multiple level cache hierarchy, such as a single or multiple level translation lookaside buffer (TLB) hierarchy ( not shown) can be implemented to store virtual memory translations for the CPU page table 206 or the PPU page table 208.

In still other embodiments, in the event that a thread executing on PPU 202 causes a PPU fault (a "faulting thread"), PPU 202 may take one or more actions. These actions include: stopping the entire PPU 202, stopping the SM executing the fault-producing thread, stopping the PPU MMU 213, stopping only the fault-producing thread, stopping a group of threads that includes the fault-producing thread, or stopping one or more levels by TLB. In some embodiments, after a PPU page fault has occurred and a page fault sequence is executed by the unified virtual memory system 200, execution of the faulting thread continues and the faulting thread again attempts to execute the memory access request that caused the page fault. In some embodiments, stopping a TLB is performed so that it appears as a long-running memory access to the faulting SM or thread, so that the SM does not have to perform a special operation on the fault.

Finally, in other alternative embodiments, the UVM driver 101 may contain instructions that cause the CPU 102 to perform one or more operations to manage the UVM system 200 and recover from a page fault, such as accessing the CPU page table 206, the PSD 210 and/or the fault buffer 216. In other embodiments, an operating system kernel (not shown) may be configured to manage the UVM system 200 and recover from a page fault by accessing the CPU page table 206, the PSD 210, and/or the error buffer 216 is accessed. In still other embodiments, an operating system kernel may work in conjunction with UVM driver 101 to manage UVM system 200 and recover a page fault by accessing CPU page table 206, PSD 210, and/or fault buffer 21 is accessed.

7 12 is a flow diagram of method steps for managing virtual memory to physical memory mappings using a page state map according to an embodiment of the present invention. Although the process steps in connection with the 1-6 are described, those skilled in the art will recognize that any system suitable for performing the method steps in any order is within the scope of the present invention.

As shown, method 700 begins at step 702 in which a PPU 202 performs a first operation that addresses a first virtual memory address. In step 704, the PPU MMU 213 reads the PPU page table 208 and determines that the PPU page table 208 does not contain an allocation associated with the first virtual memory price. The PPU MMU 213 generates a first page fault when making this determination. In step 706, the PPU error handler 215 fixes page faults and inserts commands into the command queue 214, the copy unit 212 in the PPU 202 reads the command queue 214 to determine a mapping corresponding to the first virtual memory address. In step 708, the copy unit 212 updates the PPU page table 208 to contain the allocation.

8th Figure 12 provides a flowchart of method steps for tracking page faults according to an embodiment of the present invention. Although the process steps in connection with the 1-6 are described, those skilled in the art will recognize that any system suitable for performing the method steps in any order is within the scope of the present invention.

As shown, the method 800 begins at step 802 in which a PPU 202 executes a first command associated with a first virtual memory address. In step 804, the PPU MMU 213 determines that the PPU page table 208 does not contain a first mapping associated with the first virtual memory address. In step 805, the stream multiprocessor 504 or other entity executing the first instruction is halted. In step 806, the PPU 202 sends a first page fault to the fault buffer 216.

9 Figure 12 provides a flowchart of method steps for recovering from page faults with a fault buffer according to an embodiment of the present invention. Although the process steps in connection with the 1-6 are described, those skilled in the art will recognize that any system configured to perform the method steps in any order is within the scope of the present invention.

As shown, the method 900 begins at a step 902 in which an error buffer 216 stores a plurality of error buffer entries. In step 904, a PPU error handling unit 215 reads the error buffer entries to handle the error buffer entries. In step 906, the PPU error handler 215 determines what steps to take to handle the error buffer and initiates a page fault sequence to fix one or more page faults associated with the error buffer entries. In step 908, the PPU error handler 215 sends commands to a command queue 214 to update the PPU page table 208. FIG. At step 910, the stream multiprocessor 504 or other entity that was halted continues execution.

10 12 shows a flowchart of method steps for creating and managing common pointers in a virtual memory architecture according to an embodiment of the present invention. Although the process steps in connection with the 1-6 are described, those skilled in the art will recognize that any system suitable for performing the method steps in any order is within the scope of the present invention.

As shown, the method 1000 begins at step 1002 in which the UVM driver 101 stores a first entry for a page state table containing a mapping between a first virtual memory address and a first physical memory address. In step 1004, the CPU MMU 209 or the PPU MMU 213 translates the first virtual memory address into the first physical address based on the first page state dictionary. In step 1005, a memory page associated with the first virtual address is copied or moved. In step 1006, the UVM driver 101 stores a second page state table entry (or alternatively modifies the first page state table entry) containing a mapping between the first virtual memory address and a second physical memory address. This second state table entry is stored in response to the modification of the state of the memory page linked to the first page state table entry. For example, the memory page can be moved from one memory unit to another, or copied from one memory unit to another. In step 1008, the CPU MMU 209 or the PPU MMU 213 translates the first virtual memory address to the second physical memory address based on the second page state table entry.

11 Figure 12 provides a flowchart of method steps for managing ownership in a virtual memory subsystem according to an embodiment of the present invention. Although the process steps in connection with the 1 - 6 are described, those skilled in the art will recognize that any system configured to perform the method steps in any order is within the scope of the present invention.

As shown, method 1100 begins at step 1102 in which CPU MMU 209 or PPU MMU 213 issues a page fault in response to a CPU 102 or PPU 202 memory access request. In step 1104, the CPU error handler 211 or the PPU error handler 215 locates an entry in a page state table 210 that corresponds to a memory page associated with the virtual memory address. In step 1107 the ownership state of the memory page is modified in the page state table 210 and the memory page is moved if necessary. In step 1106, the CPU error handler 211 or the PPU error handler 215 reads an ownership state associated with the memory page from the page state table 210 entry. At step 1108, the CPU error handler 211 or the PPU error handler 115 updates the local page table to include an entry corresponding to the virtual memory address and associates the virtual memory address with the memory page.

In summary, a unified virtual memory system is provided that manages memory in a common manner for a CPU and one or more PPUs. The unified virtual memory system includes a page state table that stores mappings contained in a page table associated with the CPU and a page table associated with the PPU. When a PPU or the CPU triggers a page fault, the page state table is available to provide the state of memory pages associated with the page fault. Furthermore, when a PPU triggers a page fault, the PPU sends a page fault to a fault buffer. The PPU error handler examines the contents of the error buffer to fix the page faults. The provision of a fault buffer allows the PPU fault handler to "merge" page faults executed by the PPU. Furthermore, a unified virtual memory driver manages the page state map and associated virtual memory addresses such that virtual memory addresses are common to the CPU and the PPU. Finally, the unified virtual memory driver implements a migration scheme that moves memory pages based on usage by the CPU and PPU.

An advantage of the disclosed approach is that user-level applications do not need to keep track of multiple pointers depending on where particular data items are stored. Another advantage is that memory pages are moved between memory devices based on usage, allowing memory pages to be localized to devices that access the memory pages more frequently. Another advantage is that an error buffer is provided, allowing errors generated by the PPU to be pooled for efficient handling.

An embodiment of the invention may be implemented as a program product for use in a computer system. The program(s) of the program product define functions of the embodiments (including the methods described herein) and may be embodied on a variety of computer-readable storage media. Illustrative computer-readable storage media include, without limitation: (i) non-writable storage media (e.g., read-only memory devices in a computer, such as CD-ROM disks readable by a CD-ROM drive, flash memory, ROM chips or some other type of non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks in a floppy disk drive or a hard disk drive or other type of random access semiconductor memory) on which alterable information is stored.

Claims (10)

A computer-implemented method for accessing data in a virtual storage subsystem, the method comprising: issuing a page fault in response to a memory access request, wherein a local page table (208) does not contain an entry corresponding to a virtual memory address contained in the memory access request; responsive to the page fault, executing a page fault sequence that includes: finding an entry in a page state table (210) corresponding to a memory page associated with the virtual memory address; retrieving an ownership state associated with the memory page from the entry in the page state table (210), the ownership state being determined based on a usage history of the memory page; and updating the local page table (208) to include an entry corresponding to the virtual memory address and linking the virtual memory address to the memory page. The procedure after claim 1 wherein: prior to execution of the page fault sequence, the ownership state associated with the memory page indicates that the memory page is central processing unit owned (CPU owned); Executing the page fault sequence includes: modifying the ownership state associated with the memory page to CPU shared; and the local memory table (208) comprises a parallel processing unit (PPU) page table. The procedure after claim 2 , wherein the memory page resides in a system memory (104), and wherein the PPU page table and a CPU page table (206) have entries associating the virtual memory address with the memory page. The procedure after claim 1 , wherein: prior to execution of the page fault sequence, the ownership state associated with the memory page indicates that the memory page is CPU-owned; Executing the page fault sequence further comprises: modifying the ownership state associated with the memory page to PPU owned; and the local page table (208) comprises a PPU page table. The procedure after claim 4 , wherein execution of the page fault sequence further comprises: moving the memory page from a system memory to a PPU memory. The procedure after claim 1 , wherein: prior to execution of the page fault sequence, the ownership state associated with the memory page indicates that the memory page is PPU owned; Executing the page fault sequence further comprises: modifying the ownership state associated with the memory page on CPU native; and the local page table comprises a CPU page table (206). The procedure after claim 6 , wherein executing the page fault sequence further comprises: moving the memory page from a PPU memory (204) to a system memory (104). The procedure after claim 1 , wherein: prior to execution of the page fault sequence, the ownership state associated with the memory page indicates that the memory page is PPU owned; Executing the page fault sequence further comprises: modifying the ownership state associated with the memory page to CPU shared; and the local page table (208) comprises a CPU page table. The procedure after claim 8 , wherein executing the page fault sequence further comprises: moving the memory page from a PPU memory (204) to a system memory (104). A system for accessing data in a virtual memory subsystem, comprising: a local page table (208); and a first processing unit configured to: determine that the local memory table (208) does not contain an entry corresponding to a virtual memory address, issue a page fault in response to determining that the local page table (208) contains the entry does not contain; determining an entry in a page state table (210) corresponding to a memory page associated with the virtual memory address; retrieving an ownership state associated with the memory page from the entry in the page state table (210), the ownership state being determined based on a usage history of the memory page; and update the local page table (208) to include an entry corresponding to the virtual memory address and linking the virtual memory address to the memory page.

Images