Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
  • G06F12/02—Addressing or allocation; Relocation
  • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
  • G06F12/10—Address translation
  • G06F12/109—Address translation for multiple virtual address spaces, e.g. segmentation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
  • G06F12/02—Addressing or allocation; Relocation
  • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
  • G06F12/023—Free address space management
  • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
  • G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
  • G06F12/02—Addressing or allocation; Relocation
  • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
  • G06F12/10—Address translation
  • G06F12/1009—Address translation using page tables, e.g. page table structures
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
  • G06F12/02—Addressing or allocation; Relocation
  • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
  • G06F12/10—Address translation
  • G06F12/1081—Address translation for peripheral access to main memory, e.g. direct memory access [DMA]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
  • G06F9/54—Interprogram communication
  • G06F9/544—Buffers; Shared memory; Pipes
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
  • G06F9/54—Interprogram communication
  • G06F9/545—Interprogram communication where tasks reside in different layers, e.g. user- and kernel-space
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
  • G06F2212/65—Details of virtual memory and virtual address translation
  • G06F2212/657—Virtual address space management

Definitions

• the compute device converts the pointer to the local virtual address space with the memory management unit (MMU) page table mapping. This renders the sharing of the pointer itself or of structures containing pointers inefficient and challenging.
• MMU memory management unit
• FIG. 1 is a block diagram of one implementation of a computing system.
• FIG. 2 is a diagram of one implementation of creating a unified kernel virtual address space for heterogeneous computing.
• FIG. 3 is a diagram of one implementation of sharing buffers between two separate subsystems.
• FIG. 4 is a diagram of one implementation of mapping regions of memory to multiple kernel address spaces.
• FIG. 5 is a generalized flow diagram illustrating one implementation of a method for creating a unified kernel virtual address space.
• FIG. 6 is a generalized flow diagram illustrating one implementation of a method for enabling a common shared region in multiple kernel address spaces.
• FIG. 7 is a generalized flow diagram illustrating one implementation of a method for enabling buffer pointer sharing between two different subsystems. DETAILED DESCRIPTION OF IMPLEMENTATIONS
• a system includes at least a first subsystem running a first kernel, an input/output memory management unit (IOMMU), and a second subsystem running a second kernel.
• IOMMU input/output memory management unit
• the IOMMU creates a unified kernel address space allowing the first and second subsystems to share memory buffers at the kernel level.
• the first subsystem allocates a block of memory in part of the system memory controlled by the first subsystem.
• a first mapping is created from a first logical address of the first kernel address space of the first subsystem to the block of memory.
• the IOMMU creates a second mapping to map the physical address of that block of memory from a second logical address of the second kernel address space of the second subsystem. These mappings allow the first and second subsystems to share buffer pointers in the kernel address space which reference the block of memory.
• computing system 100 includes at least first subsystem 110, second subsystem 115, input/output (I/O) interface(s) 120, input/output memory management unit (IOMMU) 125, memory subsystem 130, and peripheral device(s) 135.
• computing system 100 can include other components and/or computing system 100 can be arranged differently.
• first and second subsystems 110 and 115 have different kernel address spaces, but a unified kernel address space is created by IOMMU 125 for the first and second subsystems 110 and 115.
• the unified kernel address space allows the first and second subsystems 110 and 115 to pass pointers between each other and share buffers.
• their respective kernel address space includes kernel logical addresses and kernel virtual addresses.
• kernel logical addresses and its associated physical address differ by a constant offset. Kernel virtual addresses do not necessarily have a linear, one-to-one mapping to physical addresses that characterize kernel logical addresses. All kernel logical addresses are kernel virtual addresses, but kernel virtual addresses are not necessarily kernel logical addresses.
• each of first subsystem 110 and second subsystem 115 includes one or more processors which execute an operating system.
• the processor(s) also execute one or more software programs in various implementations.
• the processor(s) of first subsystem 110 and second subsystem 115 include any number and type of processing units (e.g., central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), field programmable gate array (FPGA), application specific integrated circuit (ASIC)).
• first subsystem 110 includes memory management unit (MMU) 112 and second subsystem 115 includes MMU 117, with each MMU handling virtual to physical address translations for its corresponding subsystem.
• MMU memory management unit
• first and second subsystems 110 and 115 have different kernel address spaces
• a unified kernel address space is created by IOMMU 125 for the first and second subsystems 110 and 115, allowing the first and second subsystems 110 and 115 to pass pointers back and forth and share buffers in memory subsystem 130.
• Memory subsystem 130 includes any number and type of memory devices.
• the type of memory in memory subsystem 130 can include high-bandwidth memory (HBM), non-volatile memory (NVM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), NAND Flash memory, NOR flash memory, Ferroelectric Random Access Memory (FeRAM), or others.
• I/O interfaces 120 are representative of any number and type of PO interfaces (e.g., peripheral component interconnect (PCI) bus, PCI-Extended (PCT X), PCIE (PCI Express) bus, gigabit Ethernet (GBE) bus, universal serial bus (USB)).
• Various types of peripheral devices 135 can be coupled to PO interfaces 120.
• Such peripheral devices 135 include (but are not limited to) displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, and so forth.
• a block of memory is allocated in a part of system memory managed by first subsystem 110. After the initial block of memory is allocated, the proper I/O virtual address (VA) is assigned to the second subsystem 115.
• VA I/O virtual address
• an IOMMU mapping is created from the kernel address space of second subsystem 115 to the physical address of the block of memory.
• IOMMU 125 performs the virtual address mapping for the second subsystem 115 to the block of memory. Then, when additional memory is allocated, the heap allocate function is called, and the address is mapped based on the same I/O VA address that was earlier created. Then, a message is sent to the second subsystem 115 notifying the second subsystem 115 of the unified address.
• computing system 100 is a computer, laptop, mobile device, game console, server, streaming device, wearable device, or any of various other types of computing systems or devices. It is noted that the number of components of computing system 100 varies from implementation to implementation. In other implementations, there are more or fewer of each component than the number shown in FIG. 1. It is also noted that in other implementations, computing system 100 includes other components not shown in FIG. 1. Additionally, in other implementations, computing system 100 is structured in other ways than shown in FIG. 1.
• FIG. 2 a diagram of one implementation of creating a unified kernel virtual address space for heterogeneous computing is shown.
• the dashed vertical lines from the top of FIG. 2, from left to right, are representative of a first subsystem 202, a MMU 204 of the first subsystem 202, a device driver 206 executing on the first subsystem 202, a shared region 208 of memory, an IOMMU 211, a MMU 212 of a second subsystem 214, and the second subsystem 214.
• first subsystem 202 has a first operating system
• second subsystem 214 has a second operating system
• the second operating system is different from the first operating system.
• first subsystem 202 and the second subsystem 214 are part of a heterogeneous computing system.
• first subsystem 202 and second subsystem 214 are each performing some portion of a workload, and in order to perform the workload, first subsystem 202 and second subsystem 214 share buffer pointers and buffers between each other.
• the diagram of FIG. 2 illustrates one example of allocating memory to be shared between first subsystem 202 and second subsystem 214.
• a first step 205 is performed by device driver 206 to create a heap (i.e., a shared memory region).
• a heap i.e., a shared memory region.
• the term“heap” is defined as a virtual memory pool that is mapped to a physical memory pool.
• a desired size of the heap is allocated in the physical memory subsystem.
• a mapping from the kernel to the heap is created.
• a new flag indicates if the heap should come from the kernel logical address space. For example, the km alloc function returns memory in the kernel logical address space.
• the memory manager manages the buffer allocated for the heap using the Linux genpool library.
• the heap_map_kernel function maps the buffer using kmap instead of vmap in step 220.
• the function kmap maps the buffer to a given virtual address based on a logical mapping.
• the selection of kmap or vmap is controlled by a new flag given during carveout heap creation.
• the heap is created together with the kernel mapping using the genpool library, the carveout application programming interface (API), and the new flag. In this case, the kernel map returns the pre-mapped address.
• any shared memory buffer allocated by first subsystem 202 is managed similar to the carveout heap wrapped genpool.
• the new API allows the addition of an external buffer to the carveout heap.
• Local tasks on second subsystem 214 allocate from this carveout heap using the same API.
• step 220 the allocation of the shared memory region is complete.
• step 225 an input/output (I/O) virtual address is allocated by IOMMU 211 for the shared memory region.
• I/O input/output
• step 230 a contiguous block of DMA address space is reserved for the shared memory region.
• step 235 the shared memory region is mapped by IOMMU 211 to the kernel address space of second subsystem 214. It is noted that the IOMMU mapping should not be freed until the device driver 206 has been shutdown.
• the mapping from the kernel address space of first subsystem 202 to the shared memory region is invalidated in step 240.
• step 245 the kernel address space mapping is freed by executing a memory release function.
• FIG. 3 a diagram of one implementation of sharing buffers between two separate subsystems is shown.
• the dashed vertical lines extending down from the top of FIG. 3, from left to right, are representative of a first subsystem 302, a MMU 304 of the first subsystem 302, a device driver 306 executing on the first subsystem 302, a shared region 308 of memory, an IOMMU 310, a MMU 312 of a second subsystem 314, and the second subsystem 314. It is assumed for the purposes of this discussion that first subsystem 302 has a first operating system, second subsystem 314 has a second operating system, and the second operating system is different from the first operating system.
• a block of memory is allocated and a mapping from the kernel address space of first subsystem 302 to the physical address of the memory block is created by device driver 306.
• the mapping is then maintained by MMU 304.
• the memory block can be assigned to second subsystem 314 exclusively, shared between first subsystem 302 and second subsystem 314, or assigned to first subsystem 302 exclusively.
• a message is sent from first subsystem 302 to second subsystem 314 with the address and size of the block of memory. In one implementation, the message can be sent out-of-band.
• a mapping from the kernel address space of second subsystem 314 to the physical address of the memory block is created and maintained by MMU 312.
• For loop exchange 315 data is exchanged between first subsystem 302 and second subsystem 314 using the shared region 308.
• the buffer pointers l st _SS_buf and 2 nd _SS_buf are the same and can be freely exchanged between first subsystem 302 and second subsystem 314 and further partitioned using the genpool library.
• a heterogeneous computing system (e.g., system 100 of FIG. 1) includes multiple different subsystems with their own independent operating systems.
• the vertically oriented rectangular blocks shown in FIG. 4 are representative of the address spaces for the different components of the heterogeneous computing system. From left to right, the address spaces shown in FIG. 4 are: first subsystem virtual address space 402, physical memory space 404, device memory space 406, and second subsystem virtual address space 408.
• first subsystem virtual address space 402 includes shared region 420 which is mapped to memory block 425 and memory block 430 of physical memory space 404.
• the mappings of shared region 420 to memory block 425 and memory block 430 are created and maintained by first subsystem MMU 412.
• memory block 425 and memory block 430 are mapped to shared region 435 of device memory space 406 by IOMMU 414.
• shared region 435 is mapped to shared region 440 of second subsystem virtual address space 408 by second subsystem MMU 416.
• the first subsystem and the second subsystem are able to share buffer pointers with each other in their kernel address space.
• FIG. 5 one implementation of a method 500 for creating a unified kernel virtual address space is shown.
• steps in this implementation and those of FIG. 6-7 are shown in sequential order.
• one or more of the elements described are performed concurrently, in a different order than shown, or are omitted entirely.
• Other additional elements are also performed as desired. Any of the various systems or apparatuses described herein are configured to implement method 500.
• a first subsystem allocates a block of memory at a first physical address in a physical address space corresponding to a memory subsystem (block 505).
• the first subsystem executes a first operating system with a first kernel address space.
• the first subsystem creates a mapping of a first logical address in the first kernel address space to the first physical address (block 510). It is noted that the first logical address space of the first kernel address space is a first linear offset from the first physical address.
• an IOMMU creates an IOMMU mapping of a second logical address in a second kernel address space to the first physical address (block 515).
• the second kernel address space is associated with a second subsystem. It is noted that the second logical address space of the second kernel address space is a second linear offset from the first physical address.
• the first subsystem conveys a buffer pointer to the second subsystem, where the buffer pointer points to the first logical address (block 520). Then, the second subsystem generates an access request with the buffer pointer and conveys the access request to the IOMMU (block 525). Next, the IOMMU translates a virtual address of the buffer pointer to the first physical address using the previously created IOMMU mapping (block 530). Then, the second subsystem accesses the block of memory at the first physical address (block 535). After block 535, method 500 ends.
• a first subsystem allocates a block of memory at a first physical address in a physical address space corresponding to a memory subsystem (block 605). It is assumed for the purposes of this discussion that the system described by method 600 includes the first subsystem and a second subsystem. It is also assumed that the first subsystem executes a first operating system with a first kernel address space and the second subsystem executes a second operating system with a second kernel address space. The first subsystem creates a mapping of a first logical address in the first kernel address space to the first physical address, where the first logical address in the first kernel address space is a first offset away from the first physical address (block 610).
• an IOMMU selects a device address that is a second offset away from the first logical address in the second kernel address space (block 615). Then, the IOMMU creates an IOMMU mapping of the selected device address in the device address space to the first physical address (block 620). The IOMMU mapping enables a common shared region in both the first kernel address space and the second kernel address space for use by both the first subsystem and the second subsystem (block 625). After block 625, method 600 ends.
• a first subsystem maps a first region of a first kernel address space to a second region in physical address space (block 705).
• the physical address space corresponds to a system memory controlled by the first subsystem.
• an IOMMU maps a third region of a second kernel address space to a fourth region in a device address space (block 710). It is assumed for the purposes of this discussion that the second kernel address space corresponds to a second subsystem which is different from the first subsystem.
• the IOMMU maps the fourth region in the device address space to the second region in the physical address space to cause addresses in the first and third regions to point to matching (i.e., identical) addresses in the second region (block 715).
• the first subsystem and second subsystem are able to share buffer pointers in the first or second kernel address spaces that reference the second region in physical address space (block 720). After block 720, method 700 ends.
• program instructions of a software application are used to implement the methods and/or mechanisms described herein.
• program instructions executable by a general or special purpose processor are contemplated.
• such program instructions can be represented by a high level programming language in other implementations, the program instructions can be compiled from a high level programming language to a binary, intermediate, or other form.
• program instructions can he written that describe the behavior or design of hardware.
• Such program instructions can he represented by a high-level programming language, such as C.
• a hardware design language (HDL) such as Verilog can he used.
• the program instructions are stored on any of a variety of non-transitory computer readable storage mediums. The storage medium is accessible by a computing system during use to provide the program instructions to the computing system for program execution.
• a computing system includes at least one or more memories and one or more processors configured to execute program instructions.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Software Systems (AREA)
• Memory System Of A Hierarchy Structure (AREA)
• Multi Processors (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=73554876

Family Applications (1)

Country Status (6)

Families Citing this family (2)

Citations (4)

Family Cites Families (40)

• 2019
  • 2019-07-23 US US16/519,311 patent/US10853263B1/en active Active
• 2020
  • 2020-07-22 WO PCT/IB2020/056906 patent/WO2021014382A1/en not_active Ceased
  • 2020-07-22 KR KR1020227005983A patent/KR20220036972A/ko active Pending
  • 2020-07-22 EP EP20844551.0A patent/EP4004743B1/en active Active
  • 2020-07-22 CN CN202080060590.0A patent/CN114286987A/zh active Pending
  • 2020-07-22 JP JP2022503804A patent/JP7682148B2/ja active Active
  • 2020-11-25 US US17/105,331 patent/US11960410B2/en active Active

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events