US20180181340A1

US20180181340A1 - Method and apparatus for direct access from non-volatile memory to local memory

Info

Publication number: US20180181340A1
Application number: US15/389,596
Authority: US
Inventors: Nima Osqueizadeh
Original assignee: ATI Technologies ULC
Current assignee: ATI Technologies ULC
Priority date: 2016-12-23
Filing date: 2016-12-23
Publication date: 2018-06-28
Also published as: US10761736B2; US20180349057A1

Abstract

Described herein is a method and system for directly accessing and transferring data between a first memory architecture and a second memory architecture associated with a graphics processing unit (GPU) or a discrete GPU (dGPU). In particular, a method is described for transferring data between the first memory architecture and the second memory architecture that bypasses interaction with a system memory of a processor and a root complex. A transfer command is sent from the processor, (or a host agent in the GPU or dGPU), to a first memory architecture controller. The first memory architecture controller initiates the transfer of the data directly between the first memory architecture and the second memory architecture. The method bypasses: 1) a host root complex; and 2) storing the data in the system memory and then having to transfer the data to the second memory architecture or the first memory architecture.

Description

CROSS-RELATED APPLICATIONS

This application is related to co-pending application entitled “Method and Apparatus for Connecting Non-volatile Memory locally to a GPU through a Local Switch”, Attorney Docket No. 160286-US-NP, filed on same date and to co-pending application entitled “Method and Apparatus for Accessing Non-volatile Memory As Byte Addressable Memory”, Attorney Docket No. 160287-US-NP, filed on same date, and to co-pending application entitled “Method and Apparatus for Integration of Non-volatile Memory”, Attorney Docket No. 160288-US-NP, filed on same date, which are incorporated by reference as if fully set forth herein.

BACKGROUND

A graphics processing unit (GPU) may be nominally configured with a certain amount of local or dedicated memory, (hereinafter referred to as local), to service operations performed on the GPU. For example, the local memory may be dynamic random access memory. Certain applications may require the transfer of data from non-volatile memory (NVM) to the local memory. In this scenario, an operating system (OS), display driver, device driver or similar hardware/software entity of a host computing system controls or manages the data transfer process. This data transfer process entails a two hop process; first from the NVM to a host memory, and then from the host memory to the local memory. This involves at least a root complex, which increases traffic and congestion.

BRIEF DESCRIPTION OF THE DRAWINGS

A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings wherein:

FIG. 1 is a processing system with a host computing system and solid state graphics (SSG) cards in accordance with certain implementations;

FIG. 2 is a flow diagram using the processing system of FIG. 1 in accordance with certain implementations; and

FIG. 3 is a block diagram of an example device in which one or more disclosed implementations may be implemented.

DETAILED DESCRIPTION

Described herein is a method and system for directly accessing and transferring data between a first memory architecture and a second memory architecture associated with a graphics processing unit (GPU) or a discrete GPU (dGPU). The first memory architecture can be a non-volatile memory (NVM) or other similarly used memories, for example, along with associated controllers. The second memory architecture can be a local memory, a high bandwidth memory (HBM), a double data rate fourth-generation synchronous dynamic random-access memory (DDR4), a double data rate type five synchronous graphics random access memory (GDDR5), a hybrid memory cube or other similarly used memories, for example, along with associated controllers. For purposes of illustration and discussion, the terms NVM and local memory will be used in the description without limiting the scope of the specification and claims.
In particular, the method describes transferring data directly between the NVM and the local memory, which bypasses interaction with a system memory of a processor and a host system root complex. A transfer command is sent from the processor, (or a host agent in the GPU or dGPU), to a NVM controller. The NVM controller initiates transfer of the data directly between the NVM and the local memory. The method bypasses: 1) a host system root complex; and 2) storing the data in the system memory and then having to transfer the data to the local memory or NVM. In effect, a multi-hop data transfer can be accomplished in a single hop.
FIG. 1 shows an example processing system 100 in accordance with certain implementations. The processing system 100 can include a host computing system 105 that is connected to one or more solid state graphics (SSG) boards or cards 110 ₁to 110 _n. The host computing system 105 includes a processor 120, such as for example a central processing unit (CPU), which may be connected to, or in communication with, a host memory 122 such as for example random access memory (RAM). The processor 120 can include an operating system (OS), a device driver and other nominal elements. The processor 120 can also be connected to, or in communication with, a number of components, including but not limited to, a bridge 124 and storage 126. The components shown are illustrative and other components may also be connected to or be in communication with the CPU 105. The components may be connected to or be in communication with the processor 120 using, for example, a high-speed serial computer expansion bus, such as but not limited to, a Peripheral Component Interconnect Express (PCIe) root complex and switch (collectively PCIe switch) 128. The PCIe switch 128 is shown for purposes of illustration and other electrical or communication interfaces may be used.
Each SSG board 110 ₁to 110 _nincludes a PCIe switch 136 ₁to 136 _n.for interfacing with PCIe switch 128. Each PCIe switch 136 ₁to 136 _n.can be connected to or in communication with one or more non-volatile memory (NVM) controllers 134 ₁to 134 _k, such as for example, a NVM Express (NVMe) or Non-Volatile Memory Host Controller Interface Specification (NVMHCI) device, for accessing associated NVMs 135 ₁to 135 _kand can also be connected to one or more dGPUs 130 ₁to 130 _m. Each dGPU 130 ₁to 130 _mis further connected to an associated local memory 132 ₁to 132 _mEach NVM controller 134 ₁to 134 _kcan manage and access an associated NVM 135 ₁to 135 _kand in particular, can decode incoming commands from host computing system 105 or dGPU 130 ₁to 130 _mas described herein below. The SSG board described herein is illustrative and other configurations can be used without departing from the scope of the description and claims. Further configurations are described in co-pending application entitled “Method and Apparatus for Connecting Non-volatile Memory locally to a GPU through a Local Switch, Attorney Docket No. 160286-US-NP, which is incorporated by reference as if fully set forth.
Operationally, when a dGPU of the one or more dGPUs 130 ₁to 130 _mis executing commands that require data transfer between an associated local memory and one or more NVMs 135 ₁to 135 _k, then the processor 120 can instruct or enable direct data transfer from the associated local memory 132 ₁to 132 _mto one or more NVMs 135 ₁to 135 _k(arrow 142) or from one or more NVMs 135 ₁to 135 _kto the associated local memory (arrow 140). The direct data transfer can be initiated by an appropriate NVM controller 134 ₁to 134 _kvia a local PCIe switch, such as for example, PCIe switch 136 ₁to 136 _n. In an implementation, the dGPU can have a hardware agent that can instruct the direct data transfer. This peer-to-peer data transfer or access can alleviate the disadvantages discussed herein. As shown in FIG. 1, this process uses a single hop data transfer process, from local memory 132 ₁to 132 _mto NVM 135 ₁to 135 _k. That is, the data transfer can be executed locally with respect to the dGPU without involvement of processor 120 or PCIe root complex/switch 128 during execution of the data transfer. Moreover, since data transfer to and from local memory 132 ₁to 132 _mis initiated by an appropriate NVM controller 134 ₁to 134 _k. This may increase the efficiency of the processor 120 as it is not involved in the actual transfer of the data, increase the efficiency of dGPU 130 ₁to 130 _mas it is not using resources, such as for example memory management resources for initiating and executing the data transfer, decrease system latency and increase system performance.
FIG. 2, in concert with FIG. 1, shows an example flowchart 200 for transferring data directly between local memory 132 ₁to 132 _mand one or more NVMs 135 ₁to 135 _k. As commands are executed by one or more dGPUs 130 ₁to 130 _m, certain commands may need access between to one or more NVMs 135 ₁to 135 _k(step 205). A data transfer command is sent by a processor 120 or hardware agents in one or more dGPUs 130 ₁to 130 _mto appropriate NVM controllers 134 ₁to 134 _k(step 210). The appropriate NVM controller 134 ₁to 134 _kinitiates the data transfer (step 215). The data is transferred between local memory 132 ₁to 132 _mand one or more NVMs 135 ₁to 135 _kvia a local PCIe switch 136 ₁to 136 _n, as appropriate (step 220).
FIG. 3 is a block diagram of an example device 300 in which one portion of one or more disclosed implementations may be implemented. The device 300 may include, for example, a head mounted device, a server, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. The device 300 includes a processor 302, a memory 304, a storage 306, one or more input devices 308, and one or more output devices 310. The device 300 may also optionally include an input driver 312 and an output driver 314. It is understood that the device 300 may include additional components not shown in FIG. 3.
The processor 302 may include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, wherein each processor core may be a CPU or a GPU. The memory 304 may be located on the same die as the processor 302, or may be located separately from the processor 302. The memory 304 may include a volatile or non-volatile memory, for example, random access memory (RAM), dynamic RAM, or a cache.
The storage 306 may include a fixed or removable storage, for example, a hard disk drive, a solid state drive, an optical disk, or a flash drive. The input devices 308 may include a keyboard, a keypad, a touch screen, a touch pad, a detector, a microphone, an accelerometer, a gyroscope, a biometric scanner, or a network connection (e.g., a wireless local area network card for transmission and/or reception of wireless IEEE 802 signals). The output devices 310 may include a display, a speaker, a printer, a haptic feedback device, one or more lights, an antenna, or a network connection (e.g., a wireless local area network card for transmission and/or reception of wireless IEEE 802 signals).
The input driver 312 communicates with the processor 302 and the input devices 308, and permits the processor 302 to receive input from the input devices 308. The output driver 314 communicates with the processor 302 and the output devices 310, and permits the processor 302 to send output to the output devices 310. It is noted that the input driver 312 and the output driver 314 are optional components, and that the device 300 will operate in the same manner if the input driver 312 and the output driver 314 are not present.
In general, in an implementation, a method for transferring data includes a data transfer command being received by a first memory architecture controller associated with a first memory architecture when a graphics processing unit (GPU) needs access to the first memory architecture. The first memory architecture controller initiates a data transfer directly from the first memory architecture to a second memory architecture associated with the GPU. Data is then transferred directly from the first memory architecture to the second memory architecture associated with the GPU using a local switch and bypassing a host processor switch. In an implementation, the data transfer command is sent by a host processor. In an implementation, the data transfer command is sent by a hardware agent of the at least one GPU. In an implementation, another data transfer command is received by the first memory architecture controller associated with the first memory architecture when the GPU needs access to the first memory architecture. The first memory architecture controller initiates a data transfer directly from the second memory architecture to the first memory architecture. Data is then transferred from the second memory architecture to the first memory architecture associated with the GPU using the local switch and bypassing the host processor switch.
In an implementation, an apparatus for transferring data includes at least one graphics processing unit (GPU), a second memory architecture associated with each GPU, at least one first memory architecture, a first memory architecture controller connected with each first memory architecture and a local switch coupled to each first memory architecture controller and the at least one GPU. The at least one first memory architecture controller receives a data transfer command when the at least one GPU needs access to a first memory architecture associated with the at least one first memory architecture controller, directly initiates a data transfer directly from the first memory architecture to the second memory architecture associated with the at least one GPU and transfers data directly from the first memory architecture to the second memory architecture associated with the at least one GPU using the local switch and bypassing a host processor switch. In an implementation, the data transfer command is sent by a host processor. In an implementation, the data transfer command is sent by a hardware agent of the at least one GPU. In an implementation, the at least one first memory architecture controller receives another data transfer command when the at least one GPU needs access to the first memory architecture associated with the at least one first memory architecture controller, initiates a data transfer directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller and transfers data directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller using the local switch and bypassing the host processor switch.
In an implementation, a system for transferring data includes a host processor including a processor and a host processor switch and at least one solid state graphics (SSG) card connected to the host processor. Each SSG card includes at least one graphics processing unit (GPU), a second memory architecture associated with each GPU, at least one first memory architecture, a first memory architecture controller connected with each first memory architecture, and a local switch coupled to each first memory architecture controller and the at least one GPU. In an implementation, the host processor switch is connected to each local switch. In an implementation, the at least one first memory architecture controller receives a data transfer command when the at least one GPU needs access to a first memory architecture associated with the at least one first memory architecture controller, directly initiates a data transfer directly from the first memory architecture to the second memory architecture associated with the at least one GPU, and transfers data directly from the first memory architecture to the second memory architecture associated with the at least one GPU using the local switch and bypassing the host processor switch. In an implementation, the data transfer command is sent by the processor. In an implementation, the data transfer command is sent by a hardware agent of the at least one GPU. In an implementation, the at least one first memory architecture controller receives another data transfer command when the at least one GPU needs access to the first memory architecture associated with the at least one first memory architecture controller, initiates a data transfer directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller, and transfers data directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller using the local switch and bypassing the host processor switch.
In an implementation, a computer readable non-transitory medium including instructions which when executed in a processing system cause the processing system to execute a method for transferring data. The method includes a data transfer command being received at a first memory architecture controller associated with a first memory architecture when a graphics processing unit (GPU) needs access to the first memory architecture. A data transfer initiated by the first memory architecture controller directly from the first memory architecture to a second memory architecture associated with the GPU. Data is then transferred directly from the first memory architecture to the second memory architecture associated with the GPU using a local switch and bypassing a host processor switch. In an implementation, the data transfer command is sent by a host processor. In an implementation, the data transfer command is sent by a hardware agent of the at least one GPU. In an implementation, another data transfer command is received by the first memory architecture controller when the at least one GPU needs access to the first memory architecture associated with the at least one first memory architecture controller. A data transfer is initiated by the first memory architecture controller to directly transfer from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller. Data is then directly transferred from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller using the local switch and bypassing the host processor switch.
In general and without limiting implementations described herein, a computer readable non-transitory medium including instructions which when executed in a processing system cause the processing system to execute a method for transferring data directly from a second memory architecture in a GPU to a first memory architecture.
It should be understood that many variations are possible based on the disclosure herein. Although features and elements are described above in particular combinations, each feature or element may be used alone without the other features and elements or in various combinations with or without other features and elements.
The methods provided may be implemented in a general purpose computer, a processor, or a processor core. Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), and/or a state machine. Such processors may be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediary data including netlists (such instructions capable of being stored on a computer readable media). The results of such processing may be maskworks that are then used in a semiconductor manufacturing process to manufacture a processor which implements aspects of the embodiments.
The methods or flow charts provided herein may be implemented in a computer program, software, or firmware incorporated in a non-transitory computer-readable storage medium for execution by a general purpose computer or a processor. Examples of non-transitory computer-readable storage mediums include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).

Claims

What is claimed is:

1. A method for transferring data, the method comprising:

receiving, at a first memory architecture controller associated with a first memory architecture, a data transfer command when a graphics processing unit (GPU) needs access to the first memory architecture;

initiating, by the first memory architecture controller, a data transfer directly from the first memory architecture to a second memory architecture associated with the GPU; and

transferring data directly from the first memory architecture to the second memory architecture associated with the GPU using a local switch and bypassing a host processor switch.

2. The method of claim 1, wherein the data transfer command is sent by a host processor.

3. The method of claim 1, wherein the data transfer command is sent by a hardware agent of the at least one GPU.

4. The method of claim 1, further comprising:

receiving, at the first memory architecture controller associated with the first memory architecture, another data transfer command when the GPU needs access to the first memory architecture;

initiating, by the first memory architecture controller, a data transfer directly from the second memory architecture to the first memory architecture; and

transferring data directly from the second memory architecture to the first memory architecture associated with the GPU using the local switch and bypassing the host processor switch.

5. The method of claim 1, wherein the first memory architecture controller is a non-volatile memory (NVM) controller and the first memory architecture is a NVM.

6. The method of claim 5, wherein the second memory architecture is a local memory.

7. An apparatus for transferring data, comprising:

at least one first memory architecture;

a first memory architecture controller connected with each first memory architecture;

at least one graphics processing unit (GPU);

a second memory architecture associated with each GPU; and

a local switch coupled to each first memory architecture controller and the at least one GPU,

wherein the at least one first memory architecture controller:

receives a data transfer command when the at least one GPU needs access to a first memory architecture associated with the at least one first memory architecture controller;

directly initiates a data transfer directly from the first memory architecture to the second memory architecture associated with the at least one GPU; and

transfers data directly from the first memory architecture to the second memory architecture associated with the at least one GPU using the local switch and bypassing a host processor switch.

8. The apparatus of claim 7, wherein the data transfer command is sent by a host processor.

9. The apparatus of claim 7, wherein the data transfer command is sent by a hardware agent of the at least one GPU.

10. The apparatus of claim 7, wherein the at least one first memory architecture controller:

receives another data transfer command when the at least one GPU needs access to the first memory architecture associated with the at least one first memory architecture controller;

initiates a data transfer directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller; and

transfers data directly from the local memory associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller using the local switch and bypassing the host processor switch.

11. The apparatus of claim 7, wherein the first memory architecture controller is a non-volatile memory (NVM) controller and the first memory architecture is a NVM.

12. The apparatus of claim 7, wherein the second memory architecture is a local memory.

13. A system for transferring data, comprising:

a host processor including a processor and a host processor switch; and

at least one solid state graphics (SSG) card connected to the host processor, wherein each SSG card includes:

at least one first memory architecture;

at least one graphics processing unit (GPU);

a second memory architecture associated with each GPU;

and

wherein the host processor switch is connected to each local switch, and

wherein the at least one first memory architecture controller:

transfers data directly from the first memory architecture to the second memory architecture associated with the at least one GPU using the local switch and bypassing the host processor switch.

14. The system of claim 13, wherein the data transfer command is sent by the host processor.

15. The system of claim 13, wherein the data transfer command is sent by a hardware agent of the at least one GPU.

16. The system of claim 13, wherein the at least one first memory architecture controller:

transfers data directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller using the local switch and bypassing the host processor switch.

17. The system of claim 13, wherein the first memory architecture controller is a non-volatile memory (NVM) controller and the first memory architecture is a NVM.

18. The system of claim 13, wherein the second memory architecture is a local memory.

19. A computer readable non-transitory medium including instructions which when executed in a processing system cause the processing system to execute a method for transferring data, the method comprising the steps of:

20. The computer readable non-transitory medium of claim 19, wherein the data transfer command is sent by a host processor.

21. The computer readable non-transitory medium of claim 19, wherein the data transfer command is sent by a hardware agent of the at least one GPU.

22. The computer readable non-transitory medium of claim 19, the method further comprising the steps of:

receiving, at the first memory architecture controller, another data transfer command when the at least one GPU needs access to the first memory architecture associated with the at least one first memory architecture controller;

initiating, by the first memory architecture controller, a data transfer directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller; and

transferring data directly from the second memory architecture associated with the at least one GPU to the first memory architecture associated with the at least one first memory architecture controller using the local switch and bypassing the host processor switch.