Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
  • G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
  • G06F9/3877—Concurrent instruction execution, e.g. pipeline or look ahead using a slave processor, e.g. coprocessor
  • G06F9/3879—Concurrent instruction execution, e.g. pipeline or look ahead using a slave processor, e.g. coprocessor for non-native instruction execution, e.g. executing a command; for Java instruction set
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
  • G06F9/54—Interprogram communication
  • G06F9/544—Buffers; Shared memory; Pipes

Definitions

• the present invention is generally directed to computing devices (e.g., computers, embedded devices, hand-held devices, and the like). More particularly, the present invention is directed to memory used by processing units of such computing devices.
• computing devices e.g., computers, embedded devices, hand-held devices, and the like. More particularly, the present invention is directed to memory used by processing units of such computing devices.
• a computing device typically includes one or more processing units, such as a central-processing unit (CPU) and a graphics-processing unit (GPU).
• the CPU coordinates the activities of the computing device by following a precise set of instructions.
• the GPU assists the CPU by performing data-parallel computing tasks, such as graphics-processing tasks and/or physics simulations which may be required by an end-user application (e.g., a video-game application).
• the GPU and CPU may be part of separate devices and/or packages or may be included in the same device and/or package. Further, each processing unit may be included in another larger device. For example, GPUs are frequently integrated into routing or bridge devices such as, for example, Northbridge devices.
• API appl i cat ion-programmi ng interface
• An API allows the end-user application to output graphics data and commands in a standardized format, rather than in a format that is dependent on the GPU.
• Several types of APIs are commercially available—including DirectX® developed by Microsoft Corporation of Redmond, Washington; OpenGL® and OpenCL maintained by Khronos Group.
• the API communicates with a driver.
• the driver translates standard code received from the API into a native format of instructions understood by the GPU.
• the driver is typically written by the manufacturer of the GPU.
• the GPU then executes the instructions from the driver. ? -
• the CPU and GPU are each typically coupled to an external memory.
• the external memory may include instructions to be executed and/or data to be used by the CPU and/or GPU.
• the external memory may be, for example, a dynamic random-access memory (DRAM).
• DRAM dynamic random-access memory
• the external memory can be configured to be quite large, thereby providing ample storage capacity to each processing unit to which it's coupled. Unfortunately, accessing the external memory may take several hundred clock cycles. Accordingly, an external memory may not provide memory sufficient bandwidth or fast memory access for high-end GPUs.
• the internal memory may be, for example, an embedded or stacked DRAM.
• an internal memory provides higher bandwidth, faster memory access, and consumes less power.
• the capacity of the internal memory cannot easily be scaled to meet the storage demands of high-end GPUs.
• a high-end GPU may require more memory than can be included in an internal memory of the GPU.
• Embodiments of the present invention meets the above-described needs by providing an internal, processing-unit memory for general-purpose use and applications thereof.
• the internal, processing-unit memory of embodiments of the present invention provides high bandwidth because it is embedded within a processing unit. It also provides sufficient storage capacity because a plurality of processing-unit memories may be combined into a sufficiently large memory pool.
• an embodiment of the present invention provides a GPU.
• the GPU includes a first internal memory, an execution unit coupled to the first internal memory, and an interface configured to couple the first internal memory to a second internal memory of another processing unit.
• the GPU is embodied in software.
• the GPU is included in a system.
• the system may comprise, for example, a supercomputer, a desktop computer, a laptop computer, a video-game console, an embedded device, a handheld device (e.g., a mobile telephone, smart phone, MP3 player, a camera, a GPS device, or the like), or another system that includes or is configured to include a GPU.
• FIGS. 1A and IB illustrate example systems that include internal, processing- unit memories for general-purpose use in accordance with an embodiment of the present invention.
• FIG. 2 illustrates details of an example GPU having an internal memory for general-purpose use in accordance with an embodiment of the present invention.
• FIG. 3 illustrates an example stacked memory that may be included in a processing element in accordance with an embodiment of the present invention.
• FIG. 4 illustrates an example method implemented by the GPU o FIG. 2 in accordance with an embodiment of the present invention.
• the present invention provides an internal, GPU memory for general-purpose use and applications thereof.
• references to "one embodiment,” “an embodiment,” “an example embodiment,” etc. indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
• a GPU includes an internal memory (e.g., an embedded or stacked DRAM) that is configured to be used by one or more other processing units.
• the GPU includes an interface and implements a protocol, allowing the one or more other GPUs access to its internal memory.
• the interface may provide each other GPUs dedicated access to the internal memory or may provide the other GPUs shared access to the internal memory. Access to the internal memory of the GPU may be controlled by the GPU itself or each other GPUs.
• the interface and protocol allows the internal memory to be combined with external memories, forming a larger memory pool accessible to the GPU.
• the external memories may be included in other GPUs.
• a computing device includes a plurality of GPUs, wherein each GPU includes an internal memory that is configured to be shared with the other GPUs.
• the internal memory of each GPU is combined into a unified memory pool. The size of the memory pool scales with the number of participating GPUs. Any participating GPU may use the memory pool for its storage needs.
• FIGS. 1A and IB illustrate an example computing system 100 having a plurality of GPUs, each including an internal memory configured for general-purpose use in accordance with embodiments of the present invention.
• the internal memories provide each GPU higher bandwidth access to data.
• the internal memories of each GPU can be combined into a larger memory pool accessible by each GPU, thereby providing sufficient storage capacity to each GPU.
• each GPU is given dedicated access to the internal memory of another GPU.
• each GPU has shared access to the internal memories of the other GPU via a shared interface.
• computing system 100 may comprise a supercomputer, a desktop computer, a laptop computer, a video-game console, an embedded device, a handheld device (e.g., a mobile telephone, smart phone, MP3 player, a camera, a GPS device, or the like), or some other device that includes or is configured to include a CPU and/or GPU.
• computing device 100 includes a CPU 102, a first GPU 1 1 OA, and a second GPU HOB.
• CPU 102 executes instructions to control the functionality of computing device 100.
• GPUs 110 assist CPU 102 by performing data-parallel processing tasks (such as, for example, graphics-processing tasks and/or general-compute tasks). Based on their design, GPUs 1 10 can typically perform data- parallel processing tasks faster than CPU 102 could perform them in software.
• First GPU 11 OA and second GPU HOB each include their own internal memory and execution unit.
• first GPU 106A includes an internal memory 106 A and an execution unit 142 A; and second GPU 106B includes an internal memory 106B and an execution unit 142B.
• CPU 102 includes a cache memory 130 and an execution unit 132.
• Internal memories 106 (and optionally cache memory 130) are available to GPUs 110 to provide faster access and higher bandwidth to certain data than would be possible if the data were externally stored (e.g., if the data were stored in a system memory 104).
• the internal memories 106 may comprise, for example, embedded or stacked DRAM.
• Internal memories 106A, 106B may be combined into a larger memory pool to provide substantial storage capacity (e.g., more than 4 GB), while also providing fast, high bandwidth memory access.
• conventional external memories may provide sufficient storage capacity (e.g., more than 4 GB)
• conventional external memories provide insufficient bandwidth for certain high-end uses.
• conventional embedded memories may provide sufficient bandwidth for these high-end uses, conventional embedded memories provide insufficient storage capacity (e.g., fewer than 4 GB) for these high-end uses.
• embodiments of the present invention not only provide sufficient storage capacity (e.g., more than 4 GB), but also provide high bandwidth by providing GPUs that include internal memories which are available to other GPUs for general-purpose use.
• a frame buffer (i.e., a buffer that stores a complete frame of data to be displayed on a display device) of a high-end GPU may require high bandwidth access to a substantially large memory (e.g., more than 4 gigabytes (GB)).
• first GPU 11 OA may use internal memories 106A, B and optionally cache memory 130 of CPU 102 to define the frame buffer of first GPU 11 OA.
• second GPU HOB may also use internal memories 106A, B and optionally cache memory 130 of CPU 102 to define the frame buffer of second GPU HOB.
• the frame buffer defined in accordance with embodiments of the present invention provides high bandwidth access to a substantially large memory (e.g., more than 4 GB).
• each GPU 1 10 is given dedicated access to internal memory 106 of another processing unit, as alluded to above.
• a first interface 101 provides first GPU 1 10A dedicated access to internal memory 106B of second GPU HOB and provides second GPU HOB dedicated access to internal memory 106A of first GPU 1 1 OA.
• Data may be written to or retrieved from either internal memory 106A or internal memory 106B based on an address range of the data.
• internal memory 106A may be assigned a first address range (e.g., less than a first predetermined address A and greater than or equal to a second predetermined address B), and internal memory 106B may be assigned a second address range (e.g., all addresses not within the first address range).
• first GPU 1 1 OA and second GPU HOB can each have access to internal memory 106A of first GPU 1 10A and internal memory 106B of second GPU H OB.
• first interface 101 comprises a display controller interface.
• the display controller interface provides a display device 140 access to the frame buffer of a GPU.
• first interface 101 can be provided on a standard pin that is already included in conventional GPU designs.
• a second interface 103 provides CPU 102 dedicated access to internal memory 106B of second GPU H OB and provides second GPU HOB dedicated access to cache memory 130 of CPU 102. In this way, second GPU HOB and CPU 102 can each have access to internal memory 106B of second GPU H OB and cache memory 130 of CPU 102.
• a third interface 105 provides first GPU 1 1 OA dedicated access to cache memory 130 of CPU 102 and provides CPU 102 dedicated access to internal memory 106A of first GPU 1 1 OA. In this way, first GPU 1 1 OA and CPU 102 can each have access to internal memory 106A of first GPU 1 10A and cache memory 130 of CPU 102.
• each processing unit has shared access to the internal memories of the other processing units via a shared interface 164.
• Shared interface 164 provides each processing unit (e.g., first GPU 1 1 OA, second GPU H OB, and CPU 102) high bandwidth access to the internal memory of the other processing units.
• Data may be written to or retrieved from internal memory 106 A, internal memory 106B, or cache memory 130 based on an address range of the data. For example, internal memory 106 A may be assigned a first address range; internal memory 106B may be assigned a second address range; and cache memory 130 may be assigned a third address range.
• first GPU 11 OA, second GPU HOB, and CPU 102 can each have access to internal memory 106 A of first GPU 1 1 OA, internal memory 106B of second GPU HOB, and cache memory 130 of CPU 102.
• computing device 100 also includes a system memory 104, a secondary memory 120, an input-output (I/O) interface 1 16, and/or display device 140.
• System memory 104 stores information that is frequently accessed by programs running on CPU 102.
• System memory 104 typically comprises volatile memory, meaning that data stored in system memory 104 are lost when power to computing device 100 is turned off.
• Secondary memory 120 stores data and/or applications used by computing device 100.
• Secondary memory 120 typically has much larger storage capacity compared to system memory 104 and typically comprises non-volatile (persistent) memory, meaning that data stored in secondary memory 120 persists even when power to computing device 100 is turned off.
• I/O interface 116 allows computing device 100 to be coupled an external device 116 (such as, an external display device, an external storage device (e.g., video-game cartridge, CD, DVD, flash drive, or the like), a network card, or some other type of external device).
• Display device 140 displays content of computing device 100.
• Display device may comprise a cathode ray tube, a liquid-crystal display (LCD), a plasma screen, or some other type of display device whether now known or later developed.
• CPUs 1 10 and CPU 102 communicate with each other and system memory
• Bus 1 14 may be any type of bus used in computing devices, including a peripheral component interface (PCI) bus, an accelerated graphics port (AGP) bus, a PCI Express (PCIE) bus, or another type of bus whether presently available or developed in the future.
• PCI peripheral component interface
• AGP accelerated graphics port
• PCIE PCI Express
• computing device 100 may include a video processing unit
• computing device 100 in lieu of or in addition to GPU 110.
• computing device 100 includes GPU 1 1 OA, CPU 102; and in lieu of GPU, H OB illustrated in FIGS. 1A and IB, computing device 100 includes a VPU.
• CPU 102 can perform general processing functions
• GPU 11 OA can perform graphics- processing functions
• the VPU can perform video-processing functions.
• FIG. 2 illustrates example details of GPU 1 10 having an internal memory 106.
• internal memory 106 can be used by another GPU, or a CPU, to increase overall system performance by combining the graphics processing power based on an augmented memory footprint size.
• GPU 110 includes execution unit 142 and internal memory 106.
• execution unit 142 includes input logic 202, a shader core 204, and output logic 206.
• Internal memory 106 includes a memory controller 210 and memory cells 212. Memory controller 210 controls access to memory cells 212. Memory cells 212 store data.
• internal memory 106 comprises an embedded, dynamic random access memory (DRAM).
• DRAM dynamic random access memory
• An embedded DRAM is a memory encapsulated in a common package with a processing unit.
• internal memory 106 comprises a stacked DRAM, as illustrated in FIG. 3.
• a stacked memory includes a plurality of memory elements stacked on top of each other in a three-dimensional structure.
• Internal memory 106 is coupled to execution unit 142 via both input logic 202 and output logic 206.
• input logic 202 can retrieve data from internal memory 106
• output logic 206 can send data to internal memory 106 to be stored in memory cells 212.
• Internal memory 106 may also be coupled to the internal memory of another
• first interface 250 provides dedicated access between internal memory 106 of GPU 1 10 and an internal memory of another GPU, as illustrated by interface 101 of FIG. 1A.
• first interface 250 is a provided on a standard pin of a conventional GPU.
• first interface 250 may comprise a display-controller interface, which provides a display device access to a local frame buffer included in internal memory 106.
• first interface 250 provides shared access between internal memory 106 of GPU 1 10 and internal memories of other processing units, as illustrated by interface 164 of FIG. IB.
• Internal memory 106 may also be coupled to cache memory 130 of CPU 102 via a second interface 260. Accordingly, the combination of internal memory 106 and cache memory 130 can increase the memory pool available to GPU 1 10.
• second interface 260 provides a dedicated connection between internal memory 106 of GPU 1 10 and cache memory 130 of CPU 102, such as connection 103 or connection 105 of FIG. 1 A.
• second interface 260 provides a connection that is shared by only GPU 110 and CPU 102, such as connection 164 of FIG. IB.
• second interface couples GPU 110 to CPU 102 on a common bus, such as bus 1 14 of FIGS. 1A and IB.
• FIG. 4 illustrates an example method 400 implemented by GPU 1 10 in accordance with an embodiment of the present invention. Method 400 is described below with reference to FIGS. 3 and 4.
• Method 400 begins at a step 402 in which instruction are received.
• input logic 202 receives instructions to be executed by GPU 1 10.
• the instructions may comprise, for example, a graphics-processing task or a data-parallel processing task provided by an end-user application running on CPU 102 of system 100.
• a location of data associated with an instruction is identified.
• the data may be included with a received instruction. Such data is commonly referred to as immediate data.
• the instruction provides the location of the data.
• the instruction may include an address within which the data is stored.
• the instruction includes information from which input logic 202 computes the address within which the data is stored.
• the data may be stored in either internal memory 106, an internal memory of another GPU to which internal memory 106 is coupled, or cache memory 130 of CPU 102.
• a step 406 the data is retrieved. If the data is immediate data, input logic
• input logic 202 simply extracts the immediate data from the instruction. If the data is stored in internal memory 106 or a memory to which internal memory 106 is coupled, input logic 202 sends a request to memory controller 210 for access to the data. If, on the one hand, the data is stored in memory cells 212, the data is retrieved and provided to input logic 202. If, on the other hand, the data is stored in another memory coupled to internal memory 106, the request from input logic 202 is forwarded to the other memory via interface 250 or interface 260. The data is then retrieved from the other memory and provided to input logic 202.
• a step 408 the instruction is executed.
• Shader core 204 executes the instruction based on the data obtained by input logic 202 in step 406.
• Output logic 206 determines whether further processing is required on these results, as indicated in decision step 412. Results provides to output logic 206 may have a flag or some other indicia to indicate whether additional processing is necessary. If in decision step 412 output logic 206 determines that further processing is necessary, then output logic 206 forwards the results back to shader core 204 and steps 408 and 410 of method 400 are repeated. If, on the other hand, output logic 206 determines, in decision step 412, that no further processing is necessary, then output logic 206 provides the results to internal memory 106, as indicated in step 414.
• the results may then be written to internal memory 106 or to a memory coupled to internal memory 106, depending on the address to which the results are to be written. If the results are to be written to internal memory 106, memory controller 210 provides access to the appropriate address in memory cells 212 and the results are stored therein. If, on the other hand, the results are to be written to a memory coupled to internal memory 106, then memory controller 210 forwards the results to the other memory via interface 250 or interface 260 and the results are stored in memory cells of the other memory.
• GPUs may also be embodied in software disposed, for example, in a computer-readable medium configured to store the software (e.g., a computer-readable program code).
• the computer-readable program code enables embodiments of the present invention, including the following embodiments: (i) the functions of the systems and techniques disclosed herein (such as, providing tasks to GPU 1 10, scheduling tasks in GPU 1 10, executing tasks in GPU 1 10, or the like); (ii) the fabrication of the systems and techniques disclosed herein (such as, the fabrication of GPU 1 10); or (iii) a combination of the functions and fabrication of the systems and techniques disclosed herein.
• the computer-readable program code can be disposed in any known computer-readable medium including semiconductor, magnetic disk, or optical disk (such as CD-ROM, DVD-ROM). As such, the computer-readable program code can be transmitted over communication networks including the Internet and internets. It is understood that the functions accomplished and/or structure provided by the systems and techniques described above can be represented in a core (such as a shader core) that is embodied in computer-readable program code and may be transformed to hardware as part of the production of integrated circuits.
• a core such as a shader core

Landscapes

• Engineering & Computer Science (AREA)
• Software Systems (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Memory System Of A Hierarchy Structure (AREA)
• Advance Control (AREA)
• Dram (AREA)
• Image Input (AREA)
• Image Processing (AREA)
• Multi Processors (AREA)

Priority Applications (5)

Applications Claiming Priority (4)

Publications (1)

Family

ID=43624191

Family Applications (1)

Country Status (7)

Cited By (1)

Families Citing this family (14)

Citations (5)

Family Cites Families (24)

• 2009
  • 2009-11-11 US US12/616,636 patent/US8803897B2/en active Active
• 2010
  • 2010-09-03 IN IN2568DEN2012 patent/IN2012DN02568A/en unknown
  • 2010-09-03 WO PCT/US2010/047784 patent/WO2011028984A1/en not_active Ceased
  • 2010-09-03 CN CN201080049244.9A patent/CN102597951B/zh active Active
  • 2010-09-03 KR KR1020127008022A patent/KR20120059590A/ko not_active Ceased
  • 2010-09-03 EP EP10754627.7A patent/EP2473915B1/en active Active
  • 2010-09-03 JP JP2012528080A patent/JP2013504130A/ja active Pending

Patent Citations (5)

Also Published As

Similar Documents

Legal Events