KR20200091206A - Monolithic 3D integration based L1 cache memory for GPUs - Google Patents

Abstract

Disclosed is a monolithic 3D integrated structure based GPU L1 cache memory capable of improving GPU performance by increasing an L1 cache capacity without deteriorating cache access time. According to the present invention, the GPU L1 cache memory device including a data cache, a texture cache, a constant cache and a shared memory comprises: a first layer in which a cell integrating a texture cache and a constant cache and a bit cell of a shared memory are disposed; a second layer placed in an upper part of the first layer and having bit cells of the data cache disposed thereon; and a monolithic 3D via vertically connecting the first layer and the second layer.

Description

Monolithic 3D integration based GPU L1 cache memory for monolithic 3D integration based L1 cache memory for GPUs

The present invention relates to a GPU L1 cache memory based on a monolithic 3D integrated structure.

A graphics processing unit (GPU) is a device that uses thousands of cores (processors) to execute a large number of threads in parallel.

GPU systems that perform graphics-only operations use General-Purpose computing on Graphics (GPGPU) using languages such as Compute Unified Device Architecture (CUDA), Open Computing Language (OpenCL), and C++ AMP (C++ Accelerated Massive Parallelism). Processing Units). GPU systems have a large number of processors and process data in a manner similar to SIMD (Single Instruction Multiple Data). Therefore, it is suitable for processing large-scale data composed of 2D and 3D.

However, many processors compete for simultaneous access to one global memory, resulting in a decrease in performance due to memory access delays.

To solve this, a technique using a shared memory has been proposed, but there is a problem in that the implementation complexity increases.

Accordingly, recently, a cache memory is provided to cache a value read from global memory as in the CPU.

Cache memory and shared memory are located on the same hardware, and the memory spaces are classified according to the execution environment. Therefore, if you use cache memory, you can see the same performance without using shared memory and avoid the implementation complexity of using shared memory. Will be able to.

In the GPU, the warp scheduler plays an important role in improving GPU performance by efficiently using GPU hardware resources.

A number of warp schedulers have been proposed to improve GPU performance, but have not yet fully utilized GPU hardware resources.

This is because the efficiency of the warp scheduler and GPU performance depend heavily on the L1 cache memory capacity rather than the warp scheduler algorithm itself.

In particular, the small capacity of L1 cache memory cannot accommodate large sets of operations that are common in general purpose computing on GPUs (GPGPUs), which not only leads to cache thrashing, but also saturates memory system resources.

Patent Registration 10-1639943

In order to solve the above-mentioned problems of the prior art, it is intended to propose a monolithic 3D integrated structure-based GPU L1 cache memory capable of improving GPU performance by increasing L1 cache capacity without deteriorating cache access time.

In order to achieve the above object, according to an embodiment of the present invention, a GPU L1 cache memory device including a data cache, a texture cache, a constant cache and a shared memory, the texture cache and a constant cache integrated cell and A first layer in which bit cells of the shared memory are disposed; A second layer positioned above the first layer and in which bit cells of a data cache are disposed; And a monolithic 3D via that vertically connects the first layer and the second layer.

Further comprising a word line driver connected to the first word line wire of the first layer, the monolithic 3D via may vertically connect the word line driver and the second word line wire of the second layer.

The word line driver is connected to an output of a row decoder, and a cache address converted by a cache address converter may be input to the row decoder.

The cache address converter may convert an existing cache address into a new cache address according to the 3D layer arrangement and output the code using code for identifying one of a data cache, a texture cache, a constant cache, and a shared memory.

A layer decoder that identifies the layer ID of the requested bit cell; A column decoder for selecting requested data using data from a bit cell transmitted through a bit line; And a sense amplifier for amplifying the selected data signal.

According to the present invention, there is an advantage of separating the data cache, the texture cache/constant cache, and the shared memory into different layers, and connecting them with monolithic 3D vias to further reduce access time.

1 is a diagram showing an SM structure of a general GPU.
2 is a diagram illustrating a GPU L1 cache memory structure according to an exemplary embodiment of the present invention.
3 is a diagram showing the configuration of a cache address translator according to an embodiment of the present invention.
4 is a diagram showing results before and after the conversion of the cache address by the cache address converter according to the present embodiment.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

1 is a diagram showing an SM structure of a general GPU.

GPU is composed of a set of SM (Streaming Multiprocessor), each SM includes a plurality of SP (Scalar Processor) and a plurality of threads.

As shown in FIG. 1, the SM may include 8 SPs and 32 threads.

Here, 32 threads are called a warp unit, and when a memory reference is made to 16 threads, it is called a half warp.

GPU operates in 1SM unit (32 threads) (refer to memory).

The SP consists of 4 threads and runs.

Here, a thread is a minimum instruction processing unit.

Such an SM includes special purpose caches including a data cache L1D, a texture cache TEX, a constant cache CONST, and a shared memory SHD.

The reason for separating caches for a specific purpose is for fast access time.

Memory requests from other SPs in the SM are queued to maintain the order of memory requests for a specific-purpose cache.

According to one preferred embodiment of the present invention, in order to ensure fast access, that is, to avoid interference in the data cache, the texture cache and the constant cache are integrated in the lowest layer, and the shared memory is also disposed in the lowest layer.

In addition, the data cache is based on a monolithic 3D integrated structure and is placed on the upper layer of the SRAM.

2 is a diagram illustrating a GPU L1 cache memory structure according to an exemplary embodiment of the present invention.

As illustrated in FIG. 2, the GPU L1 cache memory may include a first layer (Layer1, 200) which is a lower layer and a second layer (Layer2, 202) which is a higher layer.

In the first layer 200, a TEC bit cell in which a texture cache and a constant cache are integrated and an SHD bit cell in shared memory are disposed.

In addition, an L1D bit cell that is a data cache is disposed in the second layer 202.

Although the second layer 202 is shown in FIG. 2 as one example, this is only an example, and in the case of the L1 data cache, since the hit ratio is lower than that of other caches, the second layer 202 may be configured in plurality to increase capacity. have.

In this case, one or more second layers 202 may be vertically disposed on the first layer 200, and each of the plurality of second layers 202 may also be vertically disposed.

According to this embodiment, the first layer 200 and the second layer 202 are connected through a monolithic 3D via 204.

Here, the monolithic 3D integrated structure is defined as a 3D sequential integrated structure.

The monolithic 3D integrated device formation technology is a process technology for forming a new substrate layer through bonding on a thin wafer after epi growth, recrystallization or depositing an insulator layer after forming the device on one wafer and then sequentially forming the device on it. .

The monolithic 3D integrated device formation technology can change the Via specification by forming the device of each layer in a very thin channel (50~500nm) [(Via diameter (10~50nm), Via pitch (100~300nm)] TSV) It is a new alternative manufacturing technology that can solve problems according to the thickness of the wafer, which is pointed out as a limitation of (Through Si Via).

In such a monolithic 3D integrated structure, vias connecting devices of each layer are defined as inter-layer vias or inter-tier vias, and in this specification, the upper layer L1D It is defined as a monolithic 3D inter-via (MIV) in that a bit cell and a lower layer TEX bit cell and an SHD bit cell form a monolithic 3D integrated structure.

In addition, the GPU L1 cache memory according to this embodiment includes a row decoder (Row Decoder 210), a word line driver (Wordline Driver 212), a layer decoder 214, a sense amplifier and a column decoder 216, and a data input/output unit ( 218).

The vertical word line concept is used to describe the operation of the M3D based GPU L1 cache memory (SRAM) according to this embodiment.

As described above, in the M3D based SRAM, MIV (M3D Via, 204) is used to vertically connect the upper and lower layers 200 and 202.

Here, the word line of the first layer 200 becomes a global word line.

Bank routing through the 2D H-tree network is performed only in the first layer 200 that does not require a vertical bank routing path.

Since all caches are implemented in M3D based SRAM, correct access to cache addressing is required to access M3D based SRAM correctly.

3 is a diagram showing the configuration of a cache address translator according to an embodiment of the present invention, and FIG. 4 is a diagram showing the results before and after a cache address translation by the cache address translator according to this embodiment.

Referring to FIG. 3, the cache address converter according to the present embodiment may include a mux (300), a format unit (Format, 302), and a full adder (Full Adder) 304.

That is, the cache address translator converts the existing address into a new cache address according to the 3D layer arrangement through the monolithic 3D via.

The MUX 300 outputs codes for one of SHD, TEC, and L1D caches.

Here, the SHD, TEC, and L1D codes may be 0000, 0100, and 1000, respectively.

Here, 0, which is the most significant bit of the SHD and TEC codes, is a bit corresponding to the first layer 200 that is the lower layer, and 1, which is the most significant bit of the L1D code, is the bit corresponding to the second layer 202 that is the upper layer.

For example, when a cache address of 010 is input to the L1D cache, the formatter 300 adds 0 to the most significant bit and converts it to 0010 to output.

Since the above cache address is an address for the L1D cache, the MUX 300 outputs an L1D code of 1000, and the full adder 304 finally outputs 1010.

Through the above configuration, the existing cache address is modified according to the M3D based GPU L1 cache memory structure according to the present embodiment.

The cache address converter according to the present embodiment converts an existing cache address into a new cache address using unique codes (0000, 0100,1000) for each of SHD, TEC, and L1D.

The converted cache address is input to the row decoder 210.

Wordlines and bitlines of M3D based SRAM are selected based on the new cache address.

The wordline driver 212 identifies which global wordline is selected by the new cache address.

The layer decoder 214 identifies the layer ID of the requested bit cell.

As described above, the most significant bit of the new cache address is used for the selection of one of the first layer 200 and the second layer 202.

If the required bit cells are in the first layer 200, which is the lower layer, they are accessed through the first wordline wire 220.

Otherwise, the MIV 204 accesses the upper layer, the second layer 202, and then accesses a predetermined bit cell through the second word line wire 222 of the second layer 202.

The column amplifier of the sense amplifier and column decoder 216 selects an appropriate word (required data) using data from the bit cell transmitted through the bit line, and the selected data signal is amplified by the sense amplifier.

The data input/output unit 218 outputs an amplified signal.

The above-described embodiments of the present invention have been disclosed for purposes of illustration, and those skilled in the art having various knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention. It should be regarded as belonging to the following claims.

Claims (5)

A GPU L1 cache memory device including data cache, texture cache, constant cache and shared memory,
A first layer in which a cell in which a texture cache and a constant cache are integrated and a bit cell in shared memory are disposed;
A second layer positioned above the first layer and in which bit cells of a data cache are disposed; And
A GPU L1 cache memory device including a monolithic 3D via that vertically connects the first layer and the second layer. According to claim 1,
Further comprising a word line driver connected to the first word line wire of the first layer,
The monolithic 3D via is a GPU L1 cache memory device vertically connecting the wordline driver and the second wordline wire of the second layer. According to claim 2,
The word line driver is connected to the output of the row decoder,
A GPU L1 cache memory device in which a cache address converted by a cache address converter is input to the row decoder. According to claim 3,
The cache address converter uses a code to identify one of a data cache, a texture cache, a constant cache, and a shared memory, and converts an existing cache address into a new cache address according to the 3D layer arrangement and outputs the GPU L1 cache memory device . According to claim 3,
A layer decoder that identifies the layer ID of the requested bit cell;
A column decoder for selecting requested data using data from a bit cell transmitted through a bit line; And
GPU L1 cache memory device further comprising a sense amplifier for amplifying the selected data signal.

Images