KR20180058332A - Memory control apparatus for optimizing gpu memory access through pre-patched scratchpad memory data and control method thereof - Google Patents

Abstract

The present invention relates to a memory control apparatus and a method thereof for pre-patching data that frequently causes compulsory miss in an L1 cache in a GPU to a scratchpad memory and reducing overhead that is increased by the compulsory miss of the cache memory. According to the memory control apparatus and the method thereof of the present invention, data can be cached to rapidly access and use data. When a large amount of data is read in advance, the bandwidth of the memory of the GPU is fully utilized to minimize the overhead due to a memory delay time. The memory control apparatus includes a GPU kernel analysis module, a pre-patch target determination module, and a pre-patch execution module.

Description

TECHNICAL FIELD [0001] The present invention relates to a memory control device for optimizing access to a GPU memory by prefetching data into a scratch pad, and a memory control method thereof.

The present invention relates to a memory control device and a memory control method thereof, and more particularly, to a memory control device for optimizing GPU memory access through data prefetching to a scratch pad and a memory control method thereof.

A graphics processing unit (GPU) has a multi-level cache memory structure such as a central processing unit (CPU).

Figure 1 is a general GPU architecture.

Referring to FIG. 1, it can be seen that the GPU includes n SIMDs. SIMD (Single Instruction Multiple Data) is a kind of parallel processor that calculates several values simultaneously with one instruction. The SIMD may also be referred to as a different name for each GPU maker. Stream Multiprocessor in NVIDIA ^® and Compute Unit in AMD ^® . NVIDIA ^® 's recent product, GTX980 ^®, has 16 SIMDs in total, 4 warp schedulers in each SIMD, 32 CUDA cores in each scheduler, 8 load / store units, and a Special Functional Unit . However, the contents of this description are not directly related to the contents described in this specification, so a detailed explanation will be omitted. Therefore, FIG. 1 shows only a configuration that is highly related to the content to be described in this specification.

The SIMD may include a register, a scratchpad memory or a shared memory, an L1 cache, and a read only memory (ROM). The n SIMDs may share an L2 cache and a global memory (DRAM). For example, the L1 cache may have a size of 48 KB per SIMD, and the L2 cache may have a size of 2 MB. Main memory outside the right dotted line is a memory connected to the CPU and connected to the SIMD.

And may be divided into an on-chip memory and an off-chip memory according to the location of the memories. The registers, L1 cache, L2 cache, read-only memory (ROM), and scratch pad memory are on-chip memory. Global memory is off-chip memory.

Meanwhile, the L1 cache may have three types of misses, such as a compulsory miss, a capacity miss, and a conflict miss, as in the case of a CPU, It is designed to request the L2 cache and global memory for the required data when it occurs. The GPU stores the data needed for the calculation in off-chip global memory. However, it is obvious that the delay time for fetching data from the global memory which is the off-chip memory is 10-100 times longer than the on-chip memory.

To overcome this, the GPU seeks to benefit from a data reuse pattern that takes into account local and temporal aspects, using multiple levels of cache (L1 cache and L2 cache). However, in an application pattern where frequent compulsory misses occur, there is still the overhead of requesting and waiting for data in the L2 cache and global memory each time.

Korean Patent Publication No. 10-2015-0092440

The memory control apparatus and method according to the present invention is intended to reduce the overhead that is increasing due to the forced loss of the cache memory.

The solutions described herein are not limited to those mentioned above, and other solutions not mentioned may be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a memory control apparatus comprising: a GPU kernel analysis module for collecting a memory address requested in a GPU kernel; A prefetch target determination module that determines data for which a forced loss (Compulsory miss) occurs in the L1 cache above a preset reference as prefetch target data; And a prefetch execution module for copying data determined as prefetch target data to a scratch pad memory.

According to an embodiment of the present invention, the prefetch target determination module can determine whether forcible loss occurs through a memory access pattern that loads data into the global memory without reusing data stored in the L1 cache.

According to an embodiment of the present invention, the prefetch target determination module may determine data in which a compulsory miss occurs twice or more in the L1 cache as prefetch target data.

According to one embodiment of the present invention, the memory control device calculates whether or not the amount of data (hereinafter, prefetch candidate data) determined as the prefetch target data exceeds the total size of the scratch pad memory And a prefetch data amount calculating module. In this case, the prefetch execution module may copy the prefetch candidate data until the amount of the prefetch candidate data exceeds the total size of the scratch pad memory, to the scratch pad memory.

According to another embodiment of the present invention, the memory control apparatus further includes a prefetch data amount calculation unit that calculates whether or not the amount of data determined as the prefetch target data (hereinafter, prefetch candidate data) Module. ≪ / RTI >

(Size of scratch pad memory available per SM) / (number of scheduled thread blocks per SM)

In this case, the prefetch execution module may copy the prefetch candidate data before the size of the formula is exceeded to the scratch pad memory.

According to an aspect of the present invention, there is provided a memory control method including: (a) collecting a memory address requested by a GPU kernel analysis module in a GPU kernel; (b) determining, as prefetch target data, data in which a prefetch target determination module generates a compulsory miss in the L1 cache above a preset reference; And (c) copying the data determined as prefetch target data to the scratch pad memory by the prefetch execution module.

The memory control method according to the present invention for solving the above-mentioned object may be a computer program recorded in a computer readable recording medium which is created to perform each step in the computer.

According to the memory control apparatus and method of the present invention, data can be quickly accessed by accessing and caching data. When a large amount of data is read in advance, the bandwidth of the memory of the GPU is fully utilized, The overhead of memory latency can be minimized

The effects described in the present specification are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

Figure 1 is a general GPU architecture.
2 is a block diagram briefly showing a configuration of a memory control device according to an embodiment of the present invention.
3 is a block diagram briefly showing a configuration of a memory control device according to another embodiment of the present invention.
4 is a flowchart of a memory control method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Wherever possible, the same or similar parts are denoted using the same reference numerals in the drawings.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms as used herein include plural forms as long as the phrases do not expressly express the opposite meaning thereto.

Means that a particular feature, region, integer, step, operation, element and / or component is specified and that other specific features, regions, integers, steps, operations, elements, components, and / It does not exclude the existence or addition of a group.

All terms including technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Predefined terms are further interpreted as having a meaning consistent with the relevant technical literature and the present disclosure, and are not to be construed as ideal or very formal meanings unless defined otherwise.

Meanwhile, the memory control device and the memory control method according to the present invention are a memory control device for optimizing GPU memory access and a memory control method thereof. Therefore, in describing the memory control apparatus and the memory control method according to the present specification, each component of the GPU can be mentioned. Each component of the GPU will be described in detail in the 'Background of the Invention' section with reference to FIG. 1, and a repetitive description thereof will be omitted.

Hereinafter, a memory control apparatus and a memory control method thereof according to the present specification will be described with reference to the drawings.

2 is a block diagram briefly showing a configuration of a memory control device according to an embodiment of the present invention.

Referring to FIG. 2, the memory control apparatus 100 according to an embodiment of the present invention includes a GPU kernel analysis module 110, a prefetch target determination module 130, and a prefetch execution module 150.

The GPU kernel analysis module 110 may collect the memory address requested in the GPU kernel. The GPU kernel analysis module 110 may be a 'Georgia tech. GPUOcelot ^' , developed and released by Intel Corporation, to collect the memory addresses requested by the GPU kernel.

The prefetch target determination module 130 may determine pre-fetch target data in which a compulsory miss occurs in the L1 cache more than a predetermined threshold.

According to an embodiment of the present invention, the prefetch target determination module 130 may determine whether a forced loss occurs through a memory access pattern that loads data into the global memory without reusing data stored in the L1 cache. That is, the prefetch target determination module 130 determines whether a pattern that does not cause cache hit occurs because data is not reused.

According to an embodiment of the present invention, the prefetch target determination module 130 may determine data in which a compulsory miss occurs twice or more in the L1 cache as prefetch target data. However, if the number of times is only one example, it is obvious that the number of times can be variously set.

The prefetch execution module 150 may copy the data determined as the prefetch target data to the scratch pad memory.

The control logic for copying the data determined as the prefetch target data into the scratch pad memory may be required to copy the data to the scratch pad memory at the code level and modify the data to be used for calculation using the copied data . The 'modification at the source code level' refers to a source-to-source transformation technique of converting the source code to a specific rule at the source code level. More specifically, the code for loading the data into the scratch pad memory is added in advance to the source code (forcing loss) that has been previously written to be loaded from the global memory, and the code for loading the global memory is preloaded It can be a task of changing data to load from scratch pad memory.

3 is a block diagram briefly showing a configuration of a memory control device according to another embodiment of the present invention.

According to another embodiment of the present invention, the memory control apparatus 100 may further include a prefetch data amount calculating module 140. [

The prefetch data amount calculation module 140 may calculate the size of the total data by summing the amount of data (hereinafter, prefetch candidate data) determined as the prefetch target data, i.e., prefetch candidate data.

According to an embodiment of the present invention, the prefetch data amount calculation module 140 may calculate whether the amount of the prefetch candidate data exceeds the total size of the scratch pad memory. In this case, the prefetch execution module 150 may copy the prefetch candidate data until the amount of the prefetch candidate data exceeds the total size of the scratch pad memory, to the scratch pad memory.

According to another embodiment of the present invention, the prefetch data amount calculation module 140 may calculate whether the amount of the prefetch candidate data exceeds the following formula.

&Quot; (1) "

(Size of scratch pad memory available per SIMD) / (number of scheduled thread blocks per SIMD)

In this case, the prefetch execution module 150 may copy the prefetch candidate data before the size of the formula is exceeded to the scratch pad memory.

The prefetch data amount calculation module 140 calculates a proper amount of data to be prefetched in the scratch pad memory. The size of the data to be prefetched is preferably determined within the overall size of the scratch pad memory. For example, the number of data requested in one instruction and the number of requested data in the next instruction are added to the instructions causing the compulsory miss, and the sum exceeds the size of the scratch pad memory It is decided within the range not to do.

In addition, the amount of data to be prefetched in the scratch pad memory is preferably determined in consideration of the number of warps allocated to the SIMD, so as not to hinder parallelism. The prefetch candidate data may continue to be traversed through subsequent instructions until the number of thread blocks that are actively allocated is limited. In the GPU, when contention is shared among resources shared by threads, only fewer threads than the number of threads specified in the actual code are limited to be scheduled, which may lead to a loss of parallelism. At this time, a typical shared resource may be a scratch pad memory and a register. This can be expressed as Equation (1). Therefore, if the data to be prefetched while traversing the instructions exceeds the value derived from Equation (1), it is preferable to prefetch the data up to that time. After that, data to be newly prefetched can be recalculated.

Hereinafter, a memory control method according to the present invention will be described. However, the memory control method according to the present invention will be described with reference to the memory control device described above. Therefore, repetition of description of each configuration of the memory control device will be omitted.

4 is a flowchart of a memory control method according to an embodiment of the present invention.

In step S100, the GPU kernel analysis module 110 may collect the memory address requested in the GPU kernel.

In step S110, the prefetch target determination module 130 may determine that data for which a forced loss (Compulsory miss) occurs in the L1 cache above a predetermined reference value as prefetch target data. At this time, the prefetch target determination module 130 can determine whether a forced loss occurs through a memory access pattern in which data stored in the L1 cache is not reused and data is loaded into the global memory. In addition, the preset reference may be two or more times.

The memory control method according to the present invention may further include step S120.

In step S120, the prefetch data amount calculation module 140 may calculate the amount of data (hereinafter, prefetch candidate data) determined as the prefetch target data.

According to one embodiment of the present disclosure, the prefetch data amount calculation module 140 can calculate whether the amount of the prefetch candidate data exceeds the total size of the scratch pad memory.

According to another embodiment of the present invention, the prefetch data amount calculation module 140 can calculate whether the amount of the prefetch candidate data exceeds the formula (1).

In step S130, the prefetch execution module 150 may copy the data determined as the prefetch target data to the scratch pad memory.

According to an embodiment of the present invention, the prefetch execution module 150 may copy the prefetch candidate data before the size of the formula is exceeded to the scratch pad memory.

According to another embodiment of the present invention, the prefetch execution module 150 may copy the prefetch candidate data before the size of Equation (1) is exceeded to the scratch pad memory.

The GPU kernel analysis module, prefetch target determination module, and prefetch execution module may be implemented as a processor, an application-specific integrated circuit (ASIC), other chipset, logic circuit, Registers, communication modems, data processing devices, and the like. Further, when the control logic is implemented in software, the GPU kernel analysis module, prefetch target determination module, and prefetch execution module may be implemented as a set of program modules. At this time, the program module is stored in the memory device and can be executed by the processor. At this time, the GPU kernel analysis module, the prefetch target determination module, and the prefetch execution module can be function modules in the source code level of the program.

On the other hand, in the case of a pattern in which the forced loss occurs repeatedly in the loop, we can expect a generally known performance gain through loop-unrolling, so we apply loop-unrolling to copy the data to the scratch pad memory, You can modify the code to read and use it. The loop-unrolling is a technique used to obtain an optimization gain at the compiler level by unlocking code that operates in a loop. In the present specification, loop-unrolling means that if the source code that calls the data to be written to the scratch pad memory is located within the loop, the loop can be looped-unrolled and the code translation at the source code level applied to the loops .

The embodiments and the accompanying drawings described in the present specification are merely illustrative of some of the technical ideas included in the present invention. Accordingly, the embodiments disclosed herein are for the purpose of describing rather than limiting the technical spirit of the present invention, and it is apparent that the scope of the technical idea of the present invention is not limited by these embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: memory control device
110: GPU kernel analysis module
130: Prefetch target determination module
150: Prefetch execution module

Claims (11)

A GPU kernel analysis module that collects the memory addresses requested in the GPU kernel;
A prefetch target determination module that determines data for which a forced loss (Compulsory miss) occurs in the L1 cache above a preset reference as prefetch target data; And
And a prefetch execution module for copying data determined as prefetch target data to a scratch pad memory. The method according to claim 1,
The prefetch target determination module includes:
And determines whether a forced loss occurs through a memory access pattern that loads data into the global memory without reuse of data stored in the L1 cache. The method according to claim 1,
Wherein the prefetch target determination module determines data for which a forced loss (Compulsory miss) occurs twice or more in the L1 cache as prefetch target data. The method according to claim 1,
Further comprising a prefetch data amount calculating module for calculating whether or not the amount of data determined as the prefetch target data (hereinafter, prefetch candidate data) exceeds the total size of the scratch pad memory,
Wherein the prefetch execution module copies the prefetch candidate data until the amount of the prefetch candidate data exceeds the total size of the scratch pad memory to the scratch pad memory. The method according to claim 1,
Further comprising a prefetch data amount calculating module for calculating whether or not the amount of data determined as the prefetch target data (hereinafter, prefetch candidate data) exceeds the following formula,
(Size of scratch pad memory available per SM) / (number of scheduled thread blocks per SM)
Wherein the prefetch execution module copies the prefetch candidate data before the size of the formula is exceeded to the scratch pad memory. (a) collecting the memory address requested by the GPU kernel analysis module in the GPU kernel;
(b) determining, as prefetch target data, data in which a prefetch target determination module generates a compulsory miss in the L1 cache above a preset reference; And
(c) copying the data determined as prefetch target data to the scratch pad memory by the prefetch execution module. The method of claim 6,
Wherein the prefetch target determination module determines whether a forced loss occurs through a memory access pattern in which data stored in the L1 cache is not reused and the data is loaded into the global memory. . The method of claim 6,
Wherein the prefetch target determination module determines data in which a compulsory miss occurs twice or more in the L1 cache as prefetch target data. The method of claim 6,
(c-1) calculating whether the amount of data (hereinafter, prefetch candidate data) determined as the prefetch target data exceeds a total size of the scratch pad memory, by the prefetch data amount calculating module Further included,
The step (d) is a step of copying the prefetch candidate data until the amount of the prefetch candidate data exceeds the total size of the scratch pad memory, to the scratch pad memory . The method of claim 6,
The step (c-1) further includes the step of calculating whether the amount of data (hereinafter, prefetch candidate data) determined as the prefetch target data by the prefetch data amount calculating module exceeds the following formula and,
(Size of scratch pad memory available per SM) / (number of scheduled thread blocks per SM)
Wherein the step (d) copies the prefetch candidate data until the prefetch execution module exceeds the size of the formula, to the scratch pad memory. A computer program recorded on a computer-readable recording medium, the computer program being written in the computer to perform the steps of the memory control method according to any one of claims 6 to 10.

Images