KR102136792B1 - Method and Apparatus for Memory Bottleneck due to Introduction of High Bandwidth Memory - Google Patents

Abstract

A method and apparatus for solving a memory bottleneck problem due to the introduction of HBM are presented. The method for solving a memory bottleneck problem due to the introduction of HBM proposed by the present invention is performed in the first cache (L1) and the second cache (L2) to analyze the change in the main memory bottleneck point using HBM as the main memory of the GPU. The step of indicating the factor causing the RF (reservation fail), the step of determining whether the cache is accessible based on whether there is a replaceable cache line, and when a miss occurs in the cache, the miss of the same data was previously caused by MSHR. And checking if there is any, and analyzing the speed of processing misses in the memory of the lower layer, and indicating a memory bottleneck due to insufficient bandwidth as an indicator.

Description

Method and Apparatus for Memory Bottleneck Due to Introduction of HBM {Method and Apparatus for　Memory Bottleneck due to Introduction of High Bandwidth Memory}

The present invention relates to a method and apparatus for solving a memory bottleneck problem due to the introduction of HBM.

As the speed of the processor has increased dramatically for several decades, a memory bottleneck problem due to a difference in speed between the processor and the memory has been intensified. To solve this problem, the GPU (Graphics Processing Unit) also chose a deep memory hierarchy structure in which several memory layers are interposed between the processor and the main memory. In the deep memory layer, a memory bottleneck problem could be solved to some extent by placing a cache having a higher data bandwidth as it gets closer to the processor. However, in a data-intensive application with high data demand, the memory bottleneck problem remains because the main memory has insufficient bandwidth. For this, high bandwidth memory (HBM) is considered as a new system solution.

The technical problem to be achieved by the present invention is that when the HBM having a high bandwidth (bandwidth) is used as the main memory (main memory) of the GPU, it shows that there is a change in the main memory bottleneck (memory bottleneck) point, an effective indicator for this To provide.

In one aspect, the method for solving a memory bottleneck problem due to the introduction of HBM proposed by the present invention is a first cache (L1) and a second to analyze the change in the main memory bottleneck point using HBM as the main memory of the GPU In the cache (L2), the step of indicating the factor causing the RF (reservation fail), the step of determining whether the cache is accessible based on whether there is a replaceable cache line, and if a miss occurs in the cache, the same as before according to the MSHR And checking if there is a miss for the data, and merging, and analyzing the speed of processing the miss in the memory of the lower layer, and indicating the memory bottleneck due to insufficient bandwidth as an indicator.

If a miss occurs in the cache, the step of checking whether there is a miss for the same data before and merging according to the MSHR is a cause of RF when the mergeable limit is exceeded or the entry of the MSHR is insufficient. When miss) occurs, look at the MSHR, check if there are any misses for the same data before, and merge them.

Analyzing the speed of processing a miss in the memory of the lower layer, and indicating the memory bottleneck due to insufficient bandwidth as an indicator, a queue to send a data access request due to the back pressure caused by the stall of the lower layer memory is not full. If not, it indicates whether smooth access is being made between the first cache L1 and the second cache L2 and between the cache L2 and the main memory.

In another aspect, the apparatus for solving a memory bottleneck problem due to the introduction of the HBM proposed by the present invention uses the HBM as the main memory of the GPU to analyze the change in the main memory bottleneck point, the first cache (L1) and In the second cache (L2) includes an analysis unit indicating an element that causes a failure (reservation fail), the analysis unit determines whether the cache is accessible according to whether or not a replaceable cache line, and a miss occurs in the cache In the case of MSHR, it checks if there is a miss for the same data before merging, analyzes the speed of processing the miss in the lower layer memory, and shows the memory bottleneck due to insufficient bandwidth as an indicator.

According to embodiments of the present invention, when HBM having a high bandwidth is used as the main memory of the GPU, it can be solved through an index that proposes a problem of a main memory bottleneck point.

1 is a flowchart illustrating a method of solving a memory bottleneck problem due to the introduction of HBM according to an embodiment of the present invention.
2 is a view for explaining the cause of RF in the L1 and L2 cache according to an embodiment of the present invention.
FIG. 3 is a diagram showing RF_④ ratios of L1 and L2 caches in an HBM environment compared to a baseline according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a method of solving a memory bottleneck problem due to the introduction of HBM according to an embodiment of the present invention.

The proposed method of solving the memory bottleneck problem due to the introduction of HBM is analyzed in the first cache (L1) and the second cache (L2) to analyze the change in the main memory bottleneck point using the HBM as the main memory of the GPU through the analysis unit. Step (110) indicating the factor causing the RF (reservation fail), determining whether the cache is accessible based on whether the cache line is replaceable (120), if a miss occurs in the cache (miss) occurs in accordance with the MSHR before Checking whether there is a miss for the same data, and then merging (130) and analyzing the speed of processing the miss in the memory of the lower layer, and indicating a bandwith, that is, a memory bottleneck due to insufficient bandwidth as an indicator It includes.

In step 110, in order to analyze the change in the main memory bottleneck point using HBM as the main memory of the GPU, factors that cause retention failure (RF) in the first cache L1 and the second cache L2 Indicates.

In step 120, it is determined whether the cache is accessible according to whether the cache line is replaceable.

In step 130, if a miss occurs in the cache, the MSHR checks whether there is a miss for the same data before merging. If the mergeable limit is exceeded or the entry of the MSHR is insufficient, it is the cause of the RF. When a miss occurs in the cache, the MSHR is checked, and if there is a miss for the same data before, the merge is performed.

In step 140, the speed of processing misses in the memory of the lower layer is analyzed, and a memory bottleneck due to insufficient bandwidth is indicated as an indicator. When the queue to send a data access request is not full due to the back pressure caused by the stall of the lower layer memory, seamless access between the cache 1 and the cache 2, and between the cache L2 and the main memory Indicates whether this is done.

2 is a view for explaining the cause of the RF in the first cache (L1) and the second cache (L2) according to an embodiment of the present invention.

In the present invention, when using HBM having a high bandwidth (bandwidth) as the main memory (main memory) of the GPU, it is intended to show that there is a change in the main memory bottleneck (memory bottleneck) point. We suggest effective indicators for this, and experimentally observe whether data-intensive and non-intensive applications exhibit different characteristics. When using GDDR5 and HBM as the main memory in the GTX titanX environment, we compare each. In the simulation environment of gpgpu-sim3.2.0, the experiment was performed using a rodinia3.0 benchmark.

RF (reservation fail) of gpgpu-sim was used as an index for observing the memory bottleneck.

2 shows four factors that cause RF in the L1 and L2 caches. RF_① is a case where cache access is not possible because there is no replaceable cacheline. RF_② and RF_③ are caused by MSHR (miss status holding register). When a miss occurs in the cache, the MSHR is checked, and if there is a miss for the same data before merging, the merge is performed. If the limit for merging is exceeded or the entry of the MSHR is insufficient, it causes RF.

RF_④ is a case in which a queue to send a data access request is not filled due to back pressure caused by a stall in the lower layer memory. It can be said that the values of RF_④ of the L1 and L2 caches indicate whether smooth access is being made between the L1 cache and the L2 cache, and between the L2 cache and the main memory, respectively. That is, a large value of RF_④ means that the processing speed of the miss in the memory of the lower layer is slow, and it can be used as an index to know the memory bottleneck due to insufficient bandwidth.

FIG. 3 is a diagram showing RF_④ ratios of L1 and L2 caches in an HBM environment compared to a baseline according to an embodiment of the present invention.

Figure 3 shows the ratio of RF_④ generated in the HBM environment compared to the baseline GDDR5. The seven benchmarks tend to increase the RF_④ ratio of the L1 cache and decrease the RF_④ ratio of the L2 cache in the HBM environment compared to the baseline. The increase of the L1 cache RF_④ shows a situation where data is not smoothly accessed from the L1 cache to the L2 cache. And it can be seen from the decrease in RF_④ of the L2 cache that data access from the L2 cache to the main memory is smooth. Through this, it can be seen that the position of the memory bottleneck is moved between the L2 cache and the L2 cache between the L2 cache and the main memory. Benchmarks in gray shaded in Table 1 have a high miss rate in the L2 cache, which gives access to main memory. These are applications that have a large amount of required data as well as poor locality of access patterns. In the case of these benchmarks, it can be seen that RF_④ in the L1 cache increases a lot, or RF_④ in the L2 cache decreases more than other benchmarks.

The device described above may be implemented with hardware components, software components, and/or combinations of hardware components and software components. For example, the devices and components described in the embodiments include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors (micro signal processors), microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodied in The software may be distributed on networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and constructed for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (6)

Representing an element that causes a reservation failure (RF) in the first cache (L1) and the second cache (L2) to analyze the change in the main memory bottleneck point using the HBM as the main memory of the GPU;
Determining whether a cache is accessible according to whether the cache line is replaceable;
If a miss occurs in the cache, check and check whether there is a miss for the same data before according to the MSHR, and then merge them; And
Analyzing the speed of processing misses in the memory of the lower layer and indicating the memory bottleneck due to insufficient bandwidth as an indicator
Including,
The step of analyzing the speed of processing misses in the memory of the lower layer and indicating the memory bottleneck due to insufficient bandwidth,
When the queue to send a data access request is not full due to back pressure caused by the stall of the lower layer memory, between the first cache L1 and the second cache L2, between the second cache L2 and the main memory Indicates whether a smooth approach is being made,
If the value of the back pressure caused by the stall of the lower layer memory is large, it means that the processing of misses in the lower layer memory is slow, and the memory bottleneck due to insufficient bandwidth is used as an index to know,
Using the above indicator to solve the problem of the memory bottleneck point that occurs when HBM is used as the main memory of the GPU
How to troubleshoot memory bottlenecks. According to claim 1,
When a miss occurs in the cache, the step of checking whether there is a miss for the same data before and merging according to MSHR is performed.
When the mergeable limit is exceeded or the entry of the MSHR is insufficient, it is the cause of the RF, so when a miss occurs in the cache, the MSHR is checked, and if there is a miss for the same data before, the merge is performed.
How to troubleshoot memory bottlenecks. delete In order to analyze the change in the main memory bottleneck point using HBM as the main memory of the GPU, an analysis unit showing elements that cause reservation failure (RF) in the first cache L1 and the second cache L2
Including,
The analysis unit,
Determine whether the cache is accessible based on whether it is a replaceable cache line,
If there is a miss in the cache, check if there is a miss for the same data before according to MSHR, and merge it.
Analyze the speed of processing misses in the memory of the lower layer, and indicate the memory bottleneck due to insufficient bandwidth as an indicator,
When the queue to send a data access request is not full due to back pressure caused by the stall of the lower layer memory, between the first cache L1 and the second cache L2, between the second cache L2 and the main memory Indicates whether a smooth approach is being made,
If the value of the back pressure caused by the stall of the lower layer memory is large, it means that the processing of misses in the lower layer memory is slow, and the memory bottleneck due to insufficient bandwidth is used as an index to know,
Using the above indicator to solve the problem of the memory bottleneck point that occurs when HBM is used as the main memory of the GPU
Memory bottleneck troubleshooting device. According to claim 4,
The analysis unit,
When the mergeable limit is exceeded or the entry of the MSHR is insufficient, it is the cause of the RF, so when a miss occurs in the cache, the MSHR is checked, and if there is a miss for the same data before, the merge is performed.
Memory bottleneck troubleshooting device. delete

Images