KR20180012167A - Parallel processing unit, computing device including the same, and thread group scheduling method - Google Patents

Abstract

The present invention provides a parallel processing unit capable of reducing cache interference without dropping a thread level parallelism, a computing device including the same and a warp scheduling method. In the parallel processing unit including a multiprocessor, the multiprocessor comprises a memory unit which includes a level 1 data (L1D) cache and a sharing memory to which a plurality of cache lines are allocated. A thread group scheduler isolates a second thread group by changing a cache access of the second thread group into the sharing memory, when a first thread group of an active state satisfies a predetermined first condition.

Description

TECHNICAL FIELD [0001] The present invention relates to a parallel processing unit, a computing device including the same, and a thread group scheduling method.

The present invention relates to a parallel processing unit, a computing device including the same, and a method of scheduling a thread group.

A parallel processing unit, such as a graphics processing unit (GPU), has shown a lot of performance improvements for general purpose applications through massive thread-level parallelism (TLP). Current GPU architectures, which are made up of many stream multiprocessors (SMs), cluster multiple threads into a thread group [warp]. To effectively manage many warps, each SM employs a hardware-based warp scheduler. For high performance it is necessary to allocate as many warps as possible to each SM to maximize the utility of the execution pipeline.

This warp scheduling method is not sufficient to maximize TLP when memory intensive applications need to process large amounts of data. This is because Active Warp intensively generates too many memory requests and competes against a limited amount of on-chip cache. This cache contention can severely degrade performance. Although diverse cache-aware warp scheduling policies have been proposed to overcome this problem, these warp scheduling policies have been proposed to improve the L1D cache hit rate, It is aimed to improve performance by identifying warps and assigning higher scheduling priority to these warps than other warps. That is, these warp scheduling policies throttle the warp with low potential data locality to reduce the TLP, thereby increasing the overall L1D cache hit ratio and thereby improving overall performance. However, such a warp scheduling policy may not be sufficient for memory intensive applications involving irregular cache access patterns for the following reasons. Cache access of ActiveWar with high probability of data locality can often seriously interfere with other warps. As a result, simply scheduling Active Warp based on the likelihood of data localization can increase cache interference and negatively impact cache hit rate improvement. It is also undesirable to degrade TLP in exchange for cache hit rate improvement. That is, throttling the active warp can improve the cache hit rate, but the overall performance may deteriorate.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a parallel processing unit, a computing device including the parallel processing unit, and a warp scheduling method capable of reducing cache interference without degrading thread level parallelism.

According to one embodiment of the present invention, a parallel processing unit comprising a multiprocessor is provided. The multiprocessor includes a memory unit and a thread group scheduler. The memory unit includes an L1D cache and a shared memory to which a plurality of cache lines are allocated. The thread group scheduler changes the cache access of the second thread group to the shared memory to isolate the second thread group when a predetermined first condition is satisfied for the first thread group in the active state.

The first condition may include a condition that a level of cache interference for the first thread group exceeds a first cutoff.

The second thread group may include a thread group that most frequently interferes with the thread group causing the first thread group to interfere with the first thread group.

If the predetermined second condition is satisfied, the thread group scheduler may change the cache access of the second thread group back to the L1D cache.

The second condition may include a second cutoff beam or a lower condition in which the level of the cache interference for the first thread group is lower than the first cutoff.

The second condition may include a condition that execution of the first thread group is completed.

In the cycle consisting of the first and second time periods, the thread group scheduler can determine the first condition at the end of the first time and determine the second condition at the end of the second time.

If the predetermined second condition is satisfied, the thread group scheduler may stall the isolated second thread group.

The second condition may include a condition that a level of cache interference for a third thread group isolated to the shared memory exceeds a first cutoff.

If the predetermined third condition is satisfied, the thread group scheduler may reactivate the stalled second thread group.

The third condition may include a second cutoff beam level or a lower condition where the level of the cache interference for the third thread group is lower than the first cutoff.

The second condition may include a condition that execution of the third thread group is completed.

In the cycle consisting of the first and second periods, the thread group scheduler can determine the second condition at the end of the first period and determine the third condition at the end of the second period.

According to another embodiment of the present invention there is provided a computing device comprising a parallel processing unit, a CPU, a system memory, and a memory bridge connecting the parallel processing unit, the CPU and the system memory according to the preceding embodiments.

According to another embodiment of the present invention, a parallel processing unit comprising a multiprocessor is provided. The multiprocessor includes a memory unit including an L1D cache and a shared memory, and a thread group scheduler. The shared memory includes a plurality of shared memory banks having a plurality of rows, and the plurality of shared memory banks are grouped into a plurality of bank groups. A plurality of cache lines are allocated to a plurality of shared memory banks belonging to each bank group, and the thread group scheduler changes cache accesses of some of the thread groups to cache lines of the shared memory.

A bank group to which a certain cache line is allocated and a bank group to which another cache line is assigned may be assigned a number of the thread group corresponding to the tag for the certain cache line.

According to another embodiment of the present invention, there is provided a thread group scheduling method of a parallel processing unit comprising a memory unit including an L1D cache and a shared memory. Wherein the thread group scheduling method comprises the steps of: detecting a level of cache interference for a first thread group; and if a level of the cache interference exceeds a first cutoff, And isolating the second thread group by changing the cache access of the second thread group satisfying the condition to the shared memory.

Wherein the thread group scheduling method further comprises: detecting a level of cache interference for the first thread group, and if the level of the detected cache interference is lower than a second cutoff lower than the first cutoff, And changing the cache access of the thread group back to the L1D cache.

Wherein the thread group scheduling method comprises: detecting a level of cache interference for a third thread group isolated from the shared memory; and if the level of the cache interference for the third thread group exceeds a second cutoff, And stalling the isolated second thread group.

The method of claim 1, further comprising: detecting a level of cache interference for the third thread group; and detecting a level of cache interference for the third thread group from the second cut- The step of re-activating the stalled second thread group may further include the step of re-activating the stalled second thread group.

According to one embodiment of the present invention, cache interference can be reduced while maintaining thread-level parallelism.

1 is a schematic block diagram of a computing device according to one embodiment of the present invention.
2 is a schematic block diagram of a parallel processing unit according to one embodiment of the present invention.
3 is an illustration showing an example where cache interference in a typical multiprocessor degrades data locality in an L1D cache.
4 is a flowchart illustrating a method of scheduling a warp in a parallel processing unit according to an embodiment of the present invention.
5, 6, and 7 are diagrams illustrating warp partitioning in a parallel processing unit according to an embodiment of the present invention, respectively.
8 is a flowchart illustrating a method of scheduling a warp in a parallel processing unit according to another embodiment of the present invention.
9, 10, and 11 are diagrams illustrating warp throttling in the parallel processing unit according to an embodiment of the present invention, respectively.
12 is a view for explaining a method of managing a warp that most frequently causes interference in a parallel processing unit according to an embodiment of the present invention.
13 is a view for explaining cache interference in the parallel processing unit;
14 is a view for explaining a big-tag array in a parallel processing unit according to an embodiment of the present invention.
15 and 16 are flowcharts illustrating a method of a warp scheduling in a parallel processing unit according to an embodiment of the present invention, respectively.
17 is a schematic block diagram of a multiprocessor in a parallel processing unit according to an embodiment of the present invention.
18 is a view for explaining the operation of the memory unit in the parallel processing unit according to an embodiment of the present invention.
19 is a view for explaining the structure of a memory unit in the parallel processing unit according to one embodiment.
20 is a diagram illustrating address translation in a parallel processing unit according to one embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

1 is a schematic block diagram of a computing device according to one embodiment of the present invention. 1 is an example of a possible computing device and may be implemented in various other configurations of computing devices according to embodiments of the present invention.

1, a computing device according to one embodiment of the present invention includes a central processing unit (CPU) 110, a system memory 120, and a parallel processing unit 130.

The system memory 120 may communicate with the CPU 110 via the memory bridge 140. The memory bridge 140 may be, for example, a north bridge. The memory bridge 140 may also be connected to an input / output (I / O) bridge 150 via a bus or communication channel. The I / O bridge 150 may be, for example, a southbridge and may receive user input from a user input device (not shown) and forward it to the CPU 110 via the memory bridge 140 .

The parallel processing unit 130 may be connected to the memory bridge 140 via a bus or communication channel to communicate with the CPU 110 and the system 120. The parallel processing unit 130 may be, for example, a graphics processing unit (GPU).

In some embodiments, a system including the CPU 110, the system memory 120, the memory bridge 140, and the I / O bridge 150 may be referred to as a host.

2 is a schematic block diagram of a parallel processing unit according to one embodiment of the present invention.

Referring to FIG. 2, the parallel processing unit includes one or more multiprocessors 200. The parallel processing unit may be, for example, a GPU, and the multiprocessor 200 may be, for example, a streaming multiprocessor (SM).

A series of instructions passed to the multiprocessor 200 may form a thread. These threads may be instances of the program. In addition, a predetermined number (for example, 32) of threads are simultaneously executed in the multiprocessor 200, and this predetermined number of threads is called a thread group. For example, such a thread group may be referred to as a "warp", a "container", or a "wavefront". In some embodiments, a thread group may be a group of threads that simultaneously execute the same program for different input data. Also, a plurality of warps may be active simultaneously in the multiprocessor 200, and a plurality of threads of the warp may be referred to as a cooperative thread array (CTA).

The multiprocessor 200 includes an instruction buffer 210, a warp scheduler 220, a plurality of processing units 230, a plurality of load-store units (LSUs) 240, (250).

The instruction buffer 210 stores instructions transferred to the multiprocessor 200. The warp scheduler 220 schedules the instructions of the warp and selects the corresponding instruction of the warp from the instruction buffer 210 according to the scheduling, do. In some embodiments, the multiprocessor may further include a fetch / decoder 205, which fetches and decodes the instruction passed to the multiprocessor 200 and then stores it in the instruction buffer 210 do. In some embodiments, each active warp in the instruction buffer 210 may have a dedicated entry set. In some embodiments, the multiprocessor 200 may further include a scoreboard (not shown) to identify data risks. In this case, if the scoreboard indicates that there is no risk to the warp, the warp scheduler 220 may select the decoded instruction of this warp from the instruction buffer 210.

Each processing unit 230 processes the instructions output from the warp scheduler 220. The processing unit 230 may perform various computing operations, such as integer and floating point arithmetic, comparison operation, Boolean operation, bit-shifting, It can support operations such as calculation of various algebraic functions. In some embodiments, the processing unit 230 may be, for example, an arithmetic logic unit (ALU). Each load-store unit 240 loads data from the memory unit 250 or stores it in the memory 250.

The memory unit 250 includes a level 1 data (L1D) cache 251 and a shared memory 252. In some embodiments, the memory unit 250 may be an on-chip memory unit of the multiprocessor 200. The L1D cache 251 and the shared memory 252 may share a single on-chip memory structure including, for example, 32 banks with 512 or 256 rows. N rows may be allocated to the L1D cache 251 and remaining (e.g., (512-N) or (256-N)) rows may be allocated to the shared memory 252. [ For example, each multiprocessor 200 may support a 64 KB on-chip memory architecture where 16 KB (N = 128) or 48 KB (N = 384) can be allocated to the L1D cache 251.

The shared memory 252 may be managed by software. Each CTA can request an exclusive shared memory space for inter-thread communication. Thus, in some embodiments, the multiprocessor 200 may have an independent shared memory management table (SMMT) 261. Each CTA reserves one SMMT entry, and the entry records the start address and size of the shared memory space for a given CTA (CTAid). Also, in some embodiments, the multiprocessor 200 may connect the load-store unit 240 and the memory unit 250 via a crossbar network 262.

In an embodiment of the present invention, a plurality of cache lines are allocated to the shared memory 252, so that cache access of some warps is changed to the shared memory 252. [

In some embodiments, the multiprocessor 200 may include a hardware coalescing unit (e.g., a coalescing unit) to combine the plurality of memory requests generated by the threads of the warp into a small but large memory request to improve the usability of the memory bandwidth 263, and a miss status holding register (MSHR) 264.

In this multiprocessor 200, since a plurality of warps share an L1D cache 251 having a limited capacity, a plurality of warps can compete for the same cache line. Thus, in the L1D cache 251, the cached data of some active warps may be evicted by cache access of other active warps requesting memory access, thereby losing data locality. This phenomenon is called cache interference. In particular, cache irregularities can cause cache interference when the cache access pattern is irregular.

3 is an illustration showing an example where cache interference in a typical multiprocessor degrades data locality in an L1D cache.

Assume, for example, that two warps W0 and W1 repeatedly request data D0 and D4, respectively, in the same cache set (Set0) of the L1D cache, as shown in Fig. The warp W0 requesting the data D0 in the cycle CA evacuates the data D4 held by the warp W1 from the L1D cache 270. However, since there is no data D0 in the cache set Set0, (cold miss) occurs. The warp W1 requesting the data D4 in the cycle CB causes a miss (collision miss) because there is no data D4 in the cache set Set0 and the data request of the warp W1 is a warp W0 (D0) from the cache set (Set0). Similarly, the warp W0 requesting the data D0 in the cycle CE evicts the data D4 with the warp W1 from the cache set Set0 and the warp W1 requesting the data D4 from the cycle CF. ) Can evacuate the data D0 held by the warp W0 from the cache set Set0. As a result, the collision continues to occur.

On the other hand, if the same data is referenced many times during the execution of the application, that is, the kernel, data locality exists. In FIG. 3, the warps W0 and W1 continue to generate memory requests to repeatedly obtain the data D0 and D4 in the cycles CA, CB, CE and CF, respectively. These memory requests must have hit the L1D cache if the data DO and D4 have not been evicted by other warps W1 and W0. This cache hit opportunity can be referred to as the potential for data localization and is measured as the frequency of re-referencing the same data in the absence of cache interference. In FIG. 3, both warps W0 and W1 exhibit a high likelihood of data locality, but cache interference between them results in an unnecessary cache miss, which can turn regular memory access into irregular cache access.

To reduce this cache interference, the interfering warp can be isolated from the interfering warp. To this end, in some embodiments, the L1D cache space may be partitioned and the partitioned cache line allocated to the interfering warp. Various techniques for partitioning the shared cache space for the current CPU have been proposed. However, in a parallel processing unit such as a GPU, the number of threads sharing the L1D cache line is very large compared to the number of CPU threads, whereas the size of the L1D cache is not large enough to apply the CPU-based cache partitioning technique to the parallel processing unit. For example, applying a CPU-based cache partitioning technique to an L1D cache allows only two or three cache lines to be allocated to each warp. Then cache thrashing can be aggravated by a small number of cache lines per warp.

On the other hand, 75% of the shared memory was not used on average when the shared memory usage space for 21 applications was analyzed by the benchmark method such as PolyBench, Mars, Rodinia and the like. In one embodiment of the present invention, this unused shared memory space is used to allow an interfering warp to access a shared memory space that is not used instead of the L1D cache.

Hereinafter, a description will be made of a warp scheduling method for reducing cache interference in a parallel processing unit according to an embodiment of the present invention.

FIG. 4 is a flowchart illustrating a method of scheduling warps in a parallel processing unit according to an embodiment of the present invention. FIGS. 5, 6, and 7 are flowcharts of warp partitioning in a parallel processing unit according to an embodiment of the present invention warp partitioning.

4 and 5, since there is no cache interference at the beginning of kernel execution, memory requests of all warps W0, W1, W2 and W3 are directed to the L1D cache 510 (S410). As kernel execution proceeds, some warps may begin to compete with each other to obtain a particular cache line in the L1D cache 510.

If the predetermined condition is satisfied (S420), the multiprocessor of the parallel processing unit detects a warp causing interference (W3, for example, requesting data D3 in FIGS. 5 and 6) (S430). In certain embodiments, the predetermined condition may be that the intensity of the cache interference in the L1D cache 510 exceeds the threshold. That is, when the intensity of the cache interference exceeds the threshold value, the multiprocessor can detect the warp causing the interference. Next, as shown in FIG. 6, the multiprocessor partitions the active warfar into the warp that interferes with the interfering warp, and changes to the shared memory space 520 that does not use the cache access of the interfering warp (W3) (step S440) to isolate the warping W3 causing the interference by redirection. This can reduce cache contention without compromising thread-level parallelism (TLP) in the parallel processing unit.

In the case where other predetermined conditions are satisfied due to a change in the cache access pattern or the completion of execution of some warps (for example, interfering warps) (S450), the multiprocessor, as shown in FIG. 7, And changes the memory request of this warp from the shared memory 520 to the L1D cache 510 (S460). In some embodiments, the other predetermined condition may be a condition that the cache contention is significantly reduced, for example, a condition where the intensity of the cache interference is lower than the threshold value. That is, upon detecting a significant reduction in cache contention, the multiprocessor aborts the isolation policy of the interfering warp and changes the memory request of this warp back to the L1D cache 510. In one embodiment, the threshold at step S450 may be different from the threshold at step S420. For example, the threshold at step S450 may be less than the threshold at step S420. In another embodiment, the threshold at step S450 may be equal to the threshold at step S420.

In some embodiments, the multiprocessor may use an interference detector (not shown) to determine the number of cache contention in steps S420 and S450, i.e., the strength of the cache interference.

The cache interference reduction method, that is, the warp partitioning method described with reference to FIGS. 4 to 7, may be performed by a warp scheduler (220 in FIG. 2) of the multiprocessor.

In some embodiments, a memory request (i. E., A cache access request) may be communicated to the L1D cache 510 or shared memory 530 via the crossbar network 530.

On the other hand, when the memory request of the warp W3 causing the multiprocessor to interfere with is changed from the L1D cache 510 to the shared memory 520, the shared memory 520 has the data D3 requested by the warp W3 It may not be. In this case, performance degradation and coherence due to a cold miss may occur with respect to the warp W3. To solve this problem, in some embodiments, when the target data D3 is present in the L1D cache 510 when accessing the shared memory 520 is needed, the multiprocessor, as shown in Figure 6, The data D3 can be directly retired from the cache 510 to the predetermined queue 540. [ In one embodiment, when it is necessary to access the shared memory 520, the multiprocessor may check the tag array of the L1D cache 510 to see if the target data D3 is present in the L1D cache 510. [ The next multiprocessor may populate shared memory 520 with data D3 evacuated to a predetermined queue 540 as shown in FIG. Accordingly, the data can be migrated from the L1D cache 510 to the shared memory 520 to solve the inconsistency problem with the cold cache miss.

In some embodiments, the predetermined queue 540 may be a response queue used to buffer data fetched from the L2 cache and invalidate the cache line in the L1D cache.

In some embodiments, the shared memory 520 issues a fill request to an MSHR (not shown), and based on the entry in the MSHR, the target data is transferred from the given queue 540 to the shared memory 520 It can be filled directly.

According to the embodiment described above, by using the unused shared memory space as the L1D cache, it is possible to effectively prevent the interference-causing warp from being interfered with the warp. On the other hand, the blocking efficiency may depend on various runtime factors such as the number of warp blocks interrupted and space not used in shared memory. For example, it may not be possible to effectively reduce cache interference when an interfering warp is thrashing shared memory. That is, the size and / or bandwidth of unused shared memory space may not be sufficient to handle large amounts of memory requests from the interfering warp in a short period of time.

On the other hand, there are two major problems in using the unused shared memory space 520 as the L1D cache 510.

The first is that the shared memory 520 has its own address space separate from the global memory and there is no hardware support to translate global memory addresses into shared memory addresses. To that end, in some embodiments, the multiprocessor may further include an address translation unit (not shown) that translates the global memory address given in front of the shared memory into a local memory address in the shared memory 520.

The second is that the shared memory 520 does not have a data path that directly accesses the lower memory layer, such as the L2 cache and main memory. To this end, in some embodiments, the multiprocessor may adjust the data path between the L1D cache 510 and the L2 cache (not shown) so that the shared memory 520 can access the L2 cache when the shared memory space is acting as a cache . In one embodiment, the multiprocessor may further include a multiplexer 550 between the memory unit 510, 520 and a queue, such as a response queue 540, for buffering data in a lower memory, such as an L2 cache. The multiplexer 550 may selectively connect the queue to the L1D cache 510 or the shared memory 520. [ The multiprocessor may also store a shared memory address of a memory request transferred from the address translation unit. In one embodiment, the multiprocessor may add an extension field to each MSHR entry to store the shared memory address. In this case, if the shared memory 520 issues a fill request after a miss, the request reserves one MSHR entry by populating the global and translated shared memory addresses. If the response from the L2 cache, i.e., the data evicted to the response queue 540, matches the global memory address recorded in the corresponding MSHR entry, the data is stored directly in the shared memory 520 based on the translated shared memory address .

FIG. 8 is a flowchart illustrating a method of scheduling a warp in a parallel processing unit according to another embodiment of the present invention. FIGS. 9, 10, and 11 are diagrams illustrating warp throttling in a parallel processing unit according to an embodiment of the present invention, (warp throttling).

8 and 9, the data requested by the two warps W0 and W1 are respectively mapped to the cache set (Set0 and Set1) in the L1D cache 910 and the memory request of the warp W3 is mapped to the shared memory And the data requested by the warp W3 is mapped to the cache set (Set0) in the shared memory 920. [ It is also assumed that the memory request of the warp W2 causing the interference to the cache set (Set1) of the L1D cache 910 is changed to the cache set (Set1) of the shared memory 920. [ In this case, if the memory request of the warp W4 is changed to the cache set (Set1) of the shared memory 920, the two warps W2 and W4 repeatedly access the cache set (Set1) of the shared memory 920 Cache interference occurs (S810). Thus, the warps W2 and W4, which cause cache interference, can eventually thrash the shared memory 920. [

The multiprocessor monitors the strength of the shared memory interference in the warp that has been changed to the shared memory 920, i.e., the isolated warps W2, W3 and W4 (S820). As a result of the monitoring, if the predetermined condition is satisfied (S830), the multiprocessor stall (or the throttle) (for example, W2 in FIG. 10) throttle) (S840). In certain embodiments, the predetermined condition may be that the intensity of the shared memory interference exceeds the threshold. In certain embodiments, the selected warp in accordance with certain conditions may be a warp that causes the most cache interference. In some embodiments, stalling (S840) may continue until the strength of the shared memory interference falls below a threshold.

When the other predetermined conditions are satisfied due to the change of the cache access pattern or the completion of the execution of some warp (S850), the multiprocessor reactivates the stalled warp W2 as shown in Fig. 11 (S860). The re-activated warp W2 can again access the corresponding cache set (Set1) of the shared memory 920. [ Accordingly, it is possible to maintain a high TLP in the parallel processing unit and to maximize the utilization of the shared memory 920. [ In some embodiments, other predetermined conditions may be conditions under which the intensity of interference in shared memory 920 is reduced, e.g., the number of cache contention in shared memory 920 is lower than a threshold. In one embodiment, the threshold at step S850 may differ from the threshold at step S830. For example, the threshold at step S850 may be less than the threshold at step S830. In another embodiment, the threshold at step S850 may be equal to the threshold at step S830.

In some embodiments, the multiprocessor may use an interference detector (not shown) to determine the strength of the interference at steps S830 and S850. In one embodiment, the multiprocessor may use the interference detectors used in steps S420 and S450 in FIG. 4 in steps S830 and S850. This is because the isolated warp does not compete with the L1D cache 910 with the regular warp accessing the L1D cache 910 and causes interference with other isolated warps in the shared memory 920. [ Therefore, the interference in the L1D cache 910 and the shared memory 920 does not affect each other, and thus the multiprocessor can share the interference detector. In another embodiment, the multiprocessor may use steps S830 and S850 for interference detectors other than the interference detectors used in steps S20 and S450 in FIG.

According to the embodiment described above, it is possible to prevent some interfering warps from skimming the shared memory by stalling some warps set in the shared memory 920 when the intensity of the cache interference in the shared memory 920 is high .

Next, interference detection in the parallel processing unit according to one embodiment of the present invention will be described.

FIG. 12 is a view for explaining a method for managing a warp that most frequently causes interference in a parallel processing unit according to an embodiment of the present invention, and FIG. 13 is a view for explaining cache interference in a parallel processing unit.

Cache accesses of some warps may seriously interfere with cache accesses of other warps, resulting in non-uniform cache interferences. However, it is difficult to capture such non-uniform cache interference information by simply examining the application code or cache replacement policy for an application having an irregular cache access pattern. In some embodiments, it is possible to track the cache misses caused by all other warps for each warp to detect interference. In this case, since n (n-1) entry storage structures are required, high storage costs can occur. Where n is the number of active warps in the multiprocessor.

Referring to FIG. 12, the multiprocessor manages the interference list, assigns one entry of the interference list to each warp, and updates the warp number (WID) of the warp most recently interfering with the currently executed warp, It can be stored in the entry corresponding to the warp. In some embodiments, the multiprocessor can track a warp that most frequently interferes with each warp recently. In one embodiment, the multiprocessor can only track the most frequently interfering warps for each warp. This can reduce storage costs to track the warping that causes interference for each warp.

To track the most frequently interfering warp for a given warp, the multiprocessor may further include a saturation counter associated with each entry in the interference list. In one embodiment, the saturation counter may be a 2-bit saturation counter.

As shown in Fig. 13, among the warps W32, W36, W38, W40 and W42 that cause interference recently with respect to the warp W34, the warp W32 causes the interference most frequently, and the remaining warps W2- ) Is assumed to be a warp that does not interfere with the warp W34 at all. If the multiprocessor determines that the previously executed warp W32 causes the interference with the currently executed warp W34 most often, the WID of the warping W32 causing the interference as shown in Fig. ). ≪ / RTI > Each time the warp W32 interferes with the warp W34, the saturation counter increases by one. It is assumed that the saturation counter has already reached a saturation value (e.g., '11') in some cycle (S1310). When another warp W42 interferes with the warp W34 in the following cycle, the saturation counter is decreased by 1 (S1320). If the next warp W32 interferes again with the warp W34, the saturation counter is increased by one (S1330). As described above, when the saturation counter decreases to '00' due to interference of warp W32 other than the warp W32 stored in the entries of the interference list for a certain warp W34, the warp W32 in the interference list becomes' ≪ / RTI > Thus, the most frequent cache interference can be maintained in the interference list.

In some embodiments, the multiprocessor may detect the strength (level) of the cache interference experienced by the individual warp by an individual re-reference score (IRS). The IRS (IRS _i ) of the warp i Wi can be expressed, for example, by the following equation (1).

는 워프 i에 대한 빅팀 태그 어레이(victim tag array, VTA) 히트의 개수이며, N_{executed - inst}는 실행된 명령어의 총 개수이고, N_active-warp는 멀티프로세서 상에서 동작하는 액티브 워프의 개수이다.Here, i is the number of the active warp,

Is the number of victim tag array (VTA) hits for warp i, N _{executed - inst} is the total number of executed instructions, and N _{active - warp} is the number of active warps running on the multiprocessor.

In one embodiment, the VTA may be included in the memory unit of the multiprocessor to manage the cache access history of each warp. Referring to FIG. 14, the VTA includes a plurality of entries each corresponding to a plurality of warps, and the field of each entry can store the tag of the corresponding warp.

The multiprocessor can attach the WID of the warp corresponding to the cache line of the L1D cache. The WID attached to the cache line can be used to track which active warp has brought the current data to the cache line. When the cache line is evicted, the multiprocessor retrieves the WID from the evicted cache line and stores the tag of the evicted cache line in the VTA entry associated with that WID. Thus, whenever there is a miss in the L1D cache, the multiprocessor can observe the VTA. In this case, if a tag is found in the VTA entry of the corresponding warp, the multiprocessor can count the VTA hit. That is, the multiprocessor can determine that the warp shows the possibility of data localization if the VTA observes that the warp repeatedly places memory requests on the same cache line (i.e., observes VTA hits).

Referring again to Equation (1), IRS _i indicates a VTA hit for each instruction (i.e., intensity of VTA hit) for warp i. Thus, a high IRS _i can indicate that warp i experiences severe cache interference during a given epoch. N _{executed-inst} and N _active-warp May vary over time, so in one embodiment IRS _i may be periodically updated to account for (N _{executed-inst} / N _active-warp ) for a given time period. Based on the IRS _i for a given time, the warp scheduler will either (1) partition the warp causing the interference with the warp i (ie, isolate), (2) stall the interfering warp, or (3) And to re-activate the warp.

(2) low cutoff and (2) high cutoff to determine whether the warp scheduler should partition, stall, or reactivate the interfering warp. In some embodiments, Can be used. An IRS _i higher than a high cutoff may indicate that warp i experiences a high level of cache interference. Thus, if the IRS _i is higher than the high cutoff, the warp scheduler can partition or stall the warp that most recently interferes with warp i most recently. An IRS _i lower than the low cutoff may indicate that warp i is experiencing low level cache interference or has completed execution. Thus, if the IRS _i is lower than the lower IRS, the warp scheduler can reactivate the previously stowed warp or withdraw the isolation of the previously isolated warp (change the memory request of the previously isolated warp to the L1D cache).

Under the conditions illustrated in Table 1 below, the evaluation of the diverse memory intensive applications can result in, for example, a high cutoff and a low cutoff of approximately 1% and 0.5%, respectively.

As the IRS changes over time, the warp scheduler can periodically compare the IRS with the high cutoff and low cutoff to determine exactly if the warp needs to be partitioned, stalled, or reactivated. To this end, in some embodiments, the warp scheduler may divide the execution time into a high cutoff period and a low cutoff period. At the end of each high cutoff period, the warp scheduler can compare the IRS to the high cutoff when cycles of high cutoff and low cutoff times are repeated, and at the end of each low cutoff period the warp scheduler can compare the IRS to the low cutoff .

In one embodiment, the low cutoff period may be shorter than the high cutoff period. As such the low cutoff period is shorter than the high cutoff period, once the previously stalled warp begins to interfere significantly with the other warp at runtime, the negative effects of the stalled warp can be minimized by reactivating these warp . As a result, high TLP can be maintained.

As a result of evaluating the diverse memory intensive applications under the conditions illustrated in Table 1 below, for example, the lengths of the high cutoff period and the low cutoff period can be set to approximately 5000 and 100 instructions, respectively.

Next, a warp scheduling method using a high cutoff and a low cutoff will be described with reference to FIGS. 15 and 16. FIG.

15 and 16 are flowcharts illustrating a method of a warp scheduling in a parallel processing unit according to an embodiment of the present invention, respectively.

Referring to Fig. 15, the multiprocessor of the parallel processing unit detects the level of cache interference for active warp i (S1510). In some embodiments, the multiprocessor may detect a cache interference level at the end of a high cutoff period. If the cache interference level of warp i satisfies a predetermined condition (S1520), the multiprocessor detects a warp j that most frequently interferes with warp i (S1530). In some embodiments, the predetermined condition may be a condition where the cache interference level, e.g., ISR _i, is higher than the high cutoff. Accordingly, the multiprocessor isolates the cache access of warp j to the shared memory (S1540). In some embodiments, the multiprocessor may record the warp number of warp i that isolated warp j.

Next, the multiprocessor detects the cache interference level for the warp (i.e., warp i) that isolated warp j (S1550). In some embodiments, the multiprocessor may detect a cache interference level at the end of a low cutoff period. If the cache interference level of warp i satisfies another predetermined condition (S1560), the multiprocessor withdraws the isolation of warp j (S1570). As a result, warp j can again access the L1D cache. In one embodiment, some other condition may be a condition where the cache interference level of warp i, e.g. ISR _i, is lower than the low cutoff. In another embodiment, some other condition may be a condition in which the execution of warp i is complete. Otherwise, if the predetermined condition is not satisfied (S 1560), the multiprocessor continues to isolate the cache access of warp j (S 1570).

Thus, by isolating warp j, which frequently causes interference with warp i, it is possible to reduce cache interference while maintaining TLP.

Referring to FIG. 16, the multiprocessor of the parallel processing unit detects the level of cache interference for warp i (S1610). Warp i may be a warp that accesses the L1D cache or a warp that accesses the shared memory. In some embodiments, the multiprocessor may detect a cache interference level at the end of a high cutoff period. When the cache interference level of warp i satisfies a predetermined condition (S1620), the multiprocessor detects a warp j that most frequently interferes with warp i (S1630). In some embodiments, the predetermined condition may be a condition where the cache interference level, e.g., ISR _i, is higher than the high cutoff.

The multiprocessor determines whether the warp j is in an isolated state (S1640). If warp j is in an isolated state (S1640), the multiprocessor stalls warp j (S1642). In some embodiments, the multiprocessor may record the warp number of warp i that stuck warp j. If the warp j is not in the isolated state, that is, if the warp j is active (S1640), the multiprocessor isolates the cache access of warp j to the shared memory (S1644). In some embodiments, the multiprocessor may record the warp number of warp i that isolated warp j.

Next, the multiprocessor detects the cache interference level for the warp (i.e., warp i) that isolated the warp j (S1650). In some embodiments, the multiprocessor may detect a cache interference level at the end of a low cutoff period. If the cache interference level of warp i satisfies another predetermined condition (S1660), the multiprocessor withdraws the isolation of warp j (S1670). As a result, warp j can again access the L1D cache. In one embodiment, some other condition may be a condition where the cache interference level of warp i, e.g. ISR _i, is lower than the low cutoff. In another embodiment, some other condition may be a condition in which the execution of warp i is complete. On the other hand, if the predetermined condition is not satisfied (S1660), the multiprocessor continues to isolate the cache access of warp j.

Alternatively, the multiprocessor detects a cache interference level for a warp (i.e., warp i) in which warp j is stalled (S1680). In some embodiments, the multiprocessor may detect a cache interference level at the end of a low cutoff period. When the cache interference level of warp i satisfies another predetermined condition (S1690), the multiprocessor reactivates warp j (S1695). In one embodiment, some other condition may be a condition where the cache interference level of warp i, e.g. ISR _i, is lower than the low cutoff. In another embodiment, some other condition may be a condition in which the execution of warp i is complete. On the other hand, if the predetermined condition is not satisfied (S1690), the multiprocessor continues to stall the cache access of warp j.

As such, cache interference can be reduced by stowing the isolated warp j if the isolated warp j again causes cache interference. In addition, if cache interference is reduced, the TLP can be maintained by reactivating the stalled warp j.

Next, a structure of a multiprocessor in the parallel processing unit according to an embodiment of the present invention will be described.

17 is a schematic block diagram of a multiprocessor in a parallel processing unit according to an embodiment of the present invention. In Fig. 17, a warp scheduler and a memory unit are shown in a multiprocessor.

)를 기록하고, 전체 명령어 카운터(Inst-total)는 해당 멀티프로세서에 의해 실행된 명령어의 총 수(N_{executed-inst})를 기록한다. 멀티프로세서는 복수의 VTA 히트 카운터(VTACount0-VTACountk)와 전체 명령어 카운터(Inst-total)를 통해 개별 워프가 겪는 캐시 간섭의 레벨을 검출할 수 있다. 어떤 실시예에서 캐시 간섭의 레벨은 IRS로 표현될 수 있다.17, the multiprocessor's warp scheduler 1710 includes a counter 1711 for detecting the level of cache interference. In some embodiments, the counter 1711 may include a plurality of VTA hit counters (VTACount0-VTACountk) and an entire instruction counter (Inst-total). In this case, the VTA hit counter may be provided per warp. Each VTA hit counter (VTACounti) represents the number of VTA hits (< RTI ID = 0.0 >

), And the total instruction counter (Inst-total) records the total number (N _{executed-inst} ) of instructions executed by the corresponding multiprocessor. The multiprocessor can detect the level of cache interference experienced by individual warps through multiple VTA hit counters (VTACount0-VTACountk) and the entire instruction counter (Inst-total). In some embodiments, the level of cache interference may be expressed as an IRS.

The warp scheduler 1710 may further include a cutoff check unit 1712 for comparing the level of cache interference with a threshold. In some embodiments, the level of cache interference is IRS, and the threshold may include a high cutoff and a low cutoff. The cutoff checking unit 1712 then compares the IRS with the high cutoff and the low cutoff. In one embodiment, the cutoff check unit 1712 may be implemented using registers, shifting units, and compare logic. The warp scheduler may further include a sampler 1713 which counts the number of instructions executed to determine if a high cutoff period or a low cutoff period has expired.

The warp scheduler 1710 includes a list for managing the warp causing the interference. In some embodiments, the list may include a warp list 1714, an interference list 1715, and a pair list 1716.

The interference list 1714 can be used to manage information related to tracking the warp causing the interference. The interference list 1714 includes a plurality of entries each assigned to a plurality of warps. Each entry is indexed by the WID of the assigned warp and stores the WID and saturation counter (C) of the interfering warp. Every time a VTA hit occurs, the corresponding entry in the interference list can be updated. For example, each time the warp equal to the WID stored in the entry causes interference, the saturation counter C is incremented by one, and each time the WID stored in the entry and the other warp cause interference, the saturation counter C is decremented by one can do. In this case, when the saturation counter C reaches a saturation value (for example, '11'), it does not increase further, and when the saturation counter C reaches a predetermined value (for example, '00' The WID of the entry can be updated with the WID of another warp.

The interference list 1714 may be checked whenever it is necessary to partition the interfering warp based on the ISR. To this end, a warp list 1715 comprising a plurality of ready warp entries contains a status flag coupled to each ready warp entry. The warp scheduler can use the status flags to identify the status of a given warp, that is, whether the given warp is active or isolated. For example, the state flag includes the isolation flag I, and the case where the isolation flag I is 0 is the active state (I = 0) and the isolation flag I is 1 is the isolation state (= 1 ). ≪ / RTI >

In another embodiment, the warp scheduler may further manage the stall state. In this case, the interference list 1714 can be checked whenever it is necessary to partition or stall the interfering warp based on the ISR. In some embodiments, the status flag may include an active flag V and an isolation flag I. In one embodiment, the active flag and the isolation flag may each be one bit. The warp scheduler can use the status flags (V, I) to identify the status of a given warp, i.e. whether the given warp is active, isolated or stalled. For example, when the active flag is 1 and the isolation flag is 0, the active state (V = 1, I = 0) ), And when the active flag is 0, the stall state (V = 0) can be identified.

The pair list 1716 includes a plurality of entries each assigned to a plurality of warps, each entry being indexed by the WID of the assigned warp. Each entry records which interfering warp has caused warp partitioning in the past. To this end, in some embodiments, each entry may include a field for recording the WID of the interfering warp that caused the warp partitioning. Based on the WID of the field of the entry corresponding to warp i, the warp scheduler identifies the ISR (ISR _k ) of the interfering warp k that caused the partitioning of warp i previously. For example, as warp W0 is severely interfered by warp W1, the warp scheduler may decide to partition warp W1 to isolate its cache access. 17, the WID of the warp W0 is recorded in the field of the pair list entry corresponding to the warp W1, and the isolation flag I of the interference list entry corresponding to the warp W1 is set have. Warp W1 begins to access the shared memory after being partitioned. When the warp scheduler needs to withdraw the separation of the warp W1 later, the field of the pair list entry corresponding to the warp W1 and the isolation flag I of the interference list entry corresponding to the warp W1 are cleared .

In another embodiment, each entry in the pair list 1716 may additionally record which interfering warp has caused warp stalling in the past. To this end, in some embodiments, each entry may include two fields. The first field records the WID of the interfering warp that caused the warp partitioning and the second field can record the WID of the interfering warp that caused the warp stalling. Based on the WIDs of the two fields of the entry corresponding to warp i, the warp scheduler identifies the ISR (ISR _k ) of the interfering warp k that previously caused the partitioning or stalling of warp i. For example, as warp W0 is severely interfered by warp W1, the warp scheduler may decide to partition warp W1 to isolate its cache access. 17, the WID of the warp W0 is recorded in the first field of the pair list entry corresponding to the warp W1, and the isolation flag I of the interference list entry corresponding to the warp W1 is set . Warp W1 begins to access the shared memory after being partitioned. In this case, if the warp scheduler judges that the warp W1 severely interferes with another warp W3 accessing to the shared memory, the warp W1 can be stalled. The WID of the warp W3 is recorded in the second field of the pair list entry corresponding to the warp W1 and the interference list entry corresponding to the warp W1 The active flag V can be cleared. When the warp scheduler needs to reactivate the warp W1 later, the second field of the pair list entry corresponding to the warp W1 and the active flag V of the interference list entry corresponding to the warp W1 are cleared . Further, when the warp scheduler needs to withdraw the separation of the warp W1, the first field of the pair list entry corresponding to the warp W1 and the isolation flag I of the interference list entry corresponding to the warp W1 Can be cleared.

The multiprocessor memory unit 1720 includes an L1D cache 1721 and a shared memory 1722. Data and tags are stored in the cache line of the L1D cache 1721, and the WID of the corresponding warp can be attached. The unused space in the shared memory 1722 is used as a cache, data and tags can be stored in the cache line of the space used as a cache, and the WID of the corresponding warp can be attached to the tag. In some embodiments, the memory unit 1720 may further include a VTA 1730 for writing the tag in the entry of the warp corresponding to the miss of the cache line.

The warp scheduler 1710 and the memory unit 1720 may be connected through a load-store unit 1730. [

FIG. 18 is a view for explaining the operation of a memory unit in the parallel processing unit according to an embodiment of the present invention, FIG. 19 is a view for explaining the structure of a memory unit in the parallel processing unit according to an embodiment, 1 illustrates address translation in a parallel processing unit according to one embodiment.

18, the multiprocessor includes a memory unit 1810, control logic 1820 and a multiplexer 1830 and a queue 1840, and the memory unit 1810 includes an L1D cache 1811 and a shared memory 1812 ).

When using the shared memory 1812 as a cache, the shared memory 1812 is logically separated from the global memory so that the multiprocessor establishes a data path between the shared memory 1812 and the L2 cache. To this end, the multiplexer 1830 selectively connects the L1D cache 1811 or the shared memory 1812 with the queue 1840, which buffers data from the global memory or the L2 cache. This queue 1840 may include a write queue 1841 and a response queue 1842.

The control logic 1820 controls the multiplexer 1830 based on the state of the given warp. If the state of a given warp is in an isolated state (e. G., If the isolation flag I in FIG. 17 is ' 1 ', then the control logic 1820 will generate a data path between the shared memory 1812 and the queue 1840 So that the multiplexer 1830 can be controlled. The control logic 1820 may further reference the result of checking the cache tag (hit / miss) when controlling the multiplexer 1830.

The multiprocessor may also store a shared memory address (SHM Addr) of the memory request from the address translation unit. In some embodiments, the multiprocessor may add an extension field to each MSHR entry to store the shared memory address of the memory request from the address translation unit. Each MSHR entry may further include an address valid flag V, an instruction ID (e.g. a program counter (PC)) and a global memory address Gl Addr.

If the shared memory 1812 issues a fill request after a miss, the request reserves one MSHR entry by populating the global and translated shared memory addresses. If the response from the L2 cache matches the global memory address recorded in the corresponding MSHR entry, the data may be stored directly in the shared memory 1812 based on the translated shared memory address.

In contrast to the LID cache 1811, the shared memory 1812 does not have a separate memory structure for accommodating cache tags. In an embodiment, additional tag arrays may be employed to accommodate the cache tag of the shared memory 1812.

In some embodiments, a plurality of cache data blocks (i.e., cache lines) and their respective tags may be placed in shared memory 1812 instead of an additional tag array. For example, the cache data block may be a 128-byte cache data block. Accordingly, the change of the on-chip memory unit 1810 composed of the L1D cache 1811 and the shared memory 1812 can be minimized. To this end, as shown in FIG. 19, a plurality of shared memory banks (for example, 32 shared memory banks) included in the shared memory 1812 are divided into two or more bank groups. In one embodiment, the number of divided bank groups may be two. Then, the cache data blocks are striped in the row direction for a plurality of banks (for example, 16 banks) in one bank group. Since each shared memory bank in a typical parallel processing unit allows 64-bit access, a cache data block, for example a 128-byte cache data block, can be accessed in parallel.

One tag and its WID require 31 bits when a limited bit, e.g., a tag uses 25 bits and a warp number (WID) uses 6 bits, so that the shared memory space of the shared memory 1812 You can provide a tag for the line. For example, if the shared memory bank allows 64-bit access and the tag and WID use 31 bits, two tags can be formed in each shared memory bank. In one embodiment, the bank in which the two tags are located may be a bank different from the bank storing the corresponding data block (e.g., a 128 byte data block). That is, the bank group to which the tag to which the tag belongs may be a bank group different from the bank group to which the bank storing the corresponding data block belongs. In this manner, bank collision can be avoided by placing the data block corresponding to the tag in different bank groups, thereby enabling parallel access of the tag and the data block. Also, by using unused shared memory space as a direct mapped cache, a pair of data blocks and corresponding tags can be accessed with a single shared memory access.

On the other hand, the unused space of the shared memory 1812 may vary depending on the implementation of the kernel. Also, as each CTA requires a different amount of shared memory 1812, the use of shared memory 1812 may vary with the execution of the kernel. The multiprocessor may have a shared memory management table (SMMT) (not shown) to manage the shared memory space. The SMMT includes a plurality of entries, each entry corresponding to a CTA. Each CTA reserves one SMMT entry, and the entry records the start address and size of the shared memory space used by the given CTA. In some embodiments, each entry in the SMMT may correspond to a thread in a warp or a thread in a warp group.

When the CTA is launched, the multiprocessor can check the corresponding SMMT entry to determine the amount of shared memory that the CTA does not use. The multiprocessor can then put the start address and size of the shared memory that it does not use into the SMMT's new entry to free up space for storing cache lines and tags.

As shown in FIG. 19, the multiprocessor may place a hardware address translation unit 1910 in front of the shared memory 1812 to determine where in the shared memory 1812 the target cache line and its tag are.

19, the address translation unit 1910 translates the global memory address into a shared memory address. The global memory address may be decomposed into a tag (T), a block index (L) and a byte offset (F) in LSB to MSB order. For example, the global memory address may be divided into a 16-bit tag T, a 9-bit block index L, and a 7-byte byte offset F. [ The shared memory address may include a cache block address indicating the target cache data block and a tag address indicating the target tag. The cache block address includes four fields, and four fields may correspond to byte offset F, bank index B, bank group index G, and row index R in LSB to MSB order. For example, if the shared memory 1812 uses eight bytes per bank, sixteen banks, two bank groups, and 256 rows per bank group, byte offset F, bank index B, (G) and row index (R) may be 3, 4, 1 and 8 bits, respectively.

In some embodiments, the remaining bits (e.g., 16 bits of the tag T) after translation of the global memory address into a cache block address may be used as part of the tag of the target cache data block. Since the number of cache data blocks required is greater than the number of rows, the tag of the shared memory 1812 may include a warp number (WID) and a cache block index. The warp number (WID) and the cache block index may be, for example, 6 bits and 9 bits, respectively.

The address translation unit 1910 may use a N-bit mask to convert the block index L of the global memory address into a row index R that points to the location of the row in the shared memory 1812. For example, if the shared memory 1812 has 256 rows, the address translation unit 1910 may use an 8-bit mask. In some embodiments, the address translator unit 1910 may generate the row index R by computing an N-bit mask with the upper N bits of the block index L of the global memory address. In one embodiment, the address translation unit 1910 may generate the row index R by XORing the upper N bits of the block index L of the global memory address with an N bit mask. The address translation unit 1910 may convert the lower M bits of the block index L of the global memory address into a bank index B indicating the bank group in the shared memory 1812. [ For example, if the lower 8 bits in the 9-bit block index L are converted to the row index R and the number of the bank groups in the shared memory 1812 is 2, the LSB of the block index L is divided into the bank index B).

The address translation unit 1910 translates some bits of the byte offset F of the global memory address into a bank index B indicating a bank of the shared memory 1812 and a remaining number of bits of the byte offset F as a cache block address To the byte offset (F). In some embodiments, some bits of the byte offset F of the global memory address may be the bank index B and some of the remaining bytes of the byte offset F may be the byte offset F of the cache block address. In one embodiment, the upper L bits of the byte offset F of the global memory address may be the bank index B and the lower K bits of the byte offset F may be the byte offset F of the cache block address . For example, when the shared memory 1812 has 16 banks, the upper 4 bits of the 7-bit byte offset F of the global memory address becomes the bank index B, and the lower 3 bits of the byte offset F May be the byte offset F of the cache block address.

The tag address includes four fields and four fields may correspond to byte offset F, bank index B, bank group index / G, and row index R in LSB to MSB order. For example, if a physical row per bank stores two tags and one row of the bank group holds 32 tags, then the actual position of the tag is 5 bits, that is, 1 byte of byte offset F and 4 May be indicated by the bank index (B) of the bit. For this, the address translation unit 1910 converts the lower P bits of the row index R of the cache block address into the byte offset F of the tag address and the remaining lower Q bits of the row index R to the bank index of the tag address B). In some embodiments, the lower P bits of the row index R of the cache block address may be the byte offset F, and some of the remaining bits of the byte offset F may be the byte offset F of the cache block address . The LSB of the row index R of the cache block address becomes the byte offset F of the tag address and the remaining lower four bits of the row index R become the bank index B of the tag address have.

The address translation unit 1910 may also translate the remaining bits of the row index R of the cache block address into the row index R of the tag address to indicate the row index R of the tag. In some embodiments, the remaining bits of the row index R of the cache block address may be the row index R of the tag address. In the example described above, the upper three bits of the row index R of the cache block address may be the row index R of the tag address. The address translation unit 1910 translates the bank group index G of the cache block address into the bank group index / G of the tag address. In some embodiments, the bank group index G of the cache block address may be flipped to become the bank group index / G of the tag address in order to access the tag corresponding to the cache data block in parallel.

In some embodiments, the address translation unit 1910 may reorder the starting position of the index for the cache block and tag, taking into account the cache offset register and the tag offset register. The cache offset register and the tag offset register can be used to capture the varying size of the cache in unused shared memory space.

An example of an algorithm for implementing a warp scheduling method in a parallel processing unit according to an embodiment of the present invention will be described below.

In Algorithm 1, i indicates the warp number of the warp to be scheduled, and can be obtained, for example, via the getWarpToBeScheduled () function. InstNo is the total number of executed instructions, and can be obtained, for example, by the getNumInstructions () function. ActiveWarpNo indicates the number of active warps running on the multiprocessor, and can be obtained, for example, by the getNumActiveWarp () function.

For the warp i (Warp (i)), if the active flag V is '0' and the low cutoff period has expired, it is determined whether to re-activate warp i in the stalled state. For this purpose, the warp number k of the warp k (Warp (k)) causing the warp i to be stalled is obtained from the second field Pair_List [i] [1] of the pair list entry corresponding to the warp i. In this case, you need to have the level of the warp k cache interference (ISR _k) is higher than the lower cut-off warp k (Warp (k)) continue to run, i-warp stall condition of is maintained. Otherwise, warp i is reactivated. Accordingly, the active flag of the warp i is set to '1', and the second field of the pair list entry Pair_List [i] [1] is cleared (for example, set to '-1').

For the warp i, if the isolation flag I is '1' and the low cutoff period has expired, it is determined whether to withdraw the warp i from the isolated state. For this purpose, the warp number k of the warp k (Warp (k)) causing the partition of the warp i is obtained from the first field Pair_List [i] [0] of the pair list entry corresponding to the warp i. In this case, if the level (ISR _k ) of the cache interference of warp k is higher than the low cutoff and warp k needs to be continuously executed, the isolation state of warp i is still maintained. Otherwise, the quarantine of warp i is withdrawn. Thus, the isolation flag of the warp i is set to '0', and the first field Pair_List [i] [0] of the pair list entry is cleared (for example, set to '-1').

For the warp i, if the active flag V is '1' and the high cutoff period has ended, it is determined whether to partition the warp j causing the interference to the active warp i. For this purpose, the warp number j of the warp j causing the interference with the warp i is obtained from the interference list entry (Interference_List [i]) corresponding to the warp i. If warp j is in an isolated state (i.e., when the isolation flag I of warp j is '1'), the level of cache interference (ISR _i ) of warp i is higher than the high cutoff and warp j is different from warp i, , Warp j is stalled. That is, the active flag V of warp j is set to '0' and the second field Pair_List [j] [1] of the pair list entry corresponding to warp j is set to i. Otherwise, if warp j is active (i.e., if the isolation flag I of warp j is '0'), warp j is isolated. That is, the isolation flag I of warp j is set to '1', and the first field Pair_List [j] [0] of the pair list entry corresponding to warp j is set to i.

[Algorithm 1]

1 i: = getWarpToBeScheduled ()

2 InstNo: = getNumInstructions ()

3 ActiveWarpNo: = getNumActiveWarp ()

4 if Warp (i) .V == 0 and end of low cut-off epoch then

/ * Warp (i) is deactivated * /

5 k: = Pair_List [i] [1]

6 IRS _k : = VTAHit [k] / (InstNo / ActiveWarpNo)

7 if IRS _k > low-cutoff and warp (k) needs executing then

8 continue

9 else

10 Warp (i) .V: = 1

11 Pair_List [i] [1]: = -1 // cleared

12 else if Warp (i) .I == 1 and end of low cut-off epoch then

/ * Warp (i) redirects to access shared memory * /

13 k: = Pair_List [i] [0]

14 IRS _k : = VTAHit [k] / (InstNo / ActiveWarpNo)

15 if IRS _k > low-cutoff and warp (k) needs executing then

16 continue

17 else

18 Warp (i) .I: = 0

19 Pair_List [i] [0]: = -1 // cleared

20 if Warp (i) .V == 1 and end of high cut-off epoch then

/ * Warp (i) is active * /

21 IRS _i : = VTAHit [i] / (InstNo / ActiveWarpNo)

22 j: = Interference_List [i]

23 if IRS _i > high-cutoff and j! = I then

24 if Warp (j) .I == 1 then

25 Warp (j) .V: = 0

26 Pair_List [j] [1]: = i

27 else if Warp (j) .I == 0 then

28 Warp (j) .I: = 1

29 Pair_List [j] [0]: = i

The parallel processing unit according to an embodiment of the present invention is implemented in actual hardware and the measurement results are described.

As shown in Table 1 below, a parallel processing unit including 15 multiprocessors (SM) is used in the performance measurement. Also, up to 1536 threads are created per SM, and two warp schedulers are used. In addition, the L1D cache uses a 128KB cache line, a 16KB cache that supports 4-way, or a 48KB cache that supports a 6-way with a 128-byte cache line.

# of SMs 15 # of threads Max 1536 per SM # of war schedulers 2 per SM L1D cache 16KB w / 128B lines, and 4 ways
48KB w / 128B lines, and 6 ways L1D policy Write no-allocate, local write-back, global write-through, and pseudo LRU L2 cache 768KB w / 128B lines, and 8 ways L2 policy Allocate on miss, write-back, and pseudo LRU DRAM GDDR5 w / 16 banks, tCL = 12, tRCD-12, and tRAS = 28 Victim Tag Array 8 tags per set, 48 sets, and FIFO

In this case, the warp scheduling method using only warp partitioning can improve the performance by 32% as compared with the conventional warp scheduling. Also, the warp scheduling method using warp partitioning and warp throttling can improve performance by 54% compared to the conventional warp scheduling.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (20)

Includes multiprocessor,
The multi-
A memory unit including a L1D (level 1 data) cache and a shared memory to which a plurality of cache lines are allocated, and
A thread group scheduler that changes the cache access of the second thread group to the shared memory and isolates the second thread group when a predetermined first condition is satisfied for the first thread group in an active state,
≪ / RTI > The method of claim 1,
Wherein the first condition comprises a condition that the level of cache interference for the first thread group exceeds a first cutoff. 3. The method of claim 2,
Wherein the second thread group comprises a thread group that most frequently interferes with the thread group causing the first thread group to interfere with the first thread group. 3. The method of claim 2,
And if the predetermined second condition is satisfied, the thread group scheduler changes the cache access of the second thread group back to the L1D cache. 5. The method of claim 4,
Wherein the second condition includes a second cutoff beam or a lower condition where the level of cache interference for the first thread group is lower than the first cutoff. 5. The method of claim 4,
Wherein the second condition comprises a condition that execution of the first thread group is completed. 5. The method of claim 4,
Wherein the thread group scheduler determines the first condition at the end of the first period of time and the second condition at the end of the second period of time in the cycle consisting of the first period and the second period, unit. The method of claim 1,
And the thread group scheduler stall the isolated second thread group if a predetermined second condition is satisfied. 9. The method of claim 8,
Wherein the second condition comprises a condition that a level of cache interference for a third thread group isolated to the shared memory exceeds a first cutoff. The method of claim 9,
And if the predetermined third condition is satisfied, the thread group scheduler reactivates the stalled second thread group. 11. The method of claim 10,
Wherein the third condition comprises a second cutoff beam or a lower condition where the level of cache interference for the third thread group is lower than the first cutoff. 11. The method of claim 10,
Wherein the second condition comprises a condition that execution of the third thread group is completed. 11. The method of claim 10,
Wherein the thread group scheduler determines the second condition at the end of the first time and determines the third condition at the end of the second time in the cycle consisting of the first time and the second time, unit. A parallel processing unit according to claim 1,
A central processing unit (CPU)
System memory, and
A memory bridge connecting the parallel processing unit, the CPU and the system memory,
≪ / RTI > Includes multiprocessor,
The multi-
A memory unit including an L1D (level 1 data) cache and a shared memory, and
A thread group scheduler,
Wherein the shared memory comprises a plurality of shared memory banks having a plurality of rows,
Wherein the plurality of shared memory banks are grouped into a plurality of bank groups,
A plurality of cache lines are allocated to a plurality of shared memory banks belonging to each bank group,
The thread group scheduler changes the cache access of some thread groups to the cache line of the shared memory
Parallel processing unit. 17. The method of claim 16,
Wherein a bank group to which a cache line is assigned is assigned a number of the thread group corresponding to the tag for the certain cache line in the other bank group. 1. A thread group scheduling method of a parallel processing unit comprising a memory unit including an L1D (level 1 data) cache and a shared memory,
Detecting a level of cache interference for the first thread group, and
And when the level of the cache interference exceeds the first cutoff, a cache access of a second thread group satisfying a predetermined condition among the thread groups causing interference to the first thread group is changed to the shared memory, Isolating
/ RTI > The method of claim 17,
Again detecting the level of cache interference for the first thread group, and
And changing the cache access of the second thread group back to the L1D cache if the level of the re-detected cache interference is lower than a second cutoff lower than the first cutoff. The method of claim 17,
Detecting a level of cache interference for a third thread group isolated to the shared memory, and
Stalling the isolated second thread group if the level of cache interference for the third thread group exceeds a second cutoff,
The thread group scheduling method further comprising: 20. The method of claim 19,
Again detecting the level of cache interference for the third thread group, and
And reactivating the stalled second thread group if the level of the re-detected cache interference for the third thread group is lower than the second cutoff beam or the lower third cutoff
The thread group scheduling method further comprising:

Images