KR101946476B1 - Early miss prediction based periodic cache bypassing technique, streaming multiprocessor and embedded system performed by the technique - Google Patents

Abstract

The present invention relates to periodic cache bypassing technique based on cache hit ratio prediction and a streaming multiprocessor and embedded system using the same and, more specifically, to periodic cache bypassing technique capable of improving cache hit ratio while improving performance of an embedded system by decreasing the number of access to L1 data cache through periodic cache bypassing technique based on cache hit ratio prediction and a streaming multiprocessor and embedded system using the same.

Description

[0001] The present invention relates to a periodic cache bypassing method based on cache hit ratio prediction, a streaming multiprocessor and an embedded system employing the technique,

The present invention relates to a cyclic cache bypassing method based on cache hit ratio prediction, a streaming multiprocessor and an embedded system to which the technique is applied, more specifically, a cyclic cache bypassing method based on cache hit ratio prediction, And a streaming multiprocessor and an embedded system to which the technique is applied. [0003] 2. Description of the Related Art [0004]

The cache bypass technique improves the cache hit ratio and reduces the memory stall.

Recently, general-purpose computing on graphics processing units (GPGPU) capable of performing not only graphic operations but also general computing operations have been developed.

FIG. 1 is a diagram illustrating a general multiprocessing graphics processing apparatus according to an embodiment of the present invention. Referring to FIG. 1, a general purpose computing graphics processing apparatus 10 includes a plurality of streaming multiprocessors 11 (Streaming Multiprocessors) 12, an L2 cache 13, a memory controller 14, and a lower memory 15.

FIG. 2 illustrates a general general purpose graphics processing pipeline. First, a warp scheduler determines a warp execution order (S1000), and a fetch unit fetches from a physical memory The instruction is patched (S200) and stored in an instruction cache (IC) (S300).

Next, the decode unit decodes an instruction to determine which instruction is to be executed (S400). The issue unit indicates an instruction to be processed (S500). The instruction is temporarily stored in a register file (S600).

Next, an operation is performed in the ALU according to the type of the instruction stored in the register file (S800), the memory is accessed by the load / store unit and the data is fetched to the core (S700) , And the result of the operation is recorded in the register file (S900).

In step S700, data is stored in an address generator in step S710. The generated address is temporarily stored in a load / store queue. (S720).

Then, if there is data in the data cache of the L1 cache (D-cache), data is fetched to the core (S730). If not, the data is fetched to the core in the L2 cache (S750), the fetched data is stored in the data cache. At this time, the state of the miss of the data is recorded in the Miss Status Holding Registers (MSHRs) (S740).

Each block of the data cache (D-cache) is composed of a tag and data, and the tag includes tag information corresponding to the address.

Such a conventional general-purpose computing graphics processing apparatus can process a process in a wide parallel structure by using streaming multiprocessors, but it can improve the processing performance. However, many execution processes such as cache contention and resource congestion Because of performance challenges, cache efficiency is low. In particular, a pipeline stall can not be avoided in the event of a data miss, because it always accesses the L1 data cache regardless of the block state of the data cache.

[Prior Art Literature]

[Patent Literature]

1. Korean Patent No. 10-1662363, host device and method for accessing a virtual file in a storage device by bypassing cache in the host device

2. Korean Patent No. 10-1612155, a method and structure for using area locks to bypass I / O requests of a storage controller with multiple processing stacks

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a cache bypass technique capable of improving the cache hit ratio and reducing the memory stall to improve the performance of the embedded system, Multi-processor and embedded systems.

According to an aspect of the present invention, there is provided a streaming multiprocessor (SM) in a general-purpose computing graphics processing unit (GPGPU), the streaming multiprocessor including an L1 data cache If there is no desired data in the L1 data cache, accesses the lower level memory to fetch desired data, stores data in the L1 data cache, and stores the tag information of the L1 data cache (EMP unit) for outputting a miss state when the address is present in the tag information, and a miss state when the address is not present in the tag information, Early Miss Prediction unit); And a memory for receiving a hit state or a miss state from the address generated by the address generator and the miss prediction unit, and a cache bypassing period, which is a time for accessing the lower level memory, A periodic cache bypass unit (P unit) for setting a refresh period and allowing the data to be fetched by bypassing the lower level memory when the current time is the cache bypass cycle and the output of the miss prediction unit is miss, : Periodic Cache Bypassing unit). ≪ / RTI >

In a preferred embodiment, the periodic cache bypass unit accesses the L1 data cache to fetch data when the output of the misprediction unit is Hit even if the current time is the cache bypass period.

In a preferred embodiment, the periodic cache bypassing unit accesses the L1 data cache and fetches data irrespective of the output of the miss prediction unit when the current time is the refresh period.

In a preferred embodiment, the tag information of the L1 data cache is copied and stored in the miss prediction unit during the refresh period.

In a preferred embodiment, the streaming multiprocessor comprises: an address generator for generating an address where desired data is stored; And a load store queue for receiving and temporarily storing an address of the address generator and sequentially outputting the address to the periodic cache bypass unit, wherein an address is output from the load store queue to the periodic cache bypass unit The misprediction unit outputs a hit state to the periodic cache bypassing unit when a tag corresponding to an output address exists in the miss prediction unit, and if not, the misprediction unit notifies the missed state .

The invention further provides a general purpose computing graphics processing device comprising the streaming multiprocessors.

Further, the present invention further provides an embedded system in which the general-purpose computing graphics processing apparatus is embedded.

According to another aspect of the present invention, there is provided a cache bypassing method for fetching data from the L1 data cache using the streaming multiprocessor or fetching data from the lower level memory by bypassing the L1 data cache, Determining a period and storing the period in the periodic cache bypass unit; Generating an address in the address generator; The address generated by the address generator is loaded into the load store queue and the miss prediction unit checks whether the tag corresponding to the address is stored and outputs a hit state if it is stored and a miss state if not step; Outputting an address from the load store queue to the periodic cache bypass unit and simultaneously inputting a hit state or a miss state in the miss prediction unit to the periodic cache bypass unit; And if the current time is a cache bypass cycle and the output of the missprediction unit is in a miss state, accessing to the lower level memory and bypassing the L1 data cache to fetch data The cache bypassing method further includes:

In a preferred embodiment, the periodic cache bypassing unit accesses the L1 data cache to fetch data when the output from the mispredictor unit is in a Hit state even if the current time is the cache bypass period, When the time is the refresh period, the data is accessed by accessing the L1 data cache regardless of the output of the miss prediction unit.

The present invention has the following excellent effects.

First, according to the cache bypassing technique of the present invention, a streaming multiprocessor and an embedded system to which the technique is applied, the L1 data cache is not accessed every cycle, the current time is the cache bypass period, and the block state information of the miss prediction unit is miss It is possible to improve the cache hit ratio and to improve the performance of the embedded system by reducing access to the L1 data cache.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing the structure of a general purpose computing graphic processing apparatus; Fig.
2 is a diagram for explaining a pipeline of a general purpose general purpose computing graphics processing apparatus,
FIG. 3 is a diagram for explaining a cache bypassing technique according to an embodiment of the present invention;
4 is a graph showing the number of processing instructions per clock when applying the cache bypass technique according to an embodiment of the present invention,
FIG. 5 is a graph showing a miss ratio of the L1 data cache when applying the cache bypass method according to an exemplary embodiment of the present invention.

Although the terms used in the present invention have been selected as general terms that are widely used at present, there are some terms selected arbitrarily by the applicant in a specific case. In this case, the meaning described or used in the detailed description part of the invention The meaning must be grasped.

Hereinafter, the technical structure of the present invention will be described in detail with reference to preferred embodiments shown in the accompanying drawings.

However, the present invention is not limited to the embodiments described herein but may be embodied in other forms. Like reference numerals designate like elements throughout the specification.

FIG. 3 is a view for explaining a cache bypass technique according to an embodiment of the present invention. In the cache bypass technique of the present invention, when fetching data from a data cache to a core of a streaming multiprocessor, Thereby improving the cache hit ratio and reducing the memory stall.

In addition, the cache bypass technique of the present invention is performed in a streaming multiprocessor (SM).

That is, the present invention can be provided in the form of a streaming multiprocessor which is an apparatus for fetching data using the cache bypass technique of the present invention.

In addition, the present invention may be provided in a general purpose computing graphics processing unit (GPGPU) including a plurality of the streaming multiprocessors, and the general purpose graphics processing unit may be embedded in an embedded system or in the form of an embedded system.

Referring to FIG. 3, in the cache bypassing method of the present invention, only the process of fetching data (S700 'in FIG. 2) is different from the pipeline shown in FIG. 2, and the remaining steps are substantially the same.

In order to perform the cache bypassing method of the present invention, an existing EMP unit (Early Miss Prediction unit, a kind of 'memory') and a periodic cache bypass unit (P unit) unit, for example, a combination of 'multiplier and multiplexer'), which may be substantially new to existing streaming multiprocessors, or may be implemented by programmatically reconfiguring existing hardware It can also be added.

In addition, the tags of the L1 data cache are copied and stored in the miss prediction unit, and the tags include tag information corresponding to the block address of the L1 data cache.

In addition, the periodic cache bypass unit is provided between the load store queue and the L1 data cache, and determines whether to fetch data by accessing the L1 data cache or access the lower level memory (L2 cache) to fetch data.

Hereinafter, the cache bypassing method (S1000) according to the present invention will be described in detail.

First, an address generator of the streaming multiprocessor generates and outputs an address to fetch data (S1100).

The periodic cache bypass unit is provided with a cache bypass period ('a'), which is a period for giving a chance to bypass the lower level memory, and a refresh period ('b'), which is a period for accessing the L1 data cache, (S1010).

In addition, the cache bypass cycle and the refresh cycle are repeated and form one cycle T.

The ratio of the duration (0 to t1) of the cache bypass cycle and the duration (t1 to t2) of the refresh cycle can be determined experimentally by the designer. In the present invention, the cycle 'T' and the cache bypass cycle ' a 'was set to 6: 5.

Next, the address output from the address generator is temporarily stored in a load store queue (S1200) and output.

At the same time, the tag information corresponding to the address output from the address generator is retrieved from the miss prediction unit, and block state information (M-hint) indicating whether the address exists in the tag information is outputted S1300).

In addition, the status information is output in a hit state when the address exists in the status tag, and is output in a miss status if not.

The address output of the load store queue and the block state information output of the miss prediction unit are performed in the same cycle.

Next, the address output from the load store queue and the block state information output from the miss prediction unit are input to the periodic cache bypassing unit. The periodic cache bypassing unit accesses the L1 data cache according to the block state information In operation S1400, it is determined whether to fetch data or bypass data without accessing the L1 data cache to fetch data from the lower level memory immediately.

In addition, if the current time (or current clock cycle) is the cache bypass cycle and the block state information is miss, the periodic cache bypass unit detaches data from the lower level memory immediately (S1700).

Also, the fetched data is loaded into a register file and provided to the core, and the data is accessed by accessing the L1 data cache only when the data is changed.

However, if the block state information is hit even if the current time is in the cache bypass cycle, data is fetched from the L1 data cache to the core because there is desired data in the L1 data cache (S1500).

In addition, when the current time is in the refresh period, the data is accessed by accessing the L1 data cache regardless of the block state information.

In addition, a miss status of data fetch is recorded in the Miss Status Holding Registers (MSHRs) (S1600).

When the data is fetched from the L1 data cache or the data is changed in the refresh period, the tag of the L1 data cache is copied to the miss prediction unit and stored.

In other words, when the data is fetched from the L1 data cache in the cache bypass cycle, it is not copied to the miss prediction unit.

That is, the cache bypassing technique according to an embodiment of the present invention does not access the L1 data cache every cycle, and only when the current time is the refresh cycle or the cache bypass cycle, the access is performed only when the block status information is hit, The performance of the embedded system can be improved while the cache hit ratio can be improved.

FIG. 4 is a graph showing the number of instructions per clock (IPC) when applying the cache bypass technique according to an embodiment of the present invention (PEMP-16 KB is a graph showing the case where the capacity of the miss prediction unit is 16 KB And the capacity of the miss prediction unit is 32 KB), when applying the cache bypass technique of the present invention over the conventional general pipeline, the number of processing instructions per clock is reduced in all benchmark programs The same or improved.

FIG. 5 is a graph showing a miss ratio and an access count of the L1 data cache when the cache bypass method according to an embodiment of the present invention is applied.

Referring to FIG. 5, when the normalized L1 miss rate of a conventional general pipeline is '1', the cache bypass technique of the present invention is applied to all benchmark programs except for the LUD program. When applied, you can see that the Miss Rate is the same or less.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, Various changes and modifications will be possible.

10: general purpose computing graphics processing device 11: streaming multiprocessor
12: Internal network 13: L2 cache
14: Memory controller 15: Lower memory

Claims (9)

As a streaming multiprocessor (SM) in a general-purpose computing graphics processing unit (GPGPU)
The streaming multiprocessor fetches the data stored in the L1 data cache in the core and accesses the lower level memory to retrieve the desired data if the L1 data cache does not have the desired data, / RTI >
The tag information of the L1 data cache is copied and stored, and when an address generated by the address generator is received, if the address exists in the tag information, a hit state is output. Otherwise, a miss state is output An EMP unit (Early Miss Prediction unit); And
A cache bypassing period, which is a time for accessing the lower level memory, and a refresh period, which is a time for accessing the L1 data cache, from the address generated by the address generator and the hit state or the miss state from the miss prediction unit, And a periodic cache bypass unit (P unit) for setting a refresh period when the current time is the cache bypass cycle and an output of the miss prediction unit is missed, A Periodic Cache Bypassing unit).
The method according to claim 1,
Wherein the periodic cache bypass unit accesses the L1 data cache to fetch data when the output of the miss prediction unit is hit even if the current time is the cache bypass period.
3. The method of claim 2,
Wherein the periodic cache bypass unit accesses the L1 data cache and fetches data irrespective of the output of the miss prediction unit when the current time is the refresh period.
The method of claim 3,
And the tag information of the L1 data cache is copied and stored in the miss prediction unit during the refresh period.
5. The method according to any one of claims 1 to 4,
The streaming multiprocessor
An address generator for generating an address where desired data is stored; And
And a load store queue for receiving an address of the address generator, temporarily storing the address, and sequentially outputting the address to the periodic cache bypass unit,
When an address is output from the load store queue to the periodic cache bypass unit, if the tag corresponding to the output address exists in the miss prediction unit, the miss prediction unit outputs the hit state to the periodic cache bypass unit, And the miss prediction unit outputs a miss status to the periodic cache bypass unit if the miss prediction unit fails.
5. A general purpose computing graphics processing device comprising streaming multiprocessors as claimed in any one of claims 1 to 4.
An embedded system in which the general purpose computing graphics processing apparatus of claim 6 is embedded.
A cache bypassing method for fetching data from the L1 data cache using the streaming multiprocessor of claim 5 or fetching data from the lower level memory by bypassing the L1 data cache,
Determining the cache bypass cycle and the refresh cycle, and storing the cache bypass cycle and the refresh cycle in the periodic cache bypass unit;
Generating an address in the address generator;
The address generated by the address generator is loaded into the load store queue and the miss prediction unit checks whether the tag corresponding to the address is stored and outputs a hit state if it is stored and a miss state if not step;
Outputting an address from the load store queue to the periodic cache bypass unit and simultaneously inputting a hit state or a miss state in the miss prediction unit to the periodic cache bypass unit; And
If the current time is a cache bypass cycle and the output of the miss prediction unit is in a miss state, accessing the lower level memory by bypassing access to the L1 data cache and fetching data; Wherein the cache bypassing method comprises:
9. The method of claim 8,
Wherein the periodic cache bypassing unit accesses the L1 data cache and fetches data when the output from the missprediction unit is in a hit state even if the current time is the cache bypass period, The data access is made by accessing the L1 data cache regardless of the output of the miss prediction unit.

Images