GB2214669A

GB2214669A - Cache memory

Info

Publication number: GB2214669A
Application number: GB8825243A
Authority: GB
Inventors: David Paul Crane
Original assignee: Fujitsu Services Ltd
Current assignee: Fujitsu Services Ltd
Priority date: 1988-01-30
Filing date: 1988-10-28
Publication date: 1989-09-06
Anticipated expiration: 2008-10-28
Also published as: GB8825243D0; GB2214669B; GB8802102D0

Abstract

A cache memory has a RAM 23 for holding flags pointing to the most recently used data item in each of a plurality of sets. When a data item is loaded into the cache, the item to be replaced is selected by excluding the most recently used item in the set in question, and selecting one of the remaining items on a non-usage basis, for example a random basis. This not-most-recently-used (NMRU) replacement algorithm requires fewer storage bits than a conventional least-recently- used (LRU) replacement algorithm, while giving better performance than a purely non-usage based replacement algorithm. <IMAGE>

Description

CACHE MEMORY.

Background to the invention.

This invention relates to cache memories.

In a data processing system, it is known to include a cache memory between the processing unit and the main memory. The cache is of smaller size than the main memory, but has a faster access speed, and serves to match the speed requirements of the processing unit to the relatively slow access speed of the main memory.

Caches are described for example in "Cache Memories" by A.J. Smith, ACM Computing surveys, September 1982, page 473.

Whenever the processing unit requires to access a data item, the cache is inspected to determine whether that item is currently resident in the cache. If so, the item can be accessed immediately. If not, then the item must be fetched from the main memory and loaded into the cache, replacing one of the existing data items in the cache.

In general a given data item may be held in any of a number of different locations in the cache. For example, in a fully associative cache, a data item may be held anywhere in the cache, whereas in a set-associative cache, a data item may be held in any of a particular set of locations. When a new data item is to be loaded into the cache, it is necessary to decide which of the possible locations is to be replaced.

This decision is referred to as the cache replacement algorithm.

Known cache replacement algorithms are divided into usage-based and non-usage based. Usage-based algorithmns take account of the history of usage of the data items in the cache so as to select, for example, the least recently used item for replacement. In general, usage-based algorithms result in a significantly lower cache miss rate, and hence a higher performance, than non-usage based algoritmns. However, they do this at a significantly higher cost in terms of the hardware required to implement the replacement algorithm.

The object of the present invention is to provide a cache store with a replacement algorithm which provides better performance than conventional non-usage-based systems, at a lower cost than usage-based systems.

Summary of the invention.

According to the invention, there is provided a cache memory holding a plurality of data items, and including: (a) indication means for indicating the most recently used data item in said plurality of data items, and (b) replacement means responsive to said indication means, for selecting one of said data items, excluding the most recently used data item, on a non-usage basis.

It can be seen that the invention gives a better performance than conventional non-usage based algorithms, since it ensures that the most recently used item is not selected for replacement. This is consistent with the cache philosophy that information used recently is likely to be used again in the near future. On the other hand, the invention requires less hardware than a least-recently used algorithm, since substantially fewer bits are required to remember the most recently used location than to store the usage information for identifying the least-recently used location.

Brief description of the drawings.

One cache memory in accordance with the invention will now be described by way of example with reference to the accompanying drawings.

Figure 1 is a block diagram of a computer system incorporating a cache memory.

Figure 2 is a more detailed block diagram of the cache memory.

Figure 3 is a more detailed block diagram of a replacement selection circuit within the cache memory.

Description of an embodiment of the invention.

Referring to Figure 1, the computer system comprises a central processing unit 10, a main memory 11, and a cache memory 12. The cache memory is of smaller size than the main memory, and has a substantially faster access speed.

Referring to Figure 2, the cache 12 comprises a data random-access memory (RAM) 20 and a tag RAM 21. As shown, each of these RAMs consists of four sections.

The cache receives an input address ADDR from the processing unit. This address comprises two portions, referred to as INDEX and the TAG. The INDEX portion of the address is applied to the address inputs of both the data RAM and the tag RAM, so as to select a set of data items and their corresponding four tags.

The tag RAM 21 includes comparators (not shown) which compare the TAG portion of the address with the selected set of four tags. If any of the tags matches the TAG portion, a MATCH signal is produced on one of four output lines 22. This enables the corresponding one of the four sections of the data RAM 20, so as to allow the corresponding data item to be accessed for reading or writing, as required.

As described so far, the cache memory is a conventional four-way set-associative cache.

This cache also includes a further RAM 23, referred to herein as the most recently used (MRU) RAM.

This holds a 2-bit flag for each set of data items in the data RAM, indicating which of the four items in the set has been used (i.e written or read) most recently.

The MRU RAM is addressed by the INDEX portion of the address, so that whenever one of the sets of data items is selected, the corresponding MRU flag is accessed.

The four MATCH signals from the tag RAM 21 are combined in an OR gate 24, to produce a HIT signal if any one of the MATCH signals is true. The HIT signal therefore indicates that the required data item, as specified by the input address ADDR, is available in the cache. The OR gate 24 also produces a MISS signal, which is the inverse of HIT.

The four MATCH signals are also applied to a 4:2 encoder 25, which produces a 2-bit code indicating which of the four sections of the data RAM contains the required data item. This code is fed to the data input of the MRU RAM 23. The HIT signal is fed to the write enable input of the RAM 23, so that, when HIT is true, the 2-bit code is written into the currently addressed location of the RAM23.

Thus, it can be seen that whenever the required data item is found to be in the cache, the MRU RAM 23 is updated, to indicate that this data item is the most recently used item of the set in question.

On a miss, the MRU flag from the RAM 23 is applied to a replacement selection circuit 26, which generates a 2-bit signal NMRU (not most recently used) which points to one of the data items in the currently addressed set, other than the most recently used item.

The NMRU signal is fed to a 2:4 decoder 27 which, when enabled, produces one of four write enable signals WEO-WE3. These signals are applied to the write enable inputs of the four sections of the data RAM 20, and also of the tag RAM 21.

Whenever a required data item is not present in the cache, the MISS signal is produced. This enables conventional logic (not shown) for fetching the required data item from the main memory for the cache. When the data is ready, this logic produces a signal DATA READY.

The signal DATA READY and MISS together enable the decoder 27, which in turn causes one of the four sections of the data RAM 20 to be enabled for writing.

This causes the data item from the main memory to be written into one of the four locations of the currently addressed set, replacing the previous data item. The data item to be replaced is selected by the signal NMRU, and is therefore not the most-recently used item.

Referring now to Figure 3, this shows the replacement selection circuit 26 in more detail.

This circuit includes a free-running 2-bit counter 30 which produces an output count value which continuously cycles through the values 0-3. The count value is compared with the signal MRU in a comparator 31.

The replacement selection circuit also includes a multiplexer 32, one input of which receives the count value from the counter 30, and the other input of which receives the inverse of this count value. The multiplexer 32 is controlled by the output of the comparator 31, such that, when the count is not equal to MRU, the multiplexer selects the count value, and when the count value is equal to MRU, the inverse of the count value is selected.

The output of the multiplexer 32 provides the signal NMRU which selects the data item to be replaced.

It can be seen that the signal NMRU is never equal to MRU, and is selected at random from one of the three count values not equal to MRU. For example, if MRU equals 2, then NMRU is selected at random from the values 0, 1, and 3.

In an alternative form of the invention, the circuit of Figure 3 may be replaced by a circuit which simply inverts one of the bits of MRU to produce NMRU.

It can be seen that, in general, if there are N data items in each set in the cache, the NMRU replacement algorithm described above requires log2N bits of storage per set. By way of comparison, the conventional LRU (least recently used) replacement algorithm typically requires N(N-1)/2 bits of storage per set. For example, if N=4, then the NMRU algorithm requires 2 bits/set, while the LRU algorithm requires 6 bits/set. Thus, the NMRU algorithm leads to a substantial saving in hardware.

It will be appreciated that while the invention has been described above in the context of a set-associative cache, the invention is equally applicable to a fully-associative cache organisation. In that case, the MRU flag would indicate the most recently used item in the cache as a whole.

Claims

CLAIMS.

1. A cache memory holding a plurality of data items, and comprising: (a) indication means for indicating the most recently used data item in said plurality of data items, and (b) replacement means responsive to said indication means, for selecting one of said data items, excluding the most recently used data item, on a non-usage basis.

2. A cache memory according to Claim 1 wherein said replacement means comprises: (a) means for making a selection of one of the data items on a non-usage basis, and (b) means for modifying said selection in the event that the most recently used data item is selected.

3. A cache memory according to Claim 2 wherein said means for making a selection on a non-usage basis comprises means for selecting one of the data items at random.

4. A cache memory according to Claim 3 wherein said means for selecting one of the data items at random comprises a free-running counter.

5. A cache memory comprising: (a) a plurality of sets of data storage locations, each set holding a plurality of items, (b) indication means for indicating the most recently used data item in each of said sets, and (c) replacement means, responsive to the indication means, for selecting one of the data items from one of said sets, excluding the most recently used data item in that set, on a non-usage basis.

6. A cache memory according to Claim 5 wherein said indication means comprises a random-access memory having a plurality of individually addressable locations, one for each set, each location holding a flag indicating the most recently used data item in the corresponding set.

7. A data processing system comprising: (A) a data processing unit, (B) a main memory, and (C) a cache memory connected between the data processing unit and the main memory, the cache memory comprising (a) indication means for indicating the most recently used data item in said plurality of data items, and (b) replacement means responsive to said indication means, for selecting one of said data items, excluding the most recently used data item, on a non-usage basis.

8. A cache memory substantially as hereinbefore described with reference to the accompanying drawings.

9. A data processing system substantially as hereinbefore described with reference to the accompanying drawings.