GB2454808A

GB2454808A - Cache which prefetches data at a second address when an access matches a first address

Info

Publication number: GB2454808A
Application number: GB0821078A
Authority: GB
Inventors: Andrew Michael Jones; Stuart Ryan
Original assignee: STMicroelectronics Research and Development Ltd
Current assignee: STMicroelectronics Research and Development Ltd
Priority date: 2007-11-19
Filing date: 2008-11-18
Publication date: 2009-05-20
Anticipated expiration: 2028-11-18
Also published as: GB0821078D0; GB2454808A8; GB2454808B

Abstract

A cache memory stores a first address 35 and a second address 37. When a comparator 39 detects that the address 49 of an access to the system memory has a predefined relationship with the first address, then data defined by the second address is retrieved and stored in the cache. The data may be retrieved if the access address matches the first address, if some bits of the addresses match or if the access address is within a given range from the first address. The second address may be stored as an offset from the first address.

Description

Cache Memory System The present invention relates to systems comprising cache memories, and in particular to systems employing data pre-fetching.

A very large number of systems involve the retrieval of data from a system memory by a device such as a processor. Many of these systems employ a technique known as data caching which exploits a property of data access known as temporal locality. Temporal locality means data that has been accessed recently is the data most likely to be accessed again in the near future. Data caching involves storing, or caching, a copy of recently accessed data in a cache memory that is accessible more quickly and efficiently than the system memory. If the same data is requested again in the future, the cached copy of the data can be retrieved from the cache memory rather than retrieving the original data from the system memory. As the cache memory can be accessed more quickly than the system memory, this scheme generally increases the overall speed of data retrieval.

To implement caching techniques, processor circuitry typically includes an internal cache memory which is located physically closer to the CPU than the system memory, so can be accessed more quickly than the system memory.

When the processor requests data from the system memory a copy of the retrieved data is stored in the cache memory, if it is not stored there already.

Some systems provide two or more caches arranged between the CPU and the system memory in a hierarchical structure. Caches further up the hierarchy are typically smaller in size, but can be accessed more quickly by the CPU than caches lower down the hierarchy. Caches within such a structure are usually referred to as level 1 (Li), level 2 (L2), level 3 (L3), caches with the Li cache usually being the smallest and fastest.

A typical cache memory comprises a series of cache lines, each storing a predetermined sized portion of data. For example, a typical cache memory is divided into 1024 cache lines, each 32 bytes in size, giving a total capacity of 32kB. Data is usually cached in portions equal to the size of a whole number of cache lines When an item of data smaller than a cache line is cached, a block of data equal to the size of one or more cache lines containing the data item is cached. For example, the data item may be located at the beginning of the cache line sized portion of data, at the end or somewhere in the middle.

Such an approach can improve the efficiency of data accesses exploiting a principle known as spatial locality. The principle of spatial locality means that addresses referenced by programs in a short space of time are likely to span a relatively small portion of the entire address space. By caching one or more entire cache lines, not only is the requested data item cached, but also data located nearby, which, by the principle of spatial locality is more likely to be required in the near future than other data.

Each cache line of the cache memory is associated with address information, known as tags, identifying the region of the system memory from which the data stored in each cache line was retrieved. For example, the tag associated with a particular cache line may comprise the address of the system memory from which the cache line sized portion of data stored in that cache line was retrieved. The cache lines may be stored in a data memory portion of the cache, while the tags may be stored in a tag memory portion of the cache.

When a processor requests data from the system memory, the address of the requested data is first compared to the address information in the tag memory to determine whether a copy of the requested data is already located in the cache as the result of a previous data access. If so, a cache hit occurs and the copy of the data is retrieved from the cache, If not, a cache miss occurs, in which case the data is retrieved from the system memory. In addition, a copy of the retrieved data may be stored in the cache in one or more selected cache lines and the associated tags updated accordingly. In a system comprising a cache hierarchy, when data is requested from the system memory, the highest level cache is first checked to determine if a copy of the data is located there. If not, then the next highest level cache is checked, and so on, until the lowest level cache has been checked. If the data is not located in any of the caches then the data is retrieved from the system memory. A copy of the retrieved data may be stored in any of the caches in the hierarchy.

When applying caching techniques, it is important to ensure that the data stored in a cache represents a true copy of the corresponding data stored in the system memory. This requirement may be referred to as maintaining coherency between the data stored in the system memory and the data stored in the cache. Data coherency may be destroyed, for example, if data in one of the system memory and cache is modified or replaced without modifying or replacing the corresponding data in the other. For example, when the processor wishes to modify data, a copy of which is stored in the cache, the processor will typically modify the cached copy without modifying the original data stored in the system memory. This is because it is the cached copy of the data that the processor would retrieve in future accesses and so, for efficiency reasons, the original data stored in the system memory is not modified. However, without taking steps to maintain coherency, any other devices which access the data from the system memory would access the unmodified, and therefore out of date, data.

Various techniques may be applied to maintain data coherency in cache memory systems. For example, one process, referred to as write-back or copy-back, involves writing or copying data stored in one or more cache lines back to the region of system memory from which the cache lines were originally retrieved (as specified in the address information). This process may be performed in a variety of circumstances. For example, when data stored in a cache line has been modified, the cache line may be copied back to the system memory to ensure that the data stored in the cache line and the corresponding data in the system memory are identical. In another example, when data is copied into the cache as a result of a cache miss, an existing cache line of data may need to be removed to make space for the new entry.

This process is known as eviction and the cache line of data that needs to be removed is known as the victim. If the victim comprises modified data, then the victim would need to be written back to the system memory to ensure that the modifications made to the data are not lost when the victim is deleted from the cache.

In some systems, special data coherency routines implemented in software are executed to maintain data coherency. Such routines may periodically sweep the cache to ensure that data coherency is maintained, or may act only when specifically required, for example when data is modified or replaced.

These routines may include write-back or copy-back processes.

Some systems employ a technique known as data pre-fetching in which data may be retrieved, possibly speculatively, before it is actually needed in order to increase the overall speed of memory access. Data pre-fetches may be speculative in the sense that the pre-fetched data may not eventually be required. In one example of data pre-fetching, when executing a code loop in which an item of data needs to be retrieved within each iteration of the loop, the data required for a particular iteration may be pre-fetched during the preceding iteration. In this way, at the point the data is actually required, it does not need to be retrieved at that time. In another example, in highly integrated multimedia systems, very large quantities of data are manipulated, typically in a linear fashion, in a technique known as data streaming. In such applications, the future access patterns of data may be known some time in advance. In this case, data required in the future may be pre-fetched so that it is immediately available when eventually required.

Typically, pre-fetched data is stored in a cache and treated as cached data. In this way, when the pre-fetched data is actually requested, the cache will be checked to determine whether the requested data is located there. Due to the earlier data pre-fetch, a copy of the data can be retrieved from the cache, rather than accessing the system memory. Pre-fetching data into a cache is useful even in applications involving data accesses where the property of temporal locality do not apply. For example, in data streaming applications, data may only be used a single time, so temporal locality does not apply in this case. However, for the reasons given above caching pre-fetched data is advantageous.

Many processor architectures provide special pre-fetch instructions which allow software to cause data to be pre-fetched into a cache in advance of its use. Examples of such instructions include pre-fetch, preload or touch instructions. In such cases a cache normally communicate via a special interface which allows the cache to perform actions when a special instruction is executed by the processor. Data may be pre-fetched into any cache present in a cache hierarchy, such as a level 1 cache or level 2 cache In some systems, pre-fetching data into a level 2 cache may be performed as a consequence of issuing a request to pre-fetch data into the level 1 cache.

A limiting factor in the performance of many systems is the delay between a CPU requesting data from memory and the data actually being supplied to it.

This delay is known as memory latency. For example, the memory latency of highly integrated systems is typically 10-100 times the duration of the execution of a single instruction by the CPU. With the continuing development of processors, CPU clock rates are increasing rapidly, resulting in increasing demand for higher rates of data access. Even with improvements in the speed of memory access, the effects of memory latency are becoming more significant as a result.

There is a need, therefore, for a system and method for pre-fetching data which is as fast and efficient as possible. One problem with existing systems is that it may be difficult to ensure that data is pre-fetched sufficiently in advance so that it is available immediately when needed. Often, it is known in advance that data will required, and in these situations a pre-fetch can be initiated, for example using special pre-fetch instructions. However, the execution of such instructions may take a significant period of time to complete so that the data may not be available by the time it is needed.

Furthermore, modifying code to include all necessary pre-fetch instructions may be difficult. It may also be difficult to identify all occasions when pre-fetching will be necessary, particularly in a dynamic system in which the patterns of data access may not be consistent.

The present invention solves these and other problems associated with existing techniques.

According to a first aspect, the present invention provides a cache memory system for caching data comprising: a cache memory for storing a copy of a portion of data stored in a system memory; and a cache load circuit capable of retrieving the portion of data from the system memory and of storing a copy of the retrieved portion of data in the cache memory; wherein the system further comprises: means for monitoring accesses to data in the system memory; a first memory for storing a first value defining a first memory address; a comparator for comparing the access address of an access to data in the system memory with the first memory address; and a second memory for storing a second value defining a second memory address; the system being arranged such that, if a relationship between the access address and the first memory address is satisfied, the cache load circuit retrieves the portion of data stored in the system memory at the second memory address defined by the second value, and stores the retrieved portion of data in the cache memory; wherein the first and second memory addresses correspond to different physical memory addresses of the system memory.

According to a second aspect, the present invention provides a method for pre-fetching data into a cache memory system, the method comprising the steps of: retrieving a portion of data from a system memory; and storing a copy of the retrieved portion of data in a cache memory; wherein the method comprises the further steps of: monitoring accesses to data in the system memory; comparing the access address of an access to data in the system memory with a first memory address; and if a relationship between the access address and the first memory address is satisfied, retrieving the portion of data stored in the system memory at a second memory address, and storing the retrieved portion of data in the cache memory; wherein the first and second memory addresses correspond to different physical memory addresses of the system memory.

Preferred embodiments of the invention will now be described, with reference to the Figures, in which: Figure 1 is a schematic diagram of a cache memory system in a first embodiment of the invention; Figure 2 is a schematic diagram of a system comprising the cache shown in Figure 1; Figure 3 is a schematic diagram of the monitoring circuit comprised in the system illustrated in Figure 1; Figure 4 shows a system topology comprising a level 2 cache; Figure 5 shows the internal structure of a level 2 cache; Figure 6 shows a flow diagram for a pre-fetch procedure; Figure 7 shows the fields of a 32-bit physical address and how they are interpreted by the L2 cache lookup logic; Figure 8 shows internal buffering and logic for a level 2 cache; and Figure 9 shows the internal structure of a level 2 cache for a further embodiment.

Figure 1 is a schematic diagram of an exemplary cache memory system embodying the invention. The system, referred to below simply as cache 1, comprises a data memory 3 for storing one or more cache lines 5 of data and a tag memory 7 for storing address information in the form of a series of tags 9. For each cache line 5 in the data memory 3, there is a corresponding tag 9 in the tag memory 7. The cache 1 also comprises a cache load circuit 19 used to store data in the data memory 3. It is understood that the invention may be used in a variety of cache systems and is not limited to the arrangement illustrated in Figure 1.

Figure 2 illustrates a system 100 comprising the cache 1 shown in Figure 1.

As shown in Figure 2, in this embodiment, the cache 1 is a level 2 cache functionally located between a processor 101 comprising a level 1 cache 103 and a system memory 105. However, it is understood that the cache shown in Figure 1 may be used as any level of cache, in any cache hierarchy arrangement or as a sole cache. The term system memory may refer to a specific memory device or to a group of two or more memory devices. In general the system memory represents a general memory space formed from the whole, or part of, the individual memory spaces of one or more memory devices. The processor 101 directly accesses the level 1 cache 103. The level I cache 103 communicates with the level 2 cache 1 via bus lines 11, 15 and and the level 2 cache 1 communicates with the system memory 105 via bus line 29. The system 100 also comprises other modules, including a module 107 having DMA (Direct Memory Access) capability. The module 107 accesses the level 2 cache 1 via bus line 109. Other parts of the system (not shown) may also access the level 2 cache 1 via further bus lines (not shown), which may be separate from or integrated with bus line 109.

With reference to Figure 2, when the processor 101 issues a request for retrieval of data stored in the system memory 105 the following process occurs. First, the data access request is transmitted to the level 1 cache 103 which determines whether it stores a copy of the requested data. If so then the copy of the requested data is retrieved from the level 1 cache 103 and provided to the processor 101. In this case, no data retrieval involving the level 2 cache 1 or the system memory 105 is made, If the level 1 cache 103 does not store a copy of the requested data then the data access request is forwarded from the level 1 cache 103 to the level 2 cache 1. In this case, the level 2 cache 1 determines whether it stores a copy of the requested data. If so then the copy of the requested data is retrieved from the level 2 cache 1 and provided to the level 1 cache 103, which in turn provides the data to the processor 101. If the level 2 cache 1 does not store a copy of the requested data then the data is retrieved from the system memory 105. In this case, the level 2 cache 1 requests the data from the system memory 105 and provides the retrieved data to the level I cache 103, which in turn provides it to the processor 101.

With reference to Figure 1, the level 2 cache 1 performs the following process when a data access request is received by it. First, a determination is made as to whether a copy of the data specified in the data access request is already present in the data memory 3 of the cache 1. The data access request identifies the address of the system memory 105 at which the requested data is located. The address of the requested data is supplied to the tag memory 7 via line 11 and compared to the tags 9 stored in the tag memory 7. Each tag 9 comprises an address of the system memory 105 from which a corresponding cache line 5 of data was originally retrieved, If the address of the data presently being requested matches an address specified by a tag 9, this indicates that the data memory 3 does contain a copy of the requested data. A match is indicated by asserting a hit signal on line 13, which is received by the data memory 3 and the cache load circuit 19. When the hit signal is asserted, the cache line 5 of data corresponding to the tag 9 causing the hit is retrieved from the data memory 3 and output from the data memory 3 and cache 1 on line 15.

If no match is found between the address of the requested data and any of the tags 9 in the tag memory, the hit signal is not asserted. In this case the requested data is retrieved from the system memory 105 using the cache load circuit 19 in the manner described below. A copy of the data retrieved from the system memory 105 by the cache load circuit is stored in the data memory 3.

The data is then output from the data memory 3 and cache 1 on line 15.

The cache load circuit 19 comprises a memory 21 which stores a queue of pending cache load operations. Each cache load operation represents an item of data to be retrieved from the system memory 105 and includes the memory address of the data item. A cache load operation may also contain other relevant information, such as whether the data is required as the result of a pre-fetch or some other type of data access. The address received on line 11 is provided to the cache load circuit 19 via line 17. As mentioned above, the cache load circuit 19 also receives the hit signal via line 13. When the hit signal on line 13 is not asserted, the cache load circuit 19 adds a cache load operation to the queue stored in the memory 21 based on the address received on line 17. The cache load circuit 19 processes each cache load operation in turn, for example in the order in which they were added to the queue. A newly added cache load operation will eventually be processed by the cache load circuit resulting in the data being retrieved from the system memory 105, stored in the data memory 3 and output from the cache 1.

To process a cache load operation, the cache load circuit identifies the address of the data to be cached and issues a suitable data access request on line 29 which is received by the system memory 105. When the requested data is provided back to the cache load circuit, the cache load circuit identifies one or more suitable cache tines in the data memory in which to store the received data. These may comprise currently vacant cache lines. However, if there are insufficient free cache lines, it may be necessary to remove one or more existing cache lines of data to make room for the new data, in which case the write-back process described above may be required. The cache load circuit then transmits a load command to the data memory via line 31 comprising a copy of data to be cached, the system memory address from which the data was retrieved and the cache lines identified to store the data.

The copy of the data is then stored in the cache lines specified in the load command and corresponding tags are added to the tag memory based on the address information specified in the load command.

A technique by which the embodiment illustrated in Figure 1 implements pre-fetching of data into the cache will now be described. In many systems, when a specific address or region of memory is accessed, this is highly indicative that data access involving certain other addresses or regions of memory will be made shortly afterwards. Such data accesses may include both read and write accesses. For example, in some applications, data is input from a first buffer and computation then applied to that data. The resulting data is then output to a different buffer. Since this task is typically carried out repetitively, a write to the destination buffer can be used as an indication of a subsequent access to the source buffer. Data reads, as well as data writes, from a specific region of memory may also be indicative of that data access involving another region of memory will occur in the near future.

Accordingly, In order to improve data access system performance, when a data access involving a first location or region is made, data from a second location or region is automatically pre-fetched into the cache. In this way, if data from the second location is required later, it will be already be available from the cache as a result of the pre-fetch. This avoids the need to access the data from system memory at the point it is actually required thereby reducing memory latency. The memory address or region which triggers a pie-fetch of data may or may not be one from which data has been cached.

As shown in Figure 1, the cache 1 comprises monitoring means, which in this embodiment is in the form of a monitoring circuit 33, arranged to monitor data accesses within the system and, if a data access involving a first address or region of memory occurs, to cause a pie- fetch of data from a second address or region. The monitoring circuit 33, illustrated in more detail in Figure 3, comprises a first memory, memi 35, a second memory, mem2 37 and a comparator 39. Memory memi 35 stores a first address which, if accessed is highly indicative that a data access involving a second address will be made imminently. This second address is stored in memory mem2 37. The comparator 39 is used to compare the addresses of data accesses within the system to the contents of memi 35. The contents of memories memi 35 and mem2 37 may be accessed via lines 41 and 43 respectively.

When the processor requests data from the system memory, the processor issues a data access request which includes the system memory address of the requested data. As shown in Figure 1, this access address is received by the cache 1 via line 25. The access address is transmitted to the monitoring circuit 33 and received at a first input of the comparator 39. The comparator 39 also receives the value stored in memi 35 at a second input. The comparator is arranged to compare the first and second inputs, and to assert an output signal on line 45 if the two inputs match. For example, a bitwise XOR operation between the access address and the value stored in memi may be performed and the comparator output asserted if all bits of the result are equal to zero. It is understood that different ways to perform the comparison may be used. The signal on line 45 is received by mem2 which is arranged to output the value stored by mem2 on line 47 when the signal on line 45 is asserted. Thus, the value stored in mem2 will be output from the monitoring circuit 33 if the data access address received on line 25 matches the value stored in memi.

The value output from the monitoring circuit on line 47 is used to initiate a pre-fetch of data from the address defined by the output value using any suitable method. For example, the output value may be input as an address into the tag memory 7. This causes the cache to determine whether a copy of the data located at that address is already present in the data memory 3. If so, then no further action is taken since the data has already been cached. However, if a copy of the data is not located in the data memory 3 then a cache line of data located at the address is retrieved and stored in the data memory in the manner described above. Alternatively, the output value may be input as an address into the cache load circuit causing a new entry to be added to the queue of pending cache load operations. A pre-fetch of data may be initiated in any other suitable manner using the address based on the value output from the monitoring circuit.

In some cases, only a single cache line of data may be pre-fetched from the address defined by the monitoring circuit output. Alternatively, any specified number of cache lines may be pre-fetched from the address defined by the monitoring circuit output. The number of cache lines to be pre-fetched may be specified in any suitable way, for example by means of a stored value. In one embodiment, the contents of memory mem2 is divided into two parts, the first defining the address of data to be pre-fetched, and the second specifying the number of cache lines to be pre-fetched from that address. As an example, bits 31:5 of the 32 bit value stored in mem2 define bits 31:5 of the address.

Bits 4:0 of the address are implicitly assumed to be zero, in which case the addresses are ones aligned on 2=32 byte boundaries. Bits 4:0 of the value stored in mem2 are interpreted as a binary value specifying the number of cache lines to be pre-fetched from the address.

The value stored in mem2 defining the memory address of data to be pre-fetched may comprise an offset value or other means to define a memory address or region. For example, the value may represent a signed offset value defining an offset relative to a memory address defined by the value stored in memi. In this case, if the address defined by the value stored in meml is addr and the offset is os, then a data access involving the address addr will cause data to be pre-fetched from the address addr+os.

An additional feature will now be described in which a data pre-fetch may be performed even if there is not an identical match between the address of a data access and the address stored in memi. Specifically, a data pre-fetch may be initiated if there is certain relationship between the data access address and the address defined by the value stored in memi.

In one example, a data pre-fetch is initiated if there is a match between the bits of the data access address and the corresponding bits of memi at a set of specified bit positions. To this end, the monitor circuit comprises a third memory, mem3 49, which stores a mask defining those bit positions at which the access address and the contents of mem2 must match for a data pre-fetch to be initiated. For example, for each bit of the mask stored in mem3 equal to 1, the corresponding bits of the data access address and contents of mem2 must match. When a specific bit of the mask is equal to 0, then the corresponding bits of the data access address and mem2 do not need to match. The comparator receives the contents of mem3 at a third input and asserts an output if the bits of the access address and the corresponding bits of the value stored in memi match at those bit positions at which the mask has a value of 1. For example, a bitwise XOR operation between the access address and the value stored in memi may be performed. Then a bitwise AND operation between the resulting value and the mask is performed. The comparator output is asserted if all bits of the final result are equal to zero. It is understood that different ways to perform the comparison based on the mask may be used.

Using this technique, the mask value may be set, for example, so that the p highest order bits of the mask are set to 1 while the remaining q bits are set to 0 (where p+q=N is the size in bits of a memory address). This means that the memory address of a data access would only need to match the address stored in memi in the p highest order bits. Memory addresses having the same values for the p highest order bits span a memory range of 2q bits (i.e. those memory addresses differing only in the q lowest order bits).

Consequently, setting the mask in this way means that the memory address of a data access would not need not be exactly equal to the address stored in memi. Rather, the address of the data access would need to be located within a region of memory of 2q bytes in size beginning with the address defined by bits 31:8 of the value of memi (with the remaining bits being equal to zero). In this way, the system can be configured so that data accesses involving a specified region of memory initiate an automatic pre-fetch of data into the cache.

In another example, the mask may be set so that the p highest order bits are set to 0 while the remaining q bits are set to 1. In this case, the memory address of a data access would only need to match the address stored in memi in the q lowest order bits. The address stored in memi may be one aligned to a address boundary of particular size (such as a cache line sized boundary). In this case, a data access would only initiate a pre-fetch if the address of the data access was aligned to the same boundary as the address stored in memi. For example, the system may be configured so that only data accesses acting on addresses naturally aligned on a cache line sized boundaries trigger pre-fetching of data.

In the example described above, a data pre-fetch is initiated if there is a match between the bits of the data access address and the corresponding bits of memi at a set of specified bit positions. However, other relationships may be applied. For example, a data pre-fetch may be initiated if the data access address is within a certain range from the address defined by the value stored in memi. Other relationships are possible.

In some embodiments, a pre-fetch of data is automatically performed only if the data access causes a cache miss. For example, even if the address of the data access matches the address stored in memi based on the mask stored in mem3, a pre-fetch is only carried out if the hit signal on line 13 is not asserted. In this case, the procedure described above to retrieve and cache the requested data takes place. Concurrently, if the address of the data access matches the address stored in memi based on the mask, data from the location defined by the value stored in mem2 is pre-fetched into the cache.

This additional condition may be applied for the following reason. When a data access involving a first location does not result in a cache miss, this indicates that an access to the first location has already been made. If this is the case then at least the first such access would have triggered a pre-fetch of data from a second location into the cache in the manner described above.

Therefore, it would not be necessary to pre-fetch the data again from the second location following subsequent accesses to the first location. Pre-fetching of data from the second location into the cache is only performed upon the first access to the first location, indicated by a cache miss.

In the embodiment described above, memories memi, mem2 and mem3 are 32 bit registers arranged to store a 32 bit value and whose contents are modifiable. For example, values may be written to memi, mem2 and mem3 during an initialisation period following a system reset, or may be modified dynamically by the processor or other system module. In alternative embodiments, different types and sizes of memory may be used. For example, the memi, mem2 and mem3 may comprise dedicated memories of any suitable type, or may comprise reserved locations within a larger memory space. In alternative embodiments, memi, mem2 and mem3 may comprise read only memories, or may comprise memories that can be written to one time only, for example at the time of manufacture. Allowing the contents of memi, mem2 and mem3 to be modified provides a greater degree of flexibility, for example in systems in which the indicator of an imminent data access may change over time or depending on the application. In other applications in which the indicator of an imminent data access remains fixed, providing read only memories increases security by preventing the appropriate values from being modified inappropriately. The skilled person would appreciate that various further modifications of the embodiments described above may be made.

In the embodiments described above, the values stored in memories memi and mem2 may define addresses in the form of physical memory addresses, virtual memory addresses or may define addresses in any other suitable way.

The kind of memory addresses defined by the values stored in memi and mem2 may be the same, or may be different, It can be seen that, in the embodiments described, the values stored in memories memi and mem2 correspond to different physical memory addresses of the system memory. In other words, a data access involving one physical memory address causes a pre-fetch of data from a different physical memory address, regardless of how the memory addresses are actually represented.

Using the techniques described above, it is not necessary to use special pre-fetch instructions to initiate pre-fetching of data. Instead, a system and method is provided which automatically monitors data fetches and initiates pre-fetches as described above. Since pre-fetches are not initiated using pie-fetch instructions, it is not necessary to modify existing code to include such pre-fetch instructions. Also, since data accesses are being continuously monitored, it is not necessary to know in advance every occasion on which a pre-fetch is required. Any relevant data accesses will be detected dynamically and a pre-fetch initiated if necessary. Furthermore, any delays associated with executing special pie-fetch instructions are eliminated. Data is pre-fetched quickly and efficiently by dedicated autonomous hardware upon detecting a relevant data access.

A further embodiment of the invention will now be described with reference to Figures 4 to 9. In broad terms, in this embodiment there is provided a cache memory comprising storage means and a comparison means arranged to compare a first address provided to the cache memory when a write or load access is made to the cache with at least one predetermined address, and, responsive to the first address corresponding to one of the at least one predetermined addresses, cause the cache memory to fetch data from a further address of an external memory device and store the data in the storage means.

As mentioned above, a disadvantage of known systems is that they require the use of one or more special instructions to pre-fetch data into an Li cache.

Standard names for these instructions are pre-fetch, preload or touch instructions. It is commonplace to extend this functionality to L2 caches so that the aforementioned instructions can effect a similar operation on an attached L2 cache. This is an example of encoding the operation in the op-code of the instruction. In such cases the Li & L2 cache normally communicate via a special interface which allows the L2 to perform actions when a special instruction is executed by the CPU. The further embodiment addresses this disadvantage, that special instructions have to be used to pre-fetch information into the cache.

In this embodiment, an ordinary load access is used to trigger the pre-fetch of multiple cache lines. This is achieved because the L2 cache is configured to be aware that loads from certain addresses are highly indicative of imminent loads from a buffer. This is known as a buffer pre-fetch scheme. The buffer pre-fetch differs from other types of pre-fetch as the software does not need to issue any special pre-fetch instructions nor issue writes to any special registers. This is advantageous in situations which have significant amounts of legacy code which is difficult to modify (e.g. to add pre-fetch instructions) or the software engineer optimising the code cannot identify all the places that the buffer may be accessed. In this case it is possible following buffer creation to associate it with a pre-fetch. Thereafter, the L2 cache will behave automatically. Typically, the buffer pre-fetch scheme results in a buffer being pre-fetched to the cache, and, therefore, multiple cache lines are fetched.

Figures 4 to 8 illustrate a system comprising a level 2 cache. Figure 9 illustrates a further embodiment of the invention.

In the system shown in Figure 5, the level 2 (L2) cache has a target port dedicated to accessing a special register called an L2PFR (L2 pre-fetch register). The use of this register allows CPU and non-CPU requesters to cause data to be fetched into the L2 cache before it is used, therefore avoiding having to suffer the delay incurred when the CPU fetches on demand.

The L2PFR may be implemented as a 32-bit write-only register. Writing a 32- bit value to this register may cause the naturally-aligned 32-byte block -whose address is specified by bits [31:5] of the value -to be fetched into the L2 cache. The pre-fetch operation can therefore be initiated by a CPU with a standard word write operation.

The procedure followed is that first the address is looked up in the L2 cache. If there is a hit, that is the 32-byte block associated with the address is present in the cache, then there is no further activity and no data is fetched. If there is a miss, which implies that the data is not in the cache then space is allocated in the cache and the 32-byte block is fetched from main memory and placed in the level 2 cache. This pre-fetch mechanism is therefore simple to use within the structure of conventional software and conventional DMA engines.

A common use is when a data buffer is to be transferred from an I/O interface to main memory whereupon the CPU will perform some computation on the data contained in the buffer. In a conventional system a DMA engine maybe deployed to transfer data from an I/O interface (e.g. an Ethernet port, a USB port, a SATA disk interface etc.) into system dynamic random access memory (DRAM). Upon completion of the data transfer the DMA engine would send an interrupt to the CPU to signal that the data is transfer has finished. The interrupt handler in the CPU would schedule the execution of an appropriate routine to deal with the computation to be performed on the data buffer.

The routine may then execute in an expedited manner by using one of two methods: 1). A linked list which specifies the set of transfers to be performed by the DMA is extended by one or more additional items. The first additional item specifies that a single 32-bit datum is to be transferred from system memory to the address of the L2PFR register. The value of the datum is the address of the first byte of the data buffer which has been transferred. Optionally, subsequent additional items are similar except that the value of the datum transferred to the L2PFR register is numerically 32 larger than the previous item. If n additional items were specified (where 1�=n�=(buffer size/32)) then this has the effect of pre-fetching some or all of the data buffer into the L2 cache.

2). The transfer proceeds as in a conventional system and an interrupt is sent to the CPU on completion of the DMA. In addition to the conventional actions the interrupt handler writes the address of one or more blocks which contain the data buffer to the L2PFR register. This causes some or all of the data buffer to be requested to be pre-fetched into the L2 cache before the computation routine associated with the data buffer is executed.

Reference is now made to Figure 4, which illustrates a hierarchical memory arrangement. In this arrangement a CPU 1102 (which optionally has a level 1 cache) is supplemented by a separate module known as a level 2 cache 1104. Use of the term level 2 should not be taken to imply exclusive use in systems which have level 1 caches. Nor is there an implication that there is no level 3 or higher level caches. Nonetheless, the level 2 terminology is retained purely for simplicity of exposition.

The level 2 cache (L2 cache) 1104 is functionally located between the CPU 1102 and the rest of the system 1106 so that all of its high performance memory requests have to go via the L2 cache 1104. The L2 cache 1104 is able to service some of its requests from its own contents and other requests is passes on to the rest of the system to be serviced. The L2 cache 1104 also contains a number of configuration and status registers (CSRs) 1108 through which the operation of the L2 cache 1104 may be controlled and monitored.

A top-level diagram of a cache such as the L2 cache 1104 is shown in Figure 5. The cache comprises an access address 1202, which is the address which is presented by the CPU 1102 to the L2 cache 1104, and a tag RAM 1204 which is the memory to which the access address 1202 is associated. In other words the access address 1202 is compared with the contents of the tag RAM 1204 to determine which data RAM 1206 array (line) should be selected. Data RAM 1206 holds the data which is supplied to the L2 cache 1104. In a set-associative cache an address can only reside in a limited number of places in the cache. The collection of places which a single address may reside is called a set 1208. The collection of addresses which are in the same set is called a way 1210. A block of data associated with a single address in the tag RAM 1204 is a line 1212. A refill engine 1214 is present, which is a functional unit whose responsibility is fetching from main memory data which is not already held in the cache. It does this on demand from a standard access or a pre-fetch.

As mentioned, this system makes use of a special register called L2PFR 1110, which is an operational register used to initiate a pre-fetch. The L2PFR 1110 is writable by both the CPU 1102 (using the target 1 port 1112) and modules with DMA capability 1114 in the rest of the system (using the target 2 port 1116). When the register is written with a 32- bit operand, the operand is interpreted as a cache line address (see Figure 7). When an address is submitted to the cache for lookup the address is broken down into a number of fields that are used for different purposes by the hardware. The size and location of each of the fields depends on the size and internal organisation of the cache. An example arrangement of the fields is shown in Figure 7. A word selection field 1402 specifies which of the 8 4-byte words in the line is the requested word. A tag field 1404 is stored in the tag RAM to uniquely identify the address of the data held in the associated line. A set selection field 1406 is used to determine which set in the cache is looked up.

The procedure following a write to the L2PFR 108 is outlined in the flow diagram in Figure 6, with further reference to Figure 8 which illustrates internal logic and buffering of the L2 cache. A write is made into the L2PFR in step S1302. This is interpreted as a request to fetch the address into the L2 cache.

The operand is latched into the target 2 incoming buffer (1502 in Figure 8) and transferred to the first part of the control pipeline Cl (1504) whereupon logic signals are generated such that the address is looked-up in the tags (see 1204 of Figure 6).

A lookup of the L2PFR is made in step S1304. If the lookup of the L2PFR address does yields a match (in step Si 306), as indicated by assertion of the "HIT" signal (1216 in Figure 6) then this indicates that the data is already held in the cache and no further action is taken.

If the lookup of the L2PFR address does not yield a match this is indicated by de-assertion of the HIT signal (1216 in Figure 6). In this case a fetch request is passed to the refill engine (1214 in Figure 6) in step S 1308. The refill engine ensures that an appropriate entry is added to the bus queue (1506) and also to the Pending request buffer (1508). The Pending request buffer holds address and allocation attributes of all outstanding requests.

Entries in the bus queue (1506) will eventually be realized as memory requests on the system interconnect (1118 in Figure 4) in a standard manner.

The request will eventually illicit a response containing the requested data in step S1310. The requested data is buffered in the response queue (1510).

The request attributes contained in the pending request buffer (1508) are used to identify where in the cache the pre-fetched data is to be located and the tag which is to accompany it into the cache (step S1312). The data and tags are loaded into the cache using the line fill buffer (1512).

If the L2 cache is operated in copy-back mode there is a possibility that the place selected for the fetched data was previously occupied by a cache line (the victim) which has been modified since being fetched from memory (i.e. is termed dirty). A dirty victim will require writing back to memory -a process sometimes referred to as eviction. In step S1314 it is checked whether the write-back is needed, and if so, in step S1316 the L2 arranges for the write-back in a manner common to the design of caches and utilizing a write-back buffer to hold the data whose place in the cache will have been taken by the pre-fetched data. In step S1318 the victim is replaced by the fetched data, and, in step S1320, the process halts.

There is also the possibility that the data to be pre-fetched, although not currently present in the cache, is in the process of being fetched into the cache by a preceding data access miss or indeed an earlier pre-fetch. For this reason, in addition to looking up in the TAG array of the cache the pre-fetch address must also search the pending request buffer 1508. If there is a match in the pending request buffer then the pre-fetch request is discarded and no further action is taken.

Data access misses to the L2PFR address which occur when the pre-fetch request is pending will be detected by searching the pending request buffer.

The Pending request buffer is able to link together subsequent data accesses, so that when the fetched data returns it is able to be used to satisfy each of these accesses in turn. This functionality is easily implemented in standard logic and is known to the designers of caches which are able to deal with multiple misses.

An enhancement to the system described above can be achieved though the use of "jumbo pre-fetch". In this case, low-order bits in the L2PFR are used to specify the number of cache lines to be fetched. In the preceding description it should be appreciated that the low order bits are not required to specify the cache line to be fetched as they normally indicate the byte-in-line to be accessed. This is extended to allow multiple cache lines to be fetched efficiently.

This can be performed by a decrement and fetch system. In this encoding all accesses to a dedicated jumbo pre-fetch register (denoted L2PFR_J) are interpreted as in Table 1 below, where bits [4:01 function as a simple count of the cache lines remaining to be fetched, or as in Table 2 below, where bits 14:0] function as power of 2 count of the lines to be fetched. On each L2 cache clock cycle the following procedure happens.

If L2PFR_J bits [4:01 are not zero, a pre-fetch request is issued to the line specified by L2PFRJ bits [31:51 -this occurs in the manner described previously for simple L2PFR register writes. Following the pre-fetch, L2PFRJ [4:01 is decremented by 1 and L2PFR_J [31:5] is incremented by 1. In this way a sequence of pre-fetches can be implemented with a single write to the L2PFR_J.

This logic may be implemented by an additional two adders and a comparator with simple modification to the L2 cache state machine in a manner known to all skilled logic designers.

L2PFR[4:0J Lines Fetched Bytes fetched 00000 0 0 00001 1 32 2 64 00011 3 96 11111 31 992

Table 1

L2PFR[4:0] Lines Fetched Bytes fetched 0000 1 32 0001 2 64 4 128 0011 8 256 16 512 0101 32 1024 0110-1111 Reserved -

Table 2

Figure 9 illustrates the internal structure of a L2 cache with a buffer pre-fetch scheme of the further embodiment. The structure shown in Figure 9 is similar to that described above with reference to Figure 4. However, the L2 cache shown in Figure 9 makes use of three further 32-bit registers.

A level 2 comparison register (L2CR_B) 1702 contains the value against which accessed addresses are compared. A level 2 match register (L2MR_B) 1704 contains a mask which governs how the comparison is performed. In order for a match signal 1706 to be asserted, the access address 1708 must match the contents of the L2CR_B register 1702 in those bit positions where the L2MR_B register 1704 is a 1'. The comparison operation is performed by a comparator 1710.

A level 2 pre-fetch register (L2PFR_B) 1712 indicates the cache line(s) which should be fetched when the match signal 1706 is asserted. The format of this register is the same as that described with reference to the system described above and whose encoding is shown in Table 1.

In operation, when a cacheable load access is made to the L2 cache which results in a miss (i.e. the hit signal 1216 is not asserted) the L2 cache will implement its standard miss handling procedure and fetch the requested cache line into the cache. Concurrent with the miss processing the access address is also presented to the comparison logic 1710. If the access address matches the value in the L2CR_B register 1702 in all bit positions in which the L2MR_B register 1704 is a binary 1' then a pre-fetch is initiated. The size and base address of the pre-fetch is indicated in the L2PFR_B register 1712 in a manner described previously.

It is understood that the features of any of the embodiment described above may be used in any of the other embodiments, where this is possible and appropriate. For example, the address fields illustrated in Figure 7 may be used in the embodiment shown in Figure 1.

Claims

Claims 1. A cache memory system for caching data comprising: -a cache memory for storing a copy of a portion of data stored in a system memory; and -a cache load circuit capable of retrieving the portion of data from the system memory and of storing a copy of the retrieved portion of data in the cache memory; wherein the system further comprises: -means for monitoring accesses to the system memory; -a first memory for storing a first value defining a first memory address; -a comparator for comparing the access address of an access to the system memory with the first memory address; and -a second memory for storing a second value defining a second memory address; the system being arranged such that, if a relationship between the access address and the first memory address is satisfied, the cache load circuit retrieves the portion of data stored in the system memory at the second memory address defined by the second value, and stores the retrieved portion of data in the cache memory; wherein the first and second memory addresses correspond to different physical memory addresses of the system memory.
2. A cache memory system according to claim 1 in which the relationship between the access address and the first address is satisfied if the access address and the first address are equal.
3. A cache memory system according to claim 1 in which the relationship between the access address and the first address is satisfied if the access address and the first address are equal at one or more defined bit positions.
4. A cache memory system according to claim 3 in which the relationship between the access address and the first address is satisfied if the p most significant bits of the access address match the corresponding p most significant bits of the first address are equal.
5. A cache memory system according to claim 3 in which the relationship between the access address and the first address is satisfied if the q least significant bits of the access address match the corresponding q least significant bits of the first address are equal.
6. A cache memory system according to claim 3, 4 or 5 in which the bit positions are defined by a value stored in a third memory.
7. A cache memory system according to claim 1 in which the relationship between the access address and the first address is satisfied if the access address is within a defined range from the first address.
8. A cache memory system according to any preceding claim in which the access to the system memory is a data read.
9. A cache memory system according to any preceding claim in which the access to the system memory is a data write.
10. A cache memory system according to any preceding claim in which the size of the retrieved portion of data is variable.
11. A cache memory system according to claim 10 in which the size of the retrieved data portion is defined by the first value.
12. A cache memory system according to any preceding claim in which the second value is an offset value.
13. A cache memory system according to any preceding claim in which the contents of the first, second and/or third memories are modifiable.
14. A cache memory system according to any preceding claim in which the means for monitoring is one or more of: a hardware monitor, an autonomous monitor and/or a dedicated monitor.
15. A cache memory system according to any preceding claim in which the cache memory system is part of a level 2 cache.
16. A cache memory system substantially as hereinbefore described, with reference to the Figures.
17. An integrated circuit comprising a cache memory system according to any preceding claim.
18. A system comprising: -a processor; -a system memory; and -a cache memory system according to any of claims 1 to 16 functionally located between the processor and the system memory.
19. A system according to claim 18 in which the processor is capable of requesting an access to data in the system memory.
20. A method for pre-fetching data into a cache memory system, the method comprising the steps of: -retrieving a portion of data from a system memory; and -storing a copy of the retrieved portion of data in a cache memory; wherein the method comprises the further steps of: -monitoring accesses tb the system memory; -comparing the access address of an access to the system memory with a first memory address; and -if a relationship between the access address and the first memory address is satisfied, retrieving the portion of data stored in the system memory at a second memory address, and storing the retrieved portion of data in the cache memory; wherein the first and second memory addresses correspond to different physical memory addresses of the system memory.
21. A method substantially as hereinbefore described, with reference to the Figures.