WO2002027498A2 - Systeme et procede d'identification et de gestion de donnees en flux - Google Patents

Systeme et procede d'identification et de gestion de donnees en flux Download PDF

Info

Publication number
WO2002027498A2
WO2002027498A2 PCT/US2001/030395 US0130395W WO0227498A2 WO 2002027498 A2 WO2002027498 A2 WO 2002027498A2 US 0130395 W US0130395 W US 0130395W WO 0227498 A2 WO0227498 A2 WO 0227498A2
Authority
WO
WIPO (PCT)
Prior art keywords
data
cache
streaming
lines
memory system
Prior art date
Application number
PCT/US2001/030395
Other languages
English (en)
Other versions
WO2002027498A3 (fr
Inventor
Peter C. Damron
Steve Chessin
Patricia Shanahan
Aleksandr Guzovskiy
Original Assignee
Sun Microsystems, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US09/677,096 external-priority patent/US6598124B1/en
Priority claimed from US09/677,093 external-priority patent/US6578111B1/en
Application filed by Sun Microsystems, Inc. filed Critical Sun Microsystems, Inc.
Priority to AU2001294856A priority Critical patent/AU2001294856A1/en
Publication of WO2002027498A2 publication Critical patent/WO2002027498A2/fr
Publication of WO2002027498A3 publication Critical patent/WO2002027498A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0862—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches with prefetch
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/60—Details of cache memory
    • G06F2212/6028—Prefetching based on hints or prefetch instructions

Definitions

  • the present invention relates generally to memory systems, and more particularly to cache memory systems and a method of operating the same that provides efficient handling of streaming-data.
  • Modern computer systems generally include a central processing unit (CPU) or processor for processing data and a memory system for storing operating instructions and data.
  • CPU central processing unit
  • memory system for storing operating instructions and data.
  • the speed at which the processor can decode and execute instructions to process data exceeds the speed at which instructions and data can be transferred between the memory system and the processor.
  • the processor is often forced to wait for the memory system to respond.
  • One way of reducing this memory latency period is to organize computer memory into a memory hierarchy.
  • a memory hierarchy consists of multiple levels of memory each using different devices for storing data and each having different speeds, capacities and cost associated therewith.
  • the highest-level of memory commonly known as a cache, is coupled closely to the processor and uses relatively expensive, faster devices that make information, either data or instructions, available to the processor in a shorter period of time.
  • the lower-levels typically include a main-memory and mass-data-storage devices that albeit larger are slower and are therefore correspondingly cheaper.
  • a cache reduces the memory latency period by temporarily storing a small subset of data from lower-levels of the memory system.
  • the processor needs information for an application, it first checks the high-speed cache. If the information is found in the cache (known as a cache-hit), the information will be retrieved from the cache and execution of the application will resume. If the information is not found in the cache (known as a cache-miss) then the processor will proceed to access the slower, lower-level memories. Information accessed in the lower-level memories is simultaneously stored or written in the cache so that should the information be required again in the future it can be obtained directly from the cache, thereby reducing or eliminating any memory latency period.
  • a cache can reduce the memory latency period during a write operation by writing to the cache. This reduces the memory latency period in two ways. First, by enabling the processor to write at the much greater speed of the cache, and second by storing or loading the information in the cache so that, again, should the processor need the information in the future it can be obtained directly from the cache.
  • DRAM dynamic random access memory
  • the main- memory is typically implemented using slower, cheaper dynamic random access memory (DRAM) devices.
  • the cache is implemented using faster random access memory devices, such as static random access memory devices (SRAMs) so that accessing a cache takes much less time to complete than to access main-memory.
  • SRAMs static random access memory devices
  • SRAMs typically require greater number of devices per bit of information stored, and thus are more expensive than DRAM.
  • the cache may be located on the same chip as the CPU. The proximity of the cache to the CPU increases the speed with which the CPU can access the cache by eliminating delays due to transmission over external circuits.
  • a cache located on the same chip as the CPU is often known as primary or level 1 (LI) cache since the memory system typically includes a larger, slower level 2 (L2) cache outside the CPU chip.
  • L3 level
  • victim cache for temporarily storing data displaced from the L2 cache.
  • mass-storage-devices provide the largest data storage capacity and typically use the slowest and therefore cheapest technology.
  • magnetic, optical or magnetic-optical technologies to store large amounts of instructions and data on tapes, or fixed or removable disks.
  • cache 10 is divided logically into two main components or functional units.
  • Data-store 15 where the cached information is actually stored, and tag- field 20, a small area of memory used by the cache to keep track of the location in the memory where the associated data can be found.
  • the data-store is structured or organized as a number of cache-lines 25 or sets of cache-lines each having a tag-field 20 associated therewith, and each capable of storing multiple blocks or bytes of data.
  • each cache-line 25 stores 32 or 64 bytes of data.
  • the tag-field 20 for each cache-line 25 or set of cache-lines includes an index 30 that uniquely identifies each cache-line in the cache 10, and tag 35 that is used in combination with the index to identify an address in lower-level memory 40 from which data-stored in the cache-line has been read from or written to. Often the index 30 is not stored in the cache 10 but is implicit, with the address of the cache-line 25 itself providing the index.
  • the tag-field 20 for each cache-line 25 also includes one or more bits, commonly known as a validity-bit 45, to indicate whether the cache-line contains valid data.
  • the tag-field 20 may contain other bits (not shown) for example for indicating whether data at the location is dirty, that is has been modified but not written back to lower-level memory 40.
  • caches To speed up memory access operations, caches rely on principles of temporal and spacial-locality. These principles of locality are based on the assumption that, in general, a computer program accesses only a relatively small portion of the information available in computer memory in a given period of time. In particular, temporal locality holds that if some information is accessed once, it is likely to be accessed again soon, and spatial locality holds that if one memory location is accessed then other nearby memory locations are also likely to be accessed. Thus, in order to exploit temporal-locality, caches temporarily store information from a lower-level memory the first time it is accessed so that if it is accessed again soon it need not be retrieved from the lower-level memory. To exploit spatial-locality, caches transfer several blocks of data from contiguous addresses in lower-level memory, besides the requested block of data, each time data is written in the cache from lower-level memory.
  • the most important characteristics of a cache are its hit rate, that is the fraction of all memory accesses that are satisfied from the cache over a given period of time, and its access time, that is the time it takes to read from or write to the cache. These in turn depend in large part on how the cache is mapped to addresses in the lower-level memory. The choice of mapping technique is so critical to the design of the cache that the cache is often named after this choice. There are generally three different ways to map the cache to the addresses in memory. Direct-mapping, shown in FIG. 1 , is the simplest way to map a cache to addresses in main-memory.
  • the number of cache-lines is determined, the addresses in memory divided into the same number of groups of addresses, and addresses in each group associated with one cache-line. For example, for a cache having 2 n cache-lines, the addresses are divided into 2 n groups and each address in a group associated with a single cache-line. The lowest n address bits of an address corresponds to the index of the cache-line to which data from the address can be stored. The remaining top address bits are stored as a tag that identifies from which of the several possible addresses in the group the data originated.
  • each cache-line is shared by a group of 4,096 addresses in main-memory.
  • To address 64-MB of memory requires 26 address bits since 64-MB is 2 26 sbytes.
  • the lowest five of these address bits, A0 to A4 are ignored in the mapping process, although the processor will use them later to determine which of the 32 bytes of data in the cache-line to accesses.
  • the next 14 address bits, A5 to A18 provide the index of the cache-line to which the address is mapped.
  • any cache-line can hold data from any one of 4,096 possible addresses in main-memory
  • the next seven highest address bits, A19 to A25 are used as a tag to identify to the processor which of the addresses the cache-line holds data from.
  • This scheme while simple, has the disadvantage that if the program alternately accesses different addresses which map to the same cache location, i.e., addresses within the same group, then it will suffer a cache-miss on every access to these locations.
  • a fully-associative mapped cache (not shown) avoids the cache conflict of the directly mapped cache by allowing blocks of data from any address in main-memory to be, stored anywhere in the cache.
  • one nrnhlem with fiillv associative caches is
  • n-way set associative cache where n is the number of cache-lines in each set.
  • FIG. 2 shows an example of a 2-way set associative cache.
  • Memory addresses are mapped in the cache in a manner similar to the directly mapped cache. For example, to map a 64-MB main-memory having 26 address bits to a 512-KB 4-way set associative cache the cache is divided into 4,096 sets of 4 cache-lines each and 16,384 addresses in main-memory associated with each set. Address bits A5 to Al 6 of a memory address represent the index of the set to which the address maps to. The memory address can be mapped to any of the four cache-lines in the set. Because any cache-line within a set can hold data from any one of 16,384 possible memory addresses, the next nine highest address bits, A17 to A25, are used as a tag to identify to the processor which of the memory addresses the cache-line holds data from. Again, the lowest five address bits, A0 to A4, are ignored in the mapping process, although the processor will use them later to determine which of the 32 bytes of data in the cache-line to accesses.
  • a cache-line is selected to be written-back or flushed to main-memory or to a lower-level victim cache.
  • the new data is then stored in place of the flushed cache-line.
  • the cache-line to be flushed is chosen based on a replacement policy implemented via a replacement algorithm.
  • LRU Least Recently Used
  • the cache controller maintains in a register several status bits that keep track of the number of times in which the cache-line was last accessed. Each time one of the cache-lines is accessed, it is marked most recently used and the others are adjusted accordingly. A cache-line is selected to be flushed if it has been accessed (read or written to) less recently than any other cache-line.
  • the LRU replacement policy is based on the assumption that, in general, the cache-line which has not been accessed for longest time is least likely to be accessed in the near future.
  • replacement schemes include random replacement, an algorithm that picks any cache-line with equal probability, and First-In-First-Out (FIFO), algorithm that simply replaces the first cache-line loaded in a particular set or group of cache-lines.
  • FIFO First-In-First-Out
  • streaming-data replaces data already present in the cache that is more likely to be required by the processor for subsequent processing, thereby resulting in a greater number of cache misses and lower cache performance.
  • Streaming-data is particularly a problem for applications which require periodic or infrequent processing of very large amounts of streaming-data that can displace all data previously stored in the cache or even in multiple levels of caches.
  • a cache memory system and method of operating the system that is capable of identifying and efficiently handling streaming-data.
  • a system and method of operating a cache memory system having multiple levels of caches that reduces or eliminates displacement by streaming- data of data already stored in a cache that is likely to be needed in the near future.
  • a system and method of operating a cache memory system having multiple levels of caches that reduces or eliminates displacement of data in a lower- level cache that may be needed by the processor in the future by streaming-data displaced from a higher-level cache.
  • the present invention overcomes the disadvantages of the prior art by providing a cache memory system and method for operating the same that provides an improved handling of streaming-data.
  • streaming-data it is meant data that having been accessed by a processor will not be accessed again for a relatively long time.
  • the present invention provides a method for operating a cache memory system having a cache controller and at least one cache with a number of cache- lines capable of caching data.
  • a request to cache data in the cache is received in the cache controller, it is determined from the request whether the data is streaming-data and the data is cached in one of the cache-lines.
  • the cache-line includes a data-store in which the data is cached and a tag-field in which information about the data is stored, and the method further includes the step of setting a streaming- data-bit in the tag-field to identify the data as streaming-data.
  • the step of receiving a request to cache data includes the step of receiving from a processor executing a program, a request to store data from or store data to a main memory.
  • the step of determining from the request whether the data is streaming-data is accomplished by recognizing in the request from the processor a special instruction to cache streaming-data.
  • the special instruction can take the form of LOAD_STREAMING_DATA [A], RO ⁇ STORE_STREAM ⁇ NG_DATAR, [A], where A is an address of the data in memory and R is a register in the processor the data is to be loaded to.
  • the step of receiving a request to cache data includes receiving from a compiler a pre- fetch instruction to load data from main memory.
  • the step of determining whether the data is streaming-data involves the step of recognizing whether the pre-fetch instruction is a special pre-fetch instruction to pre-fetch streaming- data from an address A in main memory.
  • the pre-fetch instruction includes a kinds-field
  • the step of determining from the request whether the data is streaming-data includes the step of recognizing a special variant in the kinds-field, for example, the variant in the kinds-field can take the form of a READ_ONCE or a WRITE_ONCE.
  • the processor has an architecture supporting alternate address space indexing and alternate load instructions, and the step of receiving a request to cache data involves the steps of receiving from the processor alternate load instruction to load streaming-data.
  • the processor can have a SPARC ® architecture supporting Alternate Space Indexing (ASI) instructions, and the step of determining whether the data is streaming-data can include the step of receiving from the processor an ASI instruction to cache streaming-data.
  • the special instruction can take the form of LOAD [A], [ASI], R or STORE [A], [ASI], where A is an address of the data in memory, R is a register in the processor the data is to be loaded to and ASI is a number representing one of 256 possible ASI address spaces that can be specified.
  • the ASI instruction used can take the form of LOAD [A], [ASI], R, or STORE [A], [ASI], where A is the address of the data in main-memory 115, R is a register in processor 110 to which the data is to be loaded and ASI is a number representing one of 256 possible ASI address spaces that is pre-selected to signify streaming-data.
  • the invention is directed to a cache memory system having at least one set associative cache with a number of cache-lines capable of caching data, the cache-lines grouped into a number of sets, each set having a number of ways. In the method, at least one way in each set is reserved for caching of streaming-data.
  • the cache controller Upon receiving in the cache controller a request to cache data in the cache, it is determined whether the requested data is streaming-data and, if the data is streaming data, the data is cached in a way reserved for streaming data.
  • the step of reserving at least one way in each set for streaming data can include the step of setting a bit or bits associated with each way during booting or initialization of the cache memory system.
  • the reserved ways can be identified using a separate register to store a value identifying each reserved way, or by specifying in the cache controller during initialization of the cache memory system a way or ways reserved for streaming data in every cache-line.
  • the data includes a streaming-data-bit and the step of determining whether the requested data is streaming-data includes the step of checking the streaming-data-bit. In another embodiment, the step of determining whether the requested data is streaming-data involves determining from the request whether the data is streaming-data
  • the invention is directed to a cache memory system for providing efficient handling of streaming-data.
  • the cache memory system includes a cache having a plurality of cache-lines capable of caching data, a cache controller configured to cache data to any one of the plurality of cache-lines, means for determining whether data is streaming-data and means for identifying cached data as streaming-data.
  • the means for determining whether the data is streaming data involves a special instruction received in the cache controller requesting caching of data, and the cache controller is configured to determine from the special instruction whether the data is streaming-data.
  • each of the plurality of cache-lines includes a data-store in which the data is cached and a tag-field in which information about the data is stored, and the means for identifying cached data as streaming-data includes a streaming-data-bit in the tag-field that is set to identify the data as streaming- data.
  • the cache is a set associative cache with the plurality of cache-lines grouped into a plurality of sets, each set having a number of ways, and the means for identifying cached data as streaming-data includes at least one way in each of the plurality of sets reserved for caching of streaming-data.
  • the system and method of the present invention is particularly useful in a computer system having a processor and one or more levels of hierarchically organized memory in addition in the cache memory system.
  • the system and method of the present invention can be used in a cache memory system coupled between the processor and a lower-level main-memory.
  • the system and method of the present invention can also be used in a buffer or interface coupled between the processor or main-memory and a mass-storage-device such as a magnetic, optical or optical- magnetic disk drive.
  • the present invention provides a method for operating a cache memory system having a cache with a number of cache-lines each capable of storing data transferred between a processor and a lower-level memory.
  • data is loaded or stored into at least one of the plurality of cache-lines and checked to determine if the data is streaming-data.
  • each cache-line has a streaming-data-bit associated therewith for indicating whether data-stored therein is streaming-data
  • the cache memory system further includes a cache controller configured to determine if the streaming-data-bit is set.
  • the cache is a set associative cache with the cache-lines grouped into a number of sets
  • the cache memory system further includes a number of history queues, each history queue associated with one of the sets.
  • the history queues are adapted to hold a sequence of numbers identifying the cache-lines in the associated set accessed in a first predetermined number, n, of preceding references to the set.
  • determining if the data in the cache-line is streaming-data involves setting the streaming-data-bit if data in the cache-line has been accessed less than a second predetermined number, k, of times in the preceding n references to the set.
  • the present invention is directed to a cache memory system further including a victim cache between the cache and the lower-level memory.
  • a method of operating the cache memory system generally involves determining, using the cache controller, before loading data to an element in one of the cache-lines if the loading of data will replace earlier data already stored in the cache-line. If the loading of data will replace data in the cache-line, it is determined if the data that will be replaced is streaming-data. If the data to be replaced is not streaming-data, it is loaded in the victim cache.
  • the data to be replaced is streaming-data, it is not loaded into the victim cache, thereby improving system efficiency by eliminating the need to copy the data to be replaced and, possibly, avoiding replacing other earlier data in the victim cache that may be needed by the processor in the future.
  • the system and method of the present invention is particularly useful in a computer system having a processor and one or more levels of hierarchically organized memory in addition to the cache memory system.
  • the system and method of the present invention can be used in a cache memory system coupled between the processor and a lower-level main-memory.
  • the system and method of the present invention can also be used in a buffer or interface coupled between the processor or main-memory and a mass-storage-device such as a magnetic, optical or optical- magnetic disk drive.
  • the advantages of the present invention include: (i) the ability to identify streaming-data-stored in a cache of a cache memory system, (ii) the ability to selectively copy only non-streaming-data displaced from the cache to a victim cache and (iii) the ability to selectively load data into a fully associative or set associative cache in such a manner as to preferentially replace streaming-data.
  • FIG. 1 (prior art) is a diagram showing a cache having a number of cache-lines each having a tag-field associated therewith, and each capable of storing multiple blocks of data;
  • FIG. 2 (prior art) is a diagram showing an example of a 2-way set associative cache;
  • FIG. 3 A is a block diagram illustrating a computer system having an embodiment of a cache memory system according to the present invention
  • FIG. 3B is a block diagram illustrating a schema of a cache-line in the cache memory system of FIG 3 A according to an embodiment of the present invention
  • FIG. 3C is a block diagram illustrating a schema of a set in a four- way set associative cache according to an embodiment of the present invention
  • FIG. 4 is a flowchart showing an embodiment of a process for operating a cache memory system having a victim cache between a primary cache and a lower-level memory according to an embodiment of the present invention
  • FIG. 5 is a flowchart showing an embodiment of a process for operating a cache memory system having an associative mapping scheme to provide an improved cache replacement policy according to an embodiment of the present invention
  • FIG. 6 is a block diagram illustrating a schema of a history queue according to an embodiment of the present invention.
  • FIG. 7 is a flowchart showing an embodiment of a process for operating a cache memory system using a history queue to determine if the data is streaming-data according to an embodiment of the present invention
  • FIG. 8 is a flowchart showing an embodiment of a process for using special load and store memory instructions to identify streaming-data according to an embodiment of the present invention.
  • FIG. 9 is a flowchart showing an embodiment of a process for using read-once and write-once software pre-fetch instructions to identify streaming-data according to an embodiment of the present invention.
  • the present invention is directed to cache memory systems and methods of operating the same that provide efficient handling of streaming-data.
  • FIG. 3 A shows a block diagram of an exemplary embodiment of a computer system 100 in which an embodiment of a cache memory system of the present invention can be incorporated.
  • computer system 100 typically includes central processing unit (CPU) or processor 110 for executing instructions for a computer application or program (not shown), main-memory 115 for storing data and instructions while running the application, a mass-data-storage device, such as disk drive 120, for a more permanent storage of data and instructions, system bus 130 coupling components of the computer system, and various input and output devices such as a monitor, keyboard or pointing device (not shown).
  • CPU central processing unit
  • main-memory 115 for storing data and instructions while running the application
  • mass-data-storage device such as disk drive 120
  • system bus 130 coupling components of the computer system
  • various input and output devices such as a monitor, keyboard or pointing device (not shown).
  • Cache memory system 105 has a cache memory or cache separate and distinct from the processor, shown here as level 2 (L2) cache 135, for temporarily storing data and instructions recently read from or written to lower level main-memory 115 or mass- storage-device 120.
  • Cache controller 140 controls operation and content of cachel35 by controlling mapping of memory addresses in the cache and the replacement of data in the cache in accordance with a cache replacement policy.
  • cache memory system 105 can further include primary or level 1 (LI) cache 145 integrally formed with processor 110 and one or more level 3 (L3) or victim caches 155 for temporarily storing data replaced or displaced from the LI or L2 cache to speed up subsequent read or write operations.
  • LI level 1
  • L3 level 3
  • LI cache 145 typically has a capacity of from about 1 to 64 KB, while lower- level L2 and L3 caches 135, 155, can have capacities of from about 128 KB to 64 MB in size.
  • cache memory system 105 can also have separate caches for instructions and data, which can be accessed at the same time, thereby allowing an instruction fetch to overlap with a data read or write.
  • the caches 135, 145, 155 can be organized as directly-mapped, fully-associative or set-associative caches as described above. In one embodiment, to realize the fullest advantages of the present invention as explained below, the caches are organized as n- way set-associative caches, where n is an integer of two or more.
  • FIG. 3B illustrates a schema of cache-line 160 of cache 135, 145, 155, in cache memory system 105 of FIG 3 A.
  • Cache-line 160 includes data-store 165 capable of storing multiple blocks or bytes of data, and tag-field 175 containing address information and control bits.
  • the address information provide information on where in the cache entries in the data-store can be found in main memory 115.
  • the address information also includes index 180 (often implicit, as noted above) that uniquely identifies each cache- line 160 in cache 135, 145, 155, and a tag 185 or tags, one for each way, that is used in combination with index 180 to identify an address in main-memory 115 from which data stored in the cache line has been read from or written to.
  • Control bits can include validity bit 190 which indicates if the cache-line contains valid data, bits for implementing a replacement algorithm, a dirty-data-bit for indicating whether data in the cache-line has been modified but not written-back to lower-level memory, and, in accordance with the present invention, a FIFO-bit or streaming-data-bit 200 or bits for indicating whether the cache-line contains streaming-data or non-streaming-data.
  • Streaming-data-bit 200 can be set by cache controller 140 using various hardware or software means, as explained infra, or by an operating system or application running on computer system 100.
  • cache controller 140 is configured to monitor or check the status of streaming-data-bit 200 to provide more efficiently handling of streaming and non-streaming-data, thereby raising the cache hit-ratio and improving cache performance.
  • FIG.3C is a block diagram illustrating a schema of a set (not shown) in a four- way set associative cache according to an embodiment of the present invention.
  • Each way 170 has a data store 165 and tag-field 175 associated therewith.
  • tag-field 175 includes a streaming-data-bit 200 that is used to indicate whether data store 165 includes streaming-data.
  • at least one way 170 in each set can be reserved for storing only streaming data, and cache controller 140 upon determining the data is streaming data can cache the data in one of the reserved ways (not shown), thereby identifying it as streaming-data.
  • cache controller 140 upon determining the data is streaming data can cache the data in one of the reserved ways (not shown), thereby identifying it as streaming-data.
  • operation of cache memory system 105 will now be described.
  • Cache controller 140 checks to see if the requested information is available in one of caches 135, 145, 155, while simultaneously a memory controller (not shown) begins the process of accessing main-memory 115. This minimizes the loss of time in the event of a cache-miss.
  • Cache controller 140 checks for a cache-hit by looking in the cache at cache-line 160, or in the case of a set-associative cache a set of cache-lines, at a location derived from an address provided by processor 110. For example, for cache memory system 105 as described above in which sixty-four (64) MB of main-memory 115 is mapped to a five hundred and twelve (512) KB cache 135 having thirty-two (32) blocks per cache-line 160, processor 110 provides a twenty-six (26) bit address, A0 to 25, to cache controller 140. The lowest five bits (A0 to A4) are used to differentiate between the thirty-two different blocks mapped to different ways or elements 170 in the cache-line and hence are ignored by cache controller 140.
  • (64) MB of main-memory 115 is mapped to a five hundred and twelve (512) KB cache 135 having thirty-two (32) blocks per cache-line 160
  • processor 110 provides a twenty-six (26) bit address, A0 to 25, to cache controller 140.
  • next 14 bits represent index 180 of cache-line 160 that cache controller 140 needs to check (note that 2 14 is 16,384 the number of cache-lines in a 512 KB cache).
  • address bits A5 to A18 of the memory address represent index 180 of the set -hat cache controller 140 needs to check.
  • Cache controller 140 next reads tag 185 of cache-line 160, or set of cache-lines, indicated by address bits A5 to Al 8 to determine if the desired data is stored in that cache-line, or set of cache-lines, and compares the tag to the highest 7 address bits, A19 to A25. If tag 185 and the address bits are identical, then it is a cache-hit. Cache controller 140 reads the contents in data-store 165 of indicated cache-line 160 to processor 110 and cancels the partially-completed read of main-memory 115.
  • tag 185 doesn't match address bits Al 9 to A25 it is a cache-miss.
  • the read of main-memory 115 started earlier is completed, with 32 blocks of data being read from main-memory at the location specified by bits A5 to A25. These bytes are fed to processor 110, while simultaneously cache controller 140 stores the data to cache 135, 145, 155, at cache-line 160, or to one of the cache-lines in a set of cache-lines, indicated by address bits A5 to A25. If cache memory system 105 includes victim or L3 cache 155, as shown in FIG.3 A, earlier data already present in the indicated cache-line can be copied to the victim cache.
  • cache memory system 105 further includes victim cache 155 between higher level LI and L2 caches 145, 135, and main-memory 115.
  • Cache memory system 105 is configured and operated so that streaming-data displaced from LI or L2 caches 145, 135, is not written to victim cache 155, preventing streaming-data from displacing data in victim cache 155 that may be needed by processor 110 in the future (non-streaming-data), and thereby preserving the non-streaming-data at a higher-level more closely coupled to the processor and speeding up subsequent read and write operations.
  • FIG.4 is a flowchart showing an embodiment of a process for operating cache memory system 105 having victim cache 155 between primary cache 145 and lower-level memory 115 according to an embodiment of the present invention.
  • it is determined before loading data to an element 170 in one of the cache-lines if the loading of data will replace earlier data already stored in the cache-line.
  • Step 210 If no data will be replaced, the data to be stored is stored in LI or L2 cache 145, 135.
  • the loading of data will replace data in cache-line 160, it is determined if the data that will be replaced is streaming-data. (Step 220) If the data to be replaced is not
  • Step 225 If the data to be replaced is streaming-data, it is not loaded into victim cache 155 (step 230), and the data to be stored is stored in LI or L2 cache 145, 135. (Step 215).
  • the step can involve checking the status of a previously set streaming-data-bit 200 associated with element 170 or cache-line 160.
  • cache memory system 105 has fully-associative or set- associative caches 135, 145, 155, and is configured and operated to pro vide a replacement policy or scheme that selectively replaces streaming-data first, thereby affording longer residency of non-streaming-data in LI or L2 caches and improving the cache hit-ratio.
  • FIG. 5 is a flowchart showing an embodiment of a process for operating cache memory system 105 having an associative mapping scheme to provide an improved cache replacement policy according to an embodiment of the present invention. Again, in the method, it is determined before loading data if the loading of data will replace data already stored in cache-line 160.
  • Step 245 If no data will be replaced, the data to be stored is simply stored in LI or L2 cache 145, 135. (Step 250) However, if the loading of data will replace data in cache-line 160, elements 170 in each of the cache-lines to which the data could be loaded are identified (step 255), and it is determined if streaming- data is stored in any of these elements (step 260). If streaming-data is stored in at least one of elements 170 to which the data could be loaded, the data is loaded into an element selected to replace the streaming-data. (Step 265) Otherwise, the data is simply loaded into any one the possible elements (step 250) either at random or using a secondary replacement algorithm, such as Least-Recently-Used (LRU) algorithm, described above.
  • LRU Least-Recently-Used
  • cache memory system 105 uses a set-associative mapping scheme, and the step of determining if streaming-data is stored in at least one of the ways to which the data could be loaded, step 260, involves determining if streaming- data is stored in at least one of elements 170 in one of cache-lines 160 in a set to which data is to be loaded.
  • cache memory system 105 further includes a number of history queues 270 associated with each set in a set-associative cache to track the order in which cache-lines in the set are accessed in preceding references to the set.
  • An exemplary, block diagram illustrating a schema of an embodiment of history queue 270 is shown in FIG. 6. Referring to FIG. 6, the history queue holds N element numbers identifying cache-lines 160 of the set that were referenced for each of the last Nreferences to the set, where N is a first predetermined whole number greater than or equal to two.
  • a method for operating cache memory system 105 to determine if data in cache- line 160 is streaming-data using history queue 270 will now be described with reference to FIG. 7.
  • data is stored in at least one of cache-lines 165 of a set (Step 275) and an element number corresponding to the cache-line is stored in the history queue of the set. (Step 280)
  • history queue 270 is searched by cache controller 140 to determine if cache-line 160 of the set has been referenced less than ET times during the last Nreferences, except for the last one, where K is a second predetermined whole number between 2 and N inclusive.
  • Step 285) If cache-line 160 has been referenced less than K times during the last Nreferences, except for the last one, then the streaming-data-bit 200 is set for that cache-line (Step 290), and, as described above, the cache-line is preferentially selected for replacement by the replacement policy and/or discarded rather than being written to victim cache 155 where it could displace non-streaming-data. (Step 295) When cache-line 160 has been replaced or discarded the references in the cache-line are cleared from history queue 270. (Step 300)
  • Nis greater than three to provide a sufficient residency of non- streaming-data in LI or L2 caches 145, 135, thereby improving the cache hit-ratio. More advantageously, Nis less than about 8 to allow streaming-data to be quickly identified and to reduce the time necessary for cache controller 140 to search history queues 270.
  • the step of determining whether data is streaming-data is accomplished by recognizing software hints or special instructions in requests from processor 110 or a compiler (not shown) that indicate the request is to cache streaming- data.
  • a compiler is a program that converts another program from a source or programming language to machine language or object code.
  • special load instructions are used to load streaming-data from main-memory 115 and special store instructions are used to store streaming-data to main-memory.
  • the request to cache data to cache 135 is received in cache controller 140 (step 400), it is determined from the request whether the data is streaming-data (step 405) and the data is cached in a way in one of cache-lines 160 (step 410).
  • the special instruction can take the form of LOAD_STREAMI ⁇ G_DATA [A], R or STORE_STREAMING_DATA R, [A], where A is an address of the data in main- memory 115 and R is a register in processor 110 to which the data is to be loaded to.
  • the method further includes the step of setting streaming-data-bit 200 in tag- field 175 to identify the data as streaming-data (step 415).
  • At least one way 170 in each cache-line 160 can be reserved for storing only streaming data, and cache controller 140 upon determining the data is streaming data can cache the data in one of the reserved ways (not shown), thereby identifying it as streaming-data.
  • This embodiment has an advantage of providing a cache that can be reconfigured to handle varying amounts of streaming-data.
  • a cache memory system in a computer system running applications using large amounts of streaming data i.e., those accessing large stacks of data or using vector programming, can be configured to reserve several ways in each cache-line for streaming data.
  • cache 135 can be configured to reserve multiple ways 170 in each cache-line for different classes of streaming-data, thereby providing some permanence for different classes of streaming-data.
  • Reserved ways 170 can be identified by setting a bit or bits associated with each way during booting or initialization of cache memory system 105 identifying the way as reserved for streaming-data.
  • reserved ways 170 can be identified using a separate register to store a value identifying each reserved way, or by specifying during initialization of cache memory system 105 the reserved ways. For example, way number four of every cache-line 160 can be designated during initialization as being reserved for streaming-data.
  • the step of determining whether data is streaming-data involves receiving from the compiler a special compiler- directed pre-f etching instruction to load data from main-memory 115.
  • pre-fetching it is meant the act of identifying and moving the identified data to cache 135 before it is actually needed by processor 110.
  • the method of determining whether the data is streaming-data involves receiving in cache controller 140 a request to load data to cache 135 from main-memory 115 (step 420), and recognizing whether the request includes a special pre-fetch instruction to pre-fetch streaming-data from main- memory, (step 425)
  • the special instruction can take the form, for example, of PREFETCH [A], STREAMING DATA, where A is the address of the data in main- memory 115.
  • the pre-fetch instruction includes a kinds-field
  • the step of determining from the request whether the data is streaming- data includes the step of recognizing a special variant in the kinds-field.
  • the variant in the kinds-field can take the form of READ_ONCE or WRITE_ONCE.
  • data which is pre-fetched using a pre-fetch instruction with READ_ONCE in the kinds- field would be determined to be streaming-data.
  • the method can optionally further include the step of setting streaming-data-bit 200 in tag-field 175 to identify the data as streaming-data (step 430).
  • the data can be saved to way 170 reserved for caching streaming-data, as described above, thereby identifying it as streaming-data.
  • processor 110 has an architecture supporting alternate address space indexing and alternate or special load instructions, and the step of determining whether data is streaming-data involves receiving from processor 110 an alternate instruction that indicates the data to be cached is streaming data.
  • processor 110 can have a SPARC ® architecture supporting Alternate Space Indexing (ASI), and the step of determining whether the data is streaming-data can include the step of receiving from the processor an ASI instruction to cache streaming- data.
  • SPARC ® or scalable processor architecture is an instruction set architecture designed by Sun Microsystems, Inc., of Palo Alto, CA, to streamline and improve the speed of the processor.
  • ASI instructions are an alternate set of load (or store) instructions originally developed for running diagnostics on the processor and for providing access to memory not accessible using ordinary instructions (non-faulting memory). There are 256 possible ASI address spaces available at which instructions can be stored.
  • the ASI instruction used can take the form of LOAD [A], [ASI], R, or STORE [A], [ASI], where A is the address of the data in main-memory 115, R is a register in processor 110 to which the data is to be loaded and ASI is a number representing one of 256 possible ASI address spaces that is pre-selected to signify streaming-data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

L'invention concerne un système et un procédé destinés à un traitement efficace de données en flux dans un système de mémoire cache possédant un dispositif de commande de cache et au moins un cache avec un certain nombre de lignes de cache, chaque ligne de cache comprenant au moins une façon permettant de mettre des données en mémoire cache. Dans le procédé, une demande de mise en mémoire cache des données est reçue au niveau du dispositif de commande du cache (140), on détermine pour cette demande si les données sont transmises en continu ou mise en mémoire cache. La ligne de cache (160) peut comprendre une mémoire de données (165) dans laquelle les données sont mises en mémoire et un champ étiquette (175) dans lequel des informations concernant les données sont stockées, et le procédé comprend, en outre, l'installation de bits de données en flux (200) dans le champ étiquette afin d'identifier les données en tant que données en flux. Dans l'un des modes de réalisation, le procédé consiste à déterminer si les données sont des données en flux impliquant la reconnaissance d'une instruction spéciale afin de mettre en mémoire cache ces données en flux dans une demande provenant d'un processeur (110) exécutant un programme ou d'un compilateur compilant un programme.
PCT/US2001/030395 2000-09-29 2001-09-27 Systeme et procede d'identification et de gestion de donnees en flux WO2002027498A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001294856A AU2001294856A1 (en) 2000-09-29 2001-09-27 System and method for identifying and managing streaming-data

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US09/677,093 2000-09-29
US09/677,096 US6598124B1 (en) 2000-09-29 2000-09-29 System and method for identifying streaming-data
US09/677,096 2000-09-29
US09/677,093 US6578111B1 (en) 2000-09-29 2000-09-29 Cache memory system and method for managing streaming-data

Publications (2)

Publication Number Publication Date
WO2002027498A2 true WO2002027498A2 (fr) 2002-04-04
WO2002027498A3 WO2002027498A3 (fr) 2003-01-23

Family

ID=27101706

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/030395 WO2002027498A2 (fr) 2000-09-29 2001-09-27 Systeme et procede d'identification et de gestion de donnees en flux

Country Status (2)

Country Link
AU (1) AU2001294856A1 (fr)
WO (1) WO2002027498A2 (fr)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2468007A (en) * 2009-02-20 2010-08-25 Advanced Risc Mach Ltd Data processing apparatus and method dependent on streaming preload instruction.
US8516230B2 (en) 2009-12-29 2013-08-20 International Business Machines Corporation SPE software instruction cache
US8522225B2 (en) 2010-06-25 2013-08-27 International Business Machines Corporation Rewriting branch instructions using branch stubs
US8627051B2 (en) 2010-06-25 2014-01-07 International Business Machines Corporation Dynamically rewriting branch instructions to directly target an instruction cache location
US8782381B2 (en) 2010-06-25 2014-07-15 International Business Machines Corporation Dynamically rewriting branch instructions in response to cache line eviction
US8856452B2 (en) 2011-05-31 2014-10-07 Illinois Institute Of Technology Timing-aware data prefetching for microprocessors
US9459851B2 (en) 2010-06-25 2016-10-04 International Business Machines Corporation Arranging binary code based on call graph partitioning
US9690710B2 (en) 2015-01-15 2017-06-27 Qualcomm Incorporated System and method for improving a victim cache mode in a portable computing device
CN114860785A (zh) * 2022-07-08 2022-08-05 深圳云豹智能有限公司 缓存数据处理系统、方法、计算机设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5546559A (en) * 1993-06-07 1996-08-13 Hitachi, Ltd. Cache reuse control system having reuse information field in each cache entry to indicate whether data in the particular entry has higher or lower probability of reuse
US5732242A (en) * 1995-03-24 1998-03-24 Silicon Graphics, Inc. Consistently specifying way destinations through prefetching hints
US5944815A (en) * 1998-01-12 1999-08-31 Advanced Micro Devices, Inc. Microprocessor configured to execute a prefetch instruction including an access count field defining an expected number of access
WO1999050752A1 (fr) * 1998-03-31 1999-10-07 Intel Corporation Structure a antememoire partagee pour instructions temporelles et non-temporelles

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5546559A (en) * 1993-06-07 1996-08-13 Hitachi, Ltd. Cache reuse control system having reuse information field in each cache entry to indicate whether data in the particular entry has higher or lower probability of reuse
US5732242A (en) * 1995-03-24 1998-03-24 Silicon Graphics, Inc. Consistently specifying way destinations through prefetching hints
US5944815A (en) * 1998-01-12 1999-08-31 Advanced Micro Devices, Inc. Microprocessor configured to execute a prefetch instruction including an access count field defining an expected number of access
WO1999050752A1 (fr) * 1998-03-31 1999-10-07 Intel Corporation Structure a antememoire partagee pour instructions temporelles et non-temporelles

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"CONDITIONAL LEAST-RECENTLY-USED DATA CACHE DESIGN TO SUPPORT MULTIMEDIA APPLICATIONS" IBM TECHNICAL DISCLOSURE BULLETIN, IBM CORP. NEW YORK, US, vol. 37, no. 2B, 1 February 1994 (1994-02-01), pages 387-389, XP000433887 ISSN: 0018-8689 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2468007A (en) * 2009-02-20 2010-08-25 Advanced Risc Mach Ltd Data processing apparatus and method dependent on streaming preload instruction.
US8516230B2 (en) 2009-12-29 2013-08-20 International Business Machines Corporation SPE software instruction cache
US9459851B2 (en) 2010-06-25 2016-10-04 International Business Machines Corporation Arranging binary code based on call graph partitioning
US9600253B2 (en) 2010-06-25 2017-03-21 International Business Machines Corporation Arranging binary code based on call graph partitioning
US8631225B2 (en) 2010-06-25 2014-01-14 International Business Machines Corporation Dynamically rewriting branch instructions to directly target an instruction cache location
US8713548B2 (en) 2010-06-25 2014-04-29 International Business Machines Corporation Rewriting branch instructions using branch stubs
US8782381B2 (en) 2010-06-25 2014-07-15 International Business Machines Corporation Dynamically rewriting branch instructions in response to cache line eviction
US10324694B2 (en) 2010-06-25 2019-06-18 International Business Machines Corporation Arranging binary code based on call graph partitioning
US8522225B2 (en) 2010-06-25 2013-08-27 International Business Machines Corporation Rewriting branch instructions using branch stubs
US8627051B2 (en) 2010-06-25 2014-01-07 International Business Machines Corporation Dynamically rewriting branch instructions to directly target an instruction cache location
US10169013B2 (en) 2010-06-25 2019-01-01 International Business Machines Corporation Arranging binary code based on call graph partitioning
US9916144B2 (en) 2010-06-25 2018-03-13 International Business Machines Corporation Arranging binary code based on call graph partitioning
US8856452B2 (en) 2011-05-31 2014-10-07 Illinois Institute Of Technology Timing-aware data prefetching for microprocessors
US9690710B2 (en) 2015-01-15 2017-06-27 Qualcomm Incorporated System and method for improving a victim cache mode in a portable computing device
CN114860785A (zh) * 2022-07-08 2022-08-05 深圳云豹智能有限公司 缓存数据处理系统、方法、计算机设备和存储介质
CN114860785B (zh) * 2022-07-08 2022-09-06 深圳云豹智能有限公司 缓存数据处理系统、方法、计算机设备和存储介质

Also Published As

Publication number Publication date
WO2002027498A3 (fr) 2003-01-23
AU2001294856A1 (en) 2002-04-08

Similar Documents

Publication Publication Date Title
US6578111B1 (en) Cache memory system and method for managing streaming-data
US7266647B2 (en) List based method and apparatus for selective and rapid cache flushes
US7783837B2 (en) System and storage medium for memory management
US5717893A (en) Method for managing a cache hierarchy having a least recently used (LRU) global cache and a plurality of LRU destaging local caches containing counterpart datatype partitions
JP4486750B2 (ja) テンポラリ命令及び非テンポラリ命令用の共用キャッシュ構造
EP0695996B1 (fr) Système d'antémémoire à niveaux multiples
JP3618385B2 (ja) データをバッファリングする方法およびそのシステム
US6339813B1 (en) Memory system for permitting simultaneous processor access to a cache line and sub-cache line sectors fill and writeback to a system memory
CA1238984A (fr) Hierarchie de memoire cooperative
US6823428B2 (en) Preventing cache floods from sequential streams
US6782454B1 (en) System and method for pre-fetching for pointer linked data structures
US8412907B1 (en) System, method and computer program product for application-level cache-mapping awareness and reallocation
JP4447580B2 (ja) 分散共有メモリマルチプロセッサシステムのための分割疎ディレクトリ
US6782453B2 (en) Storing data in memory
US6668307B1 (en) System and method for a software controlled cache
US6516389B1 (en) Disk control device
US6584546B2 (en) Highly efficient design of storage array for use in first and second cache spaces and memory subsystems
CN1347526A (zh) 改进虚拟存储器系统中存储器访问的技术
US7047362B2 (en) Cache system and method for controlling the cache system comprising direct-mapped cache and fully-associative buffer
US6332179B1 (en) Allocation for back-to-back misses in a directory based cache
US5809526A (en) Data processing system and method for selective invalidation of outdated lines in a second level memory in response to a memory request initiated by a store operation
US6598124B1 (en) System and method for identifying streaming-data
US20060123196A1 (en) System, method and computer program product for application-level cache-mapping awareness and reallocation requests
US6961823B2 (en) Stream-down prefetching cache
WO2002027498A2 (fr) Systeme et procede d'identification et de gestion de donnees en flux

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PH PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP