KR20050084232A - Counter based stride prediction for data prefetch - Google Patents

Abstract

A prefetching system (400) includes hysteresis in the determination and modification of a stride value (412) that is used for prefetching data (130) in a sequential process. Once a stride value is determined, intermittent stride inconsistencies are ignored (322-330), and the stride value retains its prior value. When the stride inconsistencies become frequent (322-330), the stride value is modified (230). When the modified stride value becomes repetitive, the system adopts this value as the stride, and subsequent stride inconsistencies are again ignored, and the stride value thereafter retains is current value until inconsistencies become frequent.

Description

COUNTER BASED STRIDE PREDICTION FOR DATA PREFETCH}

TECHNICAL FIELD The present invention relates to the electronic field, and more particularly, to a method and system for predicting the location of the next data in a given process and easily prefetching data from the location.

Prefetch is a common technique to minimize latency in continuous processes. Data expected to be needed by the process in the future is retrieved from memory and stored in cache for future subsequent access. The cache is designed to provide access times much faster than that memory. Thus, if the process needs data and the data is in the cache, the data is provided to the process at a higher cache access rate. Conversely, if data is not in the cache, the data is not provided to the process until after a fairly slow memory access time, thereby introducing a memory access delay to the process.

There are a number of techniques that can be commonly used to easily predict the data that will be needed in the future. Such a technique is "stride prediction", in which the position of the next data item required is based on the sequence of positions of the previously accessed data item. For example, data may be stored as an array of data records or a list of data records, such as records of employees' names, addresses, social security numbers, and the like. Typically, these records are fixed length records, such that the beginning of each record is spaced apart from adjacent records in that memory by a fixed number of memory locations. An application that provides a printout or display of all employee names will, for example, continuously access such fixed number of spaced memory locations. If the application accesses the first employee name at location L from the memory and the second employee name at location L + S, the application accesses the data located at location L + S + S in memory and then Employee names will be obtained. If such data at location L + S + S is retrieved from memory and stored in fast cache memory before such access is requested, the third employee name may be provided to the application faster than the first and second employee names. have.

1 is an exemplary flow diagram of a prior art stride prediction prefetch process. At 110, the prefetch process runs concurrently with the application generating an access request for a new data item. Although not shown, such prefetch may also be referred to as part of a global scheme that includes multiple prefetch algorithms, and may be selectively executed depending on factors such as proximity of sequential data access. Typically, a processor maintains a Stride Prediction Table (SPT) that records information that includes the interval between sequential accesses associated with the accesses executed for that memory (herein referred to as strides of accesses). do. Such a stride prediction table is typically configured to record multiple sets of information related to accesses performed to maintain multiple potential stride tracks. For convenience and understanding, the invention contemplates the use of a paradigm of a single set of information used to keep track of a single stride between associated memory accesses. Those skilled in the art will appreciate that the principles devised herein can be applied directly to the conventional use of a stride prediction table that includes a plurality of sets of information related to multiple strides.

The current stride at 110 is determined at 120 by the difference between the address of the prior / old access and the address of the newly requested access. Assuming that the next requested address will be evenly spaced like the previous access, at 130, a prefetch is executed to retrieve data of addresses at equal intervals from the current / new address. At 140, the old address is replaced with a new address in preparation for the next data access, and at 150 the prefetch routine ends. By initiating a prefetch for the next candidate data at the time of the request access to the new data, if the application initiates an access request for this data, the prefetched data will appear in the fast cache. If the next candidate data is not the next requested data, the cache will not contain the next requested data, and memory access will be requested.

However, the prefetch process of FIG. 1 starts prefetching of all accesses to new data, regardless of the assumption that the next candidate request data will be evenly spaced from the previously requested data. This causes significant memory access traffic, which may act to greatly reduce the efficiency of the memory access. FIG. 2 shows an exemplary flow diagram for an improved prior art stride prediction prefetch process in which a predetermined prefetch is initiated when or only if two consecutive accesses represent the same stride. In this embodiment, at 220, the stride determined for the new access request is compared with the previously determined stride. At 130, if the new stride is the same as the previous stride, the likelihood that the next stride is equal to these two strides is high enough that prefetching is guaranteed. If the new stride is different from the old stride, then at 230, the old stride is replaced with the new stride in preparation for the next cycle. Those skilled in the art can extend these two-in-a-row criteria to prefetching into columns 3, 4, etc. to balance the possibility of having the next request data in the cache with excess memory access traffic. You will see that it can be maintained.

1 is an exemplary flow diagram of a prior art stride prediction prefetch process;

2 is an exemplary flow diagram of an alternative prior art stride prediction prefetch process;

3 is an exemplary flow diagram of a stride prediction prefetch process in accordance with the present invention;

4 is an exemplary block diagram of a stride prediction prefetch system in accordance with the present invention.

In the entire drawing, like reference numerals indicate similar or corresponding features or functions.

It is an object of the present invention to improve the likelihood to prefetch data that will subsequently be accessed by an application. It is another object of the present invention to provide an efficient prefetch technique that is well suited for hardware implementation.

The above and other objects of the present invention are achieved by a prefetch system that includes hysteresis in determining and correcting stride values used to prefetch data in a continuous process. Once the stride value is determined, intermittent stride inconsistency is ignored and the stride value retains its previous value. If stride inconsistency is frequent, the stride value is corrected. If the corrected stride value is repetitive, the system adopts this value as the stride and the subsequent stride inconsistency is ignored again, and the stride value held subsequently until the inconsistency becomes frequent becomes the current value.

It is assumed that the regular stride pattern in memory access may include intermittent data access that does not follow the stride pattern. For example, a nested loop structure includes an inner loop that forms a cycle through a list of data, and an outer loop that presets one or more variables used for each cycle of the inner loop. The inner loop that forms the cycle through the data list will represent a fixed stride. However, at each restart of the inner loop, memory access is made at the beginning of the list, while previous access is made at the end of the list. However, the span between the end of the list and the beginning of the list will not correspond to the stride of the inner loop. Additionally, data access by the outer loop at the beginning or end of the loop will create spans between the accesses that do not correspond to the stride of the inner loop. Another intermittent break of the stride involves the processing of data organized into a multidimensional array. Typically, data is processed for a given region along one dimension, indexes for other dimensions are incremented, and data of the next dimensioned index is processed for a given region along the first dimension. The stride along the region of the first dimension will generally be fixed, but the increase of the index for the next dimension is likely to result in an access with a predetermined span from the previous access, which does not match the stride along the first dimension.

In a typical stride prediction process, each time the stride is "disconnected" or interrupted, the process of determining the stride is repeated to determine a new stride. During the time that the new stride is determined, no prefetch occurs, and the application is delayed by memory access at each restart of the inner loop or by each increase of the high level index during processing of the multidimensional array.

According to the present invention, the stride value is maintained during the intermittent break of the stride. Performing data prefetch of the current stride depends on the number of strides with the same value within a given number of memory accesses, and adjusting the prefetch value depends on the generation of a stride with a number of different values. . In a simple example, prefetching of data is performed whenever two of the three accesses have the same value in succession, and correction of the stride value is performed whenever the two accesses have different strides in succession.

3 shows an exemplary flow diagram of a stride prediction prefetch process in accordance with the present invention. In this example, the "count" parameter is used to count the number of consecutive strides with the same value up to the selected maximum. In a preferred embodiment of the present invention, this count parameter is used to distinguish between an intermittent break in the stride and a substantial and continuous change in the stride. Those skilled in the art having regard to what is disclosed herein will appreciate that unique parameters may be employed to count the number of consecutive uneven strides.

At 220, after determining the current stride between the new access address and the previous access address, the process of the present invention compares the current stride with the previous stride. If the current stride is the same as the previous stride, the count is incremented at 326, otherwise the current count is decreased at 322. In a preferred embodiment, the count is limited in value from zero to the maximum count. Blocks 324 and 328 clip so that the increment or decremented count from blocks 322 and 326 is within this limit.

At 330, the count is compared to the upper limit UL and the lower limit LL, where preferably the lower limit LL is less than the upper limit. In this example, the upper limit UL corresponds to the number of even stride occurrences required to ensure prefetch from the next stride predicted access location. If the count equals or exceeds this upper limit UL, at 130, data prefetch is performed from the address corresponding to the addition of the current address and the current stride. For example, when the upper limit UL is a value of 2, prefetching is not executed unless the same stride value of two rows is generated initially. If the upper limit is a value of 3, the prefetch is not executed unless the same stride value of three columns is generated. Subsequent strides through 326,328 then continue to increase the count to the maximum. In the preferred embodiment, the maximum value is chosen as the number of consecutive equal values of stride required to conclude that the current stride value is "reliable".

Each time a different stride occurs, the count is reduced at 332. Thus, the difference between the maximum count and the current count value corresponds to a measure of "unreliability" of the current stride value. At 230, the lower limit is selected as the value of the current count, which constitutes a sufficient measure of instability to ensure a change to the current stride value. In general, as shown in FIG. 3, at 230, if the current stride value is considered unreliable, prefetch is not executed at 130. In a similar manner, at 130, if the current stride value is reliable enough to ensure prefetching, then at 230 the correction of the current stride value is not performed. Therefore, the lower limit LL is set lower than the upper limit, and is generally set to UL-1. Optionally, as indicated by the broken line connection between the test blocks 330, 140, the lower limit LL, for the upper limit UL, is not sufficient for the current stride's estimated reliability to ensure prefetch at 130 (count <UL). ), It is chosen so that the estimated unreliability is not sufficient to ensure correction of the current stride at 230 (count &gt; LL).

4 is an exemplary block diagram of a stride prediction prefetch system in accordance with the present invention. A fetch controller 430 prefetches data from the memory 450 into the cache based on the contents of the control register 410 and the data access request from the processor 420. The address of the requested data is used to determine whether the application requests data in an iterative manner, as described above, or represents a consistent stride or span of location between memory accesses. Control register 410 includes the address of previous data access 416, previous stride 412, and counter 414. As discussed above, two counters may be used instead of a single counter 414 to maintain independent counts of identical stride access and different stride access. The address of the currently requested data from the processor 420 is compared with the previous address 416 to determine the current stride. If the current stride corresponds to the previous stride 412, the counter 414 is incremented, otherwise it is decremented. Depending on the increment or decremented value of the counter 414, the patch controller determines whether to start prefetching data into the cache 460 at the next scheduled location in the memory 450. If the subsequent data request is for data prefetched in cache 460, cache 460 provides the data directly to the process, thereby preventing delays incurred by retrieving data from memory 450. do.

Depending on the increment or decremented value of the counter 414, the patch controller 430 determines whether to correct the stride value 412 as described above. Instead of correcting the stride value based on the occurrence of one different stride, as in conventional systems, the stride prediction prefetch system 400 is determined by determining whether to correct the stride value 412 based on a count according to the number of different strides. ) Is insensitive to intermittent breaks in the stride.

The foregoing merely illustrates the principles of the invention. Those skilled in the art will appreciate that various structures may be devised which employ the principles of the invention, although not explicitly described or illustrated in the present invention, within the spirit and scope of the following claims.

Claims (12)

As a method of prefetching data from memory 450 to cache 460: Determining (326) a first measure of identical stride memory access based on a number of identical stride memory accesses and previous stride values 412; Determining (322) a second measure of different stride memory access based on the plurality of different stride memory accesses and previous stride values 412; Prefetching data from the memory 450 based on a first measure (130), Correcting 230 the previous stride value 412 based on the second measure, Data prefetch method. The method of claim 1, Determining the first and second scales, Maintaining a count that is incremented 326 for each same stride memory access and a decrease 322 for each different stride memory access, executing the prefetch, and performing a correction based on the count 414 due to, Performed, Data prefetch method. The method of claim 2, The step 130 of executing the prefetch occurs when the count 414 is greater than or equal to the upper limit (330), The step 230 of performing the correction occurs when the count 414 is below the lower limit (330), Data prefetch method. The method of claim 3, wherein The count 414 is limited to a maximum of 3, the upper limit is 2 and the lower limit is 1, Data prefetch method. As the prefetch system 400, A control register 410 configured to include at least one measure 414 corresponding to the consistency of the stride value between the requested memory accesses; A prefetch controller 430 configured to prefetch data from the memory 450 to the cache 460 based on the measure of consistency, The consistency of the stride value depends on the comparison of the current stride and the previous stride value 412, and the prefetch controller 430 corrects the previous stride value 412 based on a measure of inconsistency. Further configured, wherein the inconsistency measure is based on a number of different strides between requested memory accesses, Prefetch System 400. The method of claim 5, The consistency measure and inconsistency measure correspond to a count 414 that is incremented up to a maximum count for each same stride requested memory access and is reduced to a minimum count for each different stride requested memory access, Prefetch System 400. The method of claim 6, The prefetch controller 430 is configured to prefetch data when the count 414 is greater than or equal to an upper threshold level and to correct a previous stride value 412 when the count 414 is less than or equal to a lower threshold level. Prefetch System 400. The method of claim 7, wherein The maximum count is 3, the upper threshold level is 2, the lower threshold level is 1, and the minimum count is 0, Prefetch System 400. As a processing system: A memory 450 configured to provide access to data based on the access address; A cache 460 operatively coupled to the memory 450, the cache 460 configured to store data accessed from the memory 450 and to quickly access the data; Operatively coupled to the memory 450 and the cache 460 to provide the access address, if the data is in the cache 460, receive the data from the cache 460, and the data is cache 460. Processor 420 configured to receive the data from memory 450, if not within; Operatively coupled to the processor 420, the memory 450, and the cache 460, based on the access address and the predicted stride value, the cache 460 from the memory 450. A patch controller 430 configured to transmit data to the The patch controller 430 is configured to maintain a measure of stride consistency based on repeated occurrences of the same stride value, and maintain a measure of stride inconsistency based on the repeated occurrence of different stride values, The patch controller 430 transmits data based on the measure of stride consistency and corrects a predicted stride value based on the measure of stride inconsistency, Processing system. The method of claim 9, The measure of stride coherence and the measure of stride inconsistency are based on a count 414 that is incremented to a maximum count for each same stride requested memory access and reduced to a minimum count for each different stride requested memory access. doing, Processing system. The method of claim 10, The patch controller 430, Transmit data when the count 414 is above the upper threshold level and correct the predicted stride value when the count 414 is below the lower threshold level, Processing system. The method of claim 11, The maximum count is 3, the upper threshold level is 2, the lower threshold level is 1, and the minimum count is 0, Processing system.