EP4662554A1

EP4662554A1 - Distance-based memory disambiguation prediction

Info

Publication number: EP4662554A1
Application number: EP23715314.3A
Authority: EP
Inventors: Vineeth Thamarassery Mekkat; Leigang KOU
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2023-03-10
Filing date: 2023-03-10
Publication date: 2025-12-17
Also published as: CN120858343A; WO2024191406A1; KR20250141825A; JP2026507378A

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a store-to-load forwarding process on a processor. In one aspect, a system comprises a processor that is configured to receive a load instruction having a load value. The processor determines that the load instmction has an entry in a memory disambiguation table (MDPT) based on a program counter of the load instruction. The processor obtains a store queue delta value from the entry in the MDPT. The store queue delta value represents a number of stores in a store queue before an entry corresponding to a conflicting store instruction that has previously conflicted with the load instruction. The processor then creates a dependency between the load instruction that has an entry in the MDPT and the conflicting store instruction.

Description

DISTANCE-BASED MEMORY DISAMBIGUATION PREDICTION

BACKGROUND

This specification relates to techniques for performing a predictive memory disambiguation process.

Loading from memory and storing to memory are performance critical memory operations for processors. A store queue of a load store unit (LSU) can support Out of Order (OOO) execution of memory operations that include a processor performing the memory operations in a different order than in the original program. In this case, the processor tracks inflight store entries in a processing window using the store queue in order to ensure correctness of data written to memory by store instructions as part of 000 execution.

Memory disambiguation violations are a major source of inefficiency for 000 execution. Memory disambiguation violations can arise in part from arithmetic address formats supported by modem processors. Consider the following example assembly code shown in Table 1 having a store instruction followed by a load instruction.

TABLE 1

The store instruction on line 1 stores the value in register 1 to an address stored by register 4 plus an offset of 4. The load instruction on line 3 loads into register 2 a value at an address stored by register 3 plus an offset of 8.

A memory disambiguation violation can occur if the value in R4 + 4 turns out to be the same as the value in R3 + 8. But from the program code itself, it’s not clear whether or not this is the case. Rather, it’s only during program execution that the addresses get disambiguated and the processor discovers out of order violations resulting from that disambiguation. If, for example, the load instruction completed while the store instruction was still in the store queue, the load instruction would need to be re-executed, as well as any other instructions that depended on the data read by the load instruction. SUMMARY

This specification describes a system for performing a predictive memory disambiguation process, which can significantly improve processor efficiency by preventing repeating memory disambiguation (MD) violations. As described above, a memory disambiguation violation occurs if a load instruction reads data before a previous store instruction for the same memory address has completed, which can cause the processor to load stale data from memory. When this happens, the load instruction needs to be re- executed, as well as any other instructions that depended on the data read by the load instruction.

To address this problem, the system can maintain a memory disambiguation table (MDPT) that stores delta values for loads that have previously incurred MD violations. Then, the next time the load instruction is encountered, the processor can introduce a dependency between the load instruction and a store instruction indicated by the delta value so that the load only executes after the store on which it depends.

Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

The techniques that use the memory disambiguation table described in this specification provide the efficiency gains of preventing MD violations while being simpler to implement than some prior techniques to predict memory disambiguation. For example, tracking program counters of both load instructions and store instructions involved in the memory disambiguation violation can result in a much higher cost of implementation in hardware. Some other conventional techniques track only the program counter of the load instruction involved in the memory disambiguation violation. Though tracking the program counter of the load instruction is easier to implement in a processor, it is far less accurate because the processor is dependent on older store instructions to predict the memory disambiguation violation, which can result in high performance overhead due to inaccuracies in predicting memory disambiguation.

In contrast, the techniques described in this specification can achieve high accuracy by leveraging the delta between the load instruction and the store instruction to perform a predictive memory disambiguation. A processor can populate a memory disambiguation table (MDPT) with infomiation about the delta (e.g., distance) between memory locations of the memory instructions, and the processor can determine that a received load instruction has an entry in the MDPT. The processor can then create a dependency between the receive load instruction and a corresponding store instruction that has previously conflicted with the load instruction, and the processor can predict a memory disambiguation violation. The processor can use the prediction as part of two-pass mechanism to prevent a memory disambiguation violation. Thus, by tracking the program counter of the load instruction and the distance information using the MDPT table, the processor can increase store queue efficiency by effectively predicting and preventing memory disambiguation.

The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example system.

FIG. 2 is a flow diagram of an example process for populating the MDPT

FIG. 3 is a flow diagram of an example process for performing the predictive disambiguation process.

FIG. 4 is a flow diagram of an example process of the two-pass mechanism to prevent a memory disambiguation violation.

Like reference numbers and designations in the various drawings indicate like elements.

DETAILED DESCRIPTION

FIG. 1 shows an example system 100. The sy stem 100 is an example of a system in which the systems, components, and techniques described below are implemented.

The system 100 includes a processor 102 coupled to a main memory 118, e.g., a random-access memory (RAM), a read-only memory (ROM), etc.

The processor 102 contains a load store unit 106 and a main execution logic 104. The processor 102 can perform a predictive memory disambiguation process by using the load store unit 106 to process a load instruction 112 received from the main execution logic 104. The load instruction 112 includes an instruction to extract or read data to move data from the main memory 118 to the processor 102.

The load store unit 106 includes a store queue 108.

The store queue 108 can hold in-flight store instructions from a program. In some examples, the load instruction 112 also includes an address corresponding to an entry in the store queue 108 associated with the data. The store queue 108 keeps track of pending load instructions 112 and store instructions within a processing window.

The processor 102 can perform a predictive memory' disambiguation process for one or more of the entries of the load instruction 112 as part of a two-pass mechanism to prevent memory disambiguation violations. The two-pass mechanism includes a first pass to determine whether to execute the load instruction 112 based on the execution of a conflicting store instruction and a second pass to determine whether the predictive memory disambiguation process succeeded by avoiding a memory disambiguation violation. In particular, the processor 102 can perform the predictive memory disambiguation process for one or more of the entries of the load instruction 112 using a memory disambiguation prediction table (MDPT) 110.

The processor 102 can populate the MDPT 110 prior to performing the predictive memory disambiguation process by tracking a program counter of the load instruction 112, as described in further detail with reference to FIG. 2. The program counter is a value that represents the memory address of an instruction. The MDPT 110 includes indices corresponding to a program counter of received load instructions, corresponding store queue delta values, and an age of each load instruction. The store queue delta value 116 represents a number of stores (e.g., entries of one or more store instructions) in the store queue 108 that were added after a store queue entry corresponding to a previously conflicting store, as described in further detail with reference to FIGs. 2, 3, and 4. In other words, the delta value represents how many store instructions were added to the store queue after the store instruction that conflicts with the load instruction.

To perform the predictive disambiguation process, the processor 102 receives the load instruction 112 from the main execution logic 104. The load instruction 112 has a program counter and an address that represents the location from which to read data.

The processor 102 determines that that the load instruction has 112 has an entry in the MDPT 110 based on the program counter of the load instruction 112, and the processor obtains a store queue delta value 116 from the MDPT 110. The processor 102 then creates a dependency between the load instruction 112 and the conflicting store instruction indicated by the delta value. This process is described in further detail with reference to FIG. 3.

The processor 102 uses the dependency as part of a two-pass mechanism to prevent a memory disambiguation violation. The system detemrines whether the memory disambiguation process succeeded as part of the two-pass mechanism. If the memory disambiguation process succeeded, the processor 102 can retire the load instruction 112 along with the retired stores 114. If the memory disambiguation process failed, the processor 102 rolls back and re-executes the load instruction 112 and the conflicting store instruction using the dependency information.

Thus, the processor 102 creates the dependency to avoid repeat memory disambiguation violations, which can greatly increase efficiency of the processor 102.

FIG. 2 is a flow diagram of an example process for populating the MDPT. In general, a processor populates the MDPT table by identifying memory disambiguation violations. The example process can be performed by a processor appropriately configured in accordance with this specification, , e.g., the processor 102 of FIG. 1.

The processor executes a store instruction and adds the store instruction to a store queue (202).

The processor determines that a younger load instruction with a same address as the store instruction executed prior to the store instruction completing (204). This situation means that the load value read by the load instruction is possibly stale. The load instruction and store instruction that access the same address (e.g., the same memory location) can have a stable distance relationship that is encountered many times, particularly when, as is often the case, code segments are executed repeatedly by a processor. Thus, the processor can use the MDPT to track the conflicting store instruction by tracking the program counter of the load instruction.

The processor determines a store queue delta value between the load instruction and the conflicting store instruction (206). The store queue delta value represents a number of entries in the store queue that have been added between the load instruction and the conflicting store instruction. For example, the processor can count from a tail pointer of the store queue representing the new est entry sequentially through older stores until reaching the conflicting store queue. In some implementations, the processor computes a difference in store queue identifiers. A store queue identifier is an index in the store queue that represents the order in which the store instructions were added. The processor can thus compute a difference between the store queue identifier of the newest entry and the store queue identifier of the conflicting store instruction.

For example, if the conflicting store instruction has a store queue identifier of 2, and there are two older entries corresponding to two other store instructions (e.g., one of the store instructions has a store identifier of 3 and the other store instruction has a store identifier of 4), the processor determines that the store queue delta value is 3. In other words, the store queue delta value indicates that the conflicting store instruction is the 3rd newest entry in the store queue. Alternatively, the processor can compute a store queue delta value of 2 to represent that the conflicting store instruction is two entries beyond the newest entry.

In some implementations, the store queue delta value represents a distance value between a re-order buffer identifier of the load instruction and a re-order buffer identifier of the conflicting store instruction. The re-order buffer is a buffer that supports the processor in performing the memory operations in order. The re-order buffer identifier is an index that represents the original order of the execution of the load instruction or the store instruction.

The processor can populate the MDPT with a value of the program counter of the load instruction and the store queue delta value (208). In some implementations, the processor also stores an age of the load instruction. The age can be represented as a time stamp of a last instance that the load instruction was added to the MDPT. Then, when the MDPT is full and the processor needs to select a victim entry to replace, the processor can select an oldest entry according to the age attribute.

For example, Table 2 shows how a store instruction associated with a memory disambiguation violation has a load program counter value of 0x403200, a store queue delta value of 3, and a timestamp of 1678406405.

TABLE 2

In some examples, the processor can populate the MDPT with a hash value of the value of the program counter of the load instruction.

FIG. 3 is a flow diagram of an example process for performing the predictive disambiguation process. For convenience, the process 300 will be described as being performed by a processor. For example, a processor, e.g., the processor 102 of FIG. 1, appropriately configured in accordance with this specification, can perform the process 300.

The processor receives a load instruction with a load value (302).

The processor determines, based on a program counter of the load instruction, that the load instruction has an entry in the MDPT (304). The processor looks up the program counter from the MDPT. For example, the processor can look up the value 0x403200, as shown in Table 2.

The processor obtains, from the entry in the MDPT, a store queue delta value (306). For example, the processor can obtain the store queue delta value of 3 mapped to the program counter value 0x403200, as shown in Table 2.

The processor then creates a dependency between the load instruction and the conflicting store instruct on (308).

In particular, the processor determines a predicted store queue identifier of the store instruction based on the store queue delta value. By identifying the conflicting store instruction, the processor can create the dependency between the load instruction and the conflicting store instruction.

The dependency between the load instruction and the conflicting store instruction can result in preventing a memory disambiguation violation. In particular, because of the dependency, the processor executes the load instruction only after the conflicting store instruction has completed (e.g., after the processor has completed the execution of the store instruction in the store queue).

FIG. 4 is a flow diagram of an example process of the two-pass mechanism to prevent a memory disambiguation violation. For convenience, the process 400 will be described as being performed by a processor. For example, a processor, e.g., the processor 102 of FIG. 1, appropriately configured in accordance with this specification, can perform the process 400.

The processor reads the load instruction at instruction dispatch (402). The processor looks up a corresponding segment of code in the program that implements the load instruction while the instructions of the program are in order (e.g., before the processor has performed OOO execution).

The processor determines whether the load instruction is present in the MDPT (404). The processor can use the program counter of the load instruction to look up the entry of the load instruction in the MDPT.

If the processor determines that the load instruction is not present in the MDPT (404), the processor can execute the load instruction normally (branch to 408).

If the processor determines that load instruction is present in the MDPT (404), the processor determines whether the corresponding store instruction has completed (406). The processor can first use the store queue delta value to identify the corresponding store instruction in the store queue. As the store queue delta value is indicative of how many store instructions were added to the store queue after the conflicting store instruction and before the load instruction was executed, the processor can count the number of entries from a current head of the store queue until reaching an entry corresponding to the store queue delta value.

If the processor determines that the corresponding store instruction has not completed (406), the processor creates a load instruction dependency (410). The load instruction dependency ensures that the load instruction will not be executed until the corresponding store instruction has completed.

The load instruction dependency can be implemented in a number of ways. For example, the processor can reissue the load instruction to the load store unit. The processor can recirculate the load instruction until the corresponding store has completed. In particular, the processor issues the load instruction into the load store unit, and the processor will refrain from executing the load instruction until the processor has executed the corresponding store instruction.

Alternatively or in addition, the load store unit can block the load instruction until the corresponding store instruction has completed. For example, the processor can wake the load instruction only upon the corresponding store instruction completing.

One advantage of the techniques described in this specification is that the load instruction is blocked only by the load-store unit rather than the entire issue queue of the processor. Therefore, other instructions are not blocked indefinitely by the load instruction being blocked.

When the store instruction completes, the processor executes the load instruction (408). The load instruction is guaranteed to read updated data now that the corresponding store instruction has completed.

After executing the load instruction (408), the processor determines whether there was a memory disambiguation failure (412). In other words, the processor determines whether an older store was fetching data from the same address while the load instruction executed. This can happen, for example, the first time that the load instruction is encountered such that the load instruction is not yet represented in the MDPT. On the other hand, if the processor created the load instruction dependency, there should not be an MD violation.

If the processor determines that there was a memory disambiguation failure , the processor undoes and re-executes the load instruction and updates the MDPT (416). The processor can undo any instructions that were dependent on the possibly stale data read by the load instruction and re-executes the load instruction and any instructions that depended on it. The processor can then populate the MDPT according to the MD failure event. For example, the processor can create a new entry in the MDPT that includes the program counter of the load instruction and a store queue delta value for the conflicting store instruction. Then, the next time that the load instruction is encountered, the processor can handle the load instruction by introducing a dependency rather than executing the load instruction right away.

If the processor determines that there was no MD failure, the processor retires the load instruction (414).

Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

In addition to the embodiments described above, the following embodiments are also innovative: Embodiment 1 is a method for performing a predictive memory disambiguation process on a processor, the method comprising: receiving a load instruction having a load value; determining, based on a program counter of the load instruction, that the load instruction has an entry in a memory disambiguation table (MDPT); obtaining, from the entry in the MDPT, a store queue delta value that identifies a a conflicting store instruction that has previously conflicted with the load instruction; and creating a dependency between the load instruction having an entry in the MDPT and the conflicting store instruction.

Embodiment 2 is the method of embodiment 1 , wherein the store queue delta value represents a number of entries in a store queue added after the conflict store instruction.

Embodiment 3 is the method of embodiment 1, wherein the store queue delta value represents a difference between a re-order buffer identifier of the load instruction and a reorder buffer identifier of the conflicting store instruction.

Embodiment 4 is the method of any one of embodiments 1-3, wherein the dependency causes the processor to execute the load instruction after the conflicting store instruction.

Embodiment 5 is the method of any one of embodiments 1 -4, wherein creating a dependency between the load instruction having an entry in the MDPT and the conflicting store instruction causes the load instruction to complete after the conflicting store instruction.

Embodiment 6 is the method of any one of embodiments 1-5, further comprising: executing a store instruction having a store value; and determining that a load value of an older load instruction having a same address are present in the store queue prior to executing the store instruction; determining a store queue delta value between the load value and the store value; and populating the MDPT with a value of the program counter of the load instruction, the store queue delta value between the load value and the store value, and an age of the load value.

Embodiment 7 is the method of embodiment 6, further comprising: undoing the load instruction; and re-executing the load instruction

Embodiment 8 is the method of any one of embodiments 1-7, wherein creating the dependency comprises: blocking the load instruction until the store instruction completes. Embodiment 9 is the method of any one of embodiments 1-7, wherein creating the dependency comprises reissuing the load instruction until the store instruction completes.

Embodiment 10 is the method of claim any one of embodiments 1-9, further comprising: determining that the predictive memory disambiguation process succeeded; and retiring the load value of the load instruction from the store queue.

Embodiment 11 is a system comprising a processor, wherein the processor is configured to perform the method of any one of embodiments 1-10.

Embodiment 12 is one or more storage media storing instructions that are operable, when executed by data processing apparatus, to cause the data processing apparatus to perform the method of any one of embodiments 1-10.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, vanous features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

What is claimed is:

Claims

1. A method for performing a predictive memory disambiguation process on a processor, the method comprising: receiving a load instruction having a load value; determining, based on a program counter of the load instruction, that the load instruction has an entry in a memory disambiguation table (MDPT); obtaining, from the entry in the MDPT, a store queue delta value that identifies a a conflicting store instruction that has previously conflicted with the load instruction; and creating a dependency between the load instruction having an entry in the MDPT and the conflicting store instruction.

2. The method of claim 1 , wherein the store queue delta value represents a number of entries in a store queue added after the conflict store instruction.

3. The method of claim 1, wherein the store queue delta value represents a difference between a re-order buffer identifier of the load instruction and a re-order buffer identifier of the conflicting store instruction.

4. The method of any one of claims 1-3, wherein the dependency causes the processor to execute the load instruction after the conflicting store instruction.

5. The method of any one of claims 1-4, wherein creating a dependency between the load instruction having an entry in the MDPT and the conflicting store instruction causes the load instruction to complete after the conflicting store instruction.

6. The method of any one of claims 1-5, further comprising: executing a store instruction having a store value; and determining that a load value of an older load instruction having a same address are present in the store queue prior to executing the store instruction; determining a store queue delta value between the load value and the store value; and populating the MDPT with a value of the program counter of the load instruction, the store queue delta value between the load value and the store value, and an age of the load value.

7. The method of claim 6, further comprising: undoing the load instruction; and re-executing the load instruction.

8. The method of any one of claims 1-7, wherein creating the dependency comprises: blocking the load instruction until the store instruction completes.

9. The method of any one of claims 1-7, wherein creating the dependency comprises reissuing the load instruction until the store instruction completes.

10. The method of claim any one of claims 1-9, further comprising: determining that the predictive memory disambiguation process succeeded; and retiring the load value of the load instruction from the store queue.

11. A system comprising a processor, wherein the processor is configured to perform the method of any one of claims 1-10.

12. One or more storage media storing instructions that are operable, when executed by data processing apparatus, to cause the data processing apparatus to perform the method of any one of claims 1-10.