KR20230160893A - Select and run wavefront - Google Patents

Abstract

Techniques for implementing wavefronts are provided. The techniques include performing a first identification, including identifying that sufficient processing resources exist to jointly execute the first set of instructions within a processing lane at a first time to issue instructions for execution; In response to the first identification, executing a first set of instructions together; At a second time for issuing instructions for execution, performing a second identification, including identifying that instructions for which there are sufficient processing resources for execution together within the processing lane are not available; and in response to the second identification, executing the instruction independently of any other instructions.

Description

Select and run wavefront

This application claims the benefit of pending U.S. regular patent application Ser. No. 17/219,775, entitled “WAVEFRONT SELECTION AND EXECUTION,” filed March 31, 2021. The entire contents of are hereby incorporated by reference.

Graphics processing units contain parallel processing elements that execute shader programs in a highly parallel manner. Improvements to the execution efficiency of shader programs are constantly being made.

A more detailed understanding may be obtained from the following description given by way of example in conjunction with the accompanying drawings.
1 is a block diagram of an example device in which one or more features of the present disclosure may be implemented.
Figure 2 is a block diagram of the device of Figure 1, illustrating additional details.
3 is a block diagram illustrating a graphics processing pipeline, according to an example.
4 illustrates details of a SIMD unit, according to an example.
5 is a flow diagram of a method for executing instructions, according to an example.

Techniques for implementing wavefronts are provided. The techniques include performing a first identification, including identifying that sufficient processing resources exist to jointly execute the first set of instructions within a processing lane at a first time to issue instructions for execution; In response to the first identification, executing a first set of instructions together; At a second time for issuing instructions for execution, performing a second identification, including identifying that instructions for which there are sufficient processing resources for execution together within the processing lane are not available; and in response to the second identification, executing the instruction independently of any other instructions.

1 is a block diagram of an example device 100 in which one or more features of the present disclosure may be implemented. Device 100 may include, for example, a computer, gaming device, handheld device, set-top box, television, mobile phone, or tablet computer. Device 100 includes a processor 102, memory 104, storage 106, one or more input devices 108, and one or more output devices 110. Device 100 may also optionally include input driver 112 and output driver 114. It is understood that device 100 may include additional components not shown in FIG. 1 .

In various alternatives, processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU located on the same die, or one or more processor cores, where each processor core is a CPU. Or it could be a GPU. In various alternatives, memory 104 is located on the same die as processor 102, or is located separately from processor 102. Memory 104 includes volatile or non-volatile memory, such as RAM (RAM), dynamic RAM, or cache.

Storage 106 includes fixed or removable storage, such as a hard disk drive, solid state drive, optical disk, or flash drive. Input devices 108 may include, without limitation, a keyboard, keypad, touch screen, touch pad, detector, microphone, accelerometer, gyroscope, biometric scanner, or network connection (e.g., transmission of wireless IEEE 802 signals and/or or a wireless local area network card for reception). Output devices 110 may include, without limitation, a display, speaker, printer, haptic feedback device, one or more emitters, antennas, or a network connection (e.g., a wireless local area network for transmitting and/or receiving wireless IEEE 802 signals). card).

Input driver 112 communicates with processor 102 and input devices 108 and allows processor 102 to receive input from input devices 108. Output driver 114 communicates with processor 102 and output devices 110 and allows processor 102 to send output to output devices 110 . Note that input driver 112 and output driver 114 are optional components, and device 100 will operate in the same manner even if input driver 112 and output driver 114 are not present. Output driver 116 includes an accelerated processing device (“APD”) 116 coupled to display device 118 . The APD receives compute commands and graphics rendering commands from processor 102, processes these compute and graphics rendering commands, and provides pixel output to display device 118 for display. As described in more detail below, APD 116 includes one or more parallel processing units to perform calculations according to the single-instruction-multiple-data (“SIMD”) paradigm. Accordingly, although various functions are described herein as being performed by or in connection with APD 116, in various alternatives, functions described as being performed by APD 116 may additionally or alternatively be performed by a host processor ( For example, it is not driven by processor 102 but is performed by other computing devices with similar capabilities to provide graphical output to display device 118. For example, it is contemplated that any processing system that performs processing tasks according to the SIMD paradigm can perform the functionality described herein. Alternatively, it is contemplated that a computing system that does not perform processing tasks according to the SIMD paradigm would perform the functionality described herein.

2 is a block diagram of device 100, illustrating additional details related to execution of processing tasks in APD 116. Processor 102 maintains, in system memory 104, one or more control logic modules for execution by processor 102. The control logic module includes an operating system 120, drivers 122, and applications 126. These control logic modules control various aspects of the operation of processor 102 and APD 116. For example, operating system 120 communicates directly with the hardware and provides the hardware with an interface to other software running on processor 102. Driver 122 provides, for example, an “application programming interface” (“API”) to software running on processor 102 (e.g., applications 126) to access various functions of APD 116. The operation of the APD 116 is controlled by providing Driver 122 also includes a just-in-time (JIT) compiler that compiles programs for execution by processing components of APD 116 (e.g., SIMD units 138, discussed in more detail below). do.

APD 116 executes commands and programs for selected functions, such as graphical operations and non-graphical operations that may be suitable for parallel processing. APD 116 may be used to perform graphics pipeline operations, such as pixel operations, geometry calculations, and render images to display device 118 based on commands received from processor 102. APD 116 also executes compute processing operations not directly related to graphics operations, such as video, physics simulations, computational fluid dynamics, or other tasks, based on commands received from processor 102.

APD 116 includes a compute unit 132 that includes one or more SIMD units 138 that perform operations at the request of processor 102 in a parallel manner according to the SIMD paradigm. The SIMD paradigm is a paradigm in which multiple processing elements share a single program control flow unit and program counters and thus execute the same program, but may execute that program with different data. In one example, each SIMD unit 138 includes 16 lanes, where each lane executes the same instructions at the same time as the other lanes within the SIMD unit 138 but may execute those instructions with different data. Lanes can be switched off speculatively when not all lanes need to execute a given instruction. Predictions can also be used to execute programs with branching control flow. More specifically, for programs with conditional branches or other instructions whose control flow is based on computations performed by individual lanes, prediction of lanes corresponding to control flow paths that are not currently executing, and different control Serial execution of flow paths enables arbitrary control flow.

The basic unit of execution in compute unit 132 is the work-item. Each work-item represents a single instantiation of a program to be executed in parallel on a specific lane. Work-items may be executed simultaneously as “wavefronts” on a single SIMD processing unit 138. One or more wavefronts are included in a “work group” that contains a set of work-items designated to execute the same program. A task group can be executed by executing each of the wavefronts that make up the task group. In alternatives, the wavefronts are executed sequentially on a single SIMD unit 138, or partially or completely in parallel on different SIMD units 138.

Command processor 136 performs operations related to scheduling various task groups on different compute units 132 and SIMD units 138. Typically, command processor 136 receives commands from an entity such as processor 102, where the commands direct APD 116 to perform tasks such as rendering graphics, executing general-purpose shaders, etc.

The parallelism provided by compute unit 132 is suitable for graphics-related operations such as pixel value calculations, vertex transformations, and other graphics operations. Accordingly, in some cases, graphics pipeline 134, which receives graphics processing commands from processor 102, provides computational tasks to compute units 132 for parallel execution.

Compute unit 132 may also perform computational tasks that are non-graphics related or not performed as part of the “normal” operation of graphics pipeline 134 (e.g., performed for the operation of graphics pipeline 134). It is used to perform custom operations performed to complement the processing being performed. Application 126 or other software running on processor 102 transmits programs defining such computational tasks to APD 116 for execution.

FIG. 3 is a block diagram illustrating additional details of the graphics processing pipeline 134 illustrated in FIG. 2 . The graphics processing pipeline 134 includes stages each performing a specific function. Stages represent a subdivision of functionality of graphics processing pipeline 134. Each stage is implemented, partially or entirely, as a shader program executing in programmable processing unit 202, or as fixed-function, non-programmable hardware external to programmable processing unit 202.

Input assembler stage 302 reads primitive data from a user-filled buffer (e.g., a buffer filled with requests from software executing by processor 102, such as application 126). and assembles the data into primitives for use by the rest of the pipeline. The input assembler stage 302 may generate different types of primitives based on primitive data contained in the user-field buffer. The input assembler stage 302 formats the assembled primitives for use by the rest of the pipeline.

Vertex shader stage 304 processes vertices of primitives assembled by input assembler stage 302. The vertex shader stage 304 performs various per-vertex operations such as transformation, skinning, morphing, and per-vertex lighting. Transformation operations include various operations to transform the coordinates of vertices. These operations include one or more of modeling transformations, viewing transformations, projection transformations, perspective splitting, and viewport transformations. Here, such transformations are considered to modify the coordinates or “position” of the vertices on which the transformation is performed. Other operations in the vertex shader stage 304 modify properties other than coordinates.

Vertex shader stage 304 is implemented, in part or entirely, as a vertex shader program to run on one or more compute units 132. The vertex shader program is provided by processor 102 and is based on a program pre-written by a computer programmer. Driver 122 compiles such computer programs to produce a vertex shader program having a format suitable for execution within compute unit 132.

The hull shader stage 306, tessellator stage 308, and domain shader stage 310 work together to implement tessellation, which converts simple primitives into more complex primitives by subdividing the primitives. The hull shader stage 306 generates patches for tessellation based on input primitives. The tessellator stage 308 generates a set of samples for the patch. The domain shader stage 310 calculates vertex positions for vertices corresponding to samples for patching. Hull shader stage 306 and domain shader stage 310 may be implemented as shader programs to be executed on programmable processing unit 202.

The geometry shader stage 312 performs vertex operations on a primitive basis. A variety of different operations, including operations such as point sprint scaling, dynamic particle system operations, fur-fin generation, shadow volume generation, single pass render-to-cubemap, per-primitive material swapping, and per-primitive material setup. Types of operations may be performed by geometry shader stage 312. In some cases, shader programs executing on programmable processing unit 202 perform operations on geometry shader stage 312.

Rasterizer stage 314 receives and rasterizes simple primitives and generated upstream. Rasterization involves determining which screen pixels (or sub-pixel samples) are covered by a particular primitive. Rasterization is performed by fixed-function hardware.

Pixel shader stage 316 calculates output values for screen pixels based on primitives generated upstream and the results of rasterization. Pixel shader stage 316 can apply textures from texture memory. Operations on pixel shader stage 316 are performed by shader programs executing on programmable processing unit 202.

The output merge stage 318 receives the output from the pixel shader stage 316 and merges these outputs to perform operations such as z-test and alpha blending to determine the final color for the screen pixel.

Although graphics processing pipeline 134 is illustrated as being included within APD 116, compute units that do not include graphics processing pipeline 134 (but execute shader programs, such as general-purpose compute shader programs) It should be understood that implementations of APD 116 (including 132) are contemplated by this disclosure.

As explained elsewhere herein, wavefronts include multiple work-items. SIMD units 138 execute wavefronts by executing the wavefront's work-items together in lockstep across lanes of SIMD unit 138. SIMD units 138 facilitate highly parallel execution in an efficient manner by utilizing the same instruction flow circuitry (e.g., instruction fetch and instruction pointer manipulation circuitry) for multiple execution instances (task-items). Each lane contains functional hardware (e.g., addition, subtraction, multiplication, division, matrix multiplication, transcendental functions, and other functions, such as APD) to execute one instruction instance on the data for one work-item. an arithmetic logic unit and other circuitry for executing instructions of the instruction set architecture of 116).

The hardware to execute all of these instruction types on one work-item per lane is sufficient to execute the instructions together on multiple work-items per lane simultaneously, if certain conditions are met. For example, because SIMD units 138 can perform a matrix multiplication operation for each work-item, and because matrix multiplication involves a relatively large number of multiply-add operations, SIMD units 138 ) has hardware for executing many “simple” instructions, such as multiply, add, or fused-multiply-add instructions, at a higher rate than instructions that do not have as much such functional hardware. Although increased execution rates for some instructions may be achieved in some cases, the hardware of SIMD units 138 is sufficient to unconditionally support only the “normal” execution rate. In other words, there are limitations to execution rate due to other reasons such as register file bandwidth related limitations. Nonetheless, SIMD unit 138 of the present disclosure attempts to execute instructions at a higher rate in many situations.

The increased execution rate described above is sometimes referred to herein as “increased execution rate.” Execution of instructions without performing a technique to increase execution rate is sometimes referred to herein as being performed at a "normal execution rate." In one implementation, the distinguishing characteristic between increased and normal execution rates is that, at the increased execution rate, each lane executes multiple instructions, whereas at the normal execution rate, each lane executes one instruction. is to execute. The increased execution rate makes it possible for multiple instructions executing on each lane to come from different wavefronts. Alternatively, it is possible for multiple instructions to come from the same wavefront. For example, it is possible for the next instruction in the program sequence to be executed along with the instruction after that instruction in the program sequence and from the same wavefront.

4 illustrates details of SIMD unit 138, according to an example. In some examples, each of the illustrated blocks of SIMD unit 138 (e.g., arbiter 404, functional units 406, vector register file 412, or scalar register file 414) is It can be implemented with hard-wired circuitry configured to perform the operations described in the specification. In other embodiments, some or all of the illustrated blocks of SIMD unit 138 are implemented as software running on a processor, or a combination of software and hardware circuitry.

SIMD unit 138 in FIG. 4 includes functional units 406, arbiter 404, vector register file 412, and scalar register file 414. Arbiter 404 is configured to select one or more pending instructions 403 from one or more pending task groups 402 for execution. Each outstanding wavefront 402 is a wavefront assigned to a SIMD unit 138 for execution. In various examples, this allocation is performed by a scheduler within compute unit 132 or by another scheduler not described in detail. In some implementations, SIMD unit 138 is “oversubscribed” for latency concealment purposes. The term “oversubscription” means that there are multiple work groups assigned to SIMD unit 138, and that SIMD unit 138 has sufficient hardware (e.g., functional units 406) to execute each of these assigned work groups simultaneously. It means that you do not have a . Accordingly, SIMD unit 138 alternates scheduling instructions from different task groups. Latency concealment occurs because groups of tasks sometimes stall for a variety of reasons, such as waiting for a memory access to complete or waiting for long instructions to complete.

Each pending task group 402 has one or more outstanding instructions 403 representing instructions to be executed by functional units 406 . In some examples, outstanding instructions 403 include the next instruction in program order. In some examples, outstanding instructions 403 include more than one instruction for one or more task groups 402. In the example, outstanding instructions 403 include the next instruction in program order and the next instruction after that instruction in program order.

Arbiter 404 examines outstanding instructions 403 and determines whether to issue the instructions at the normal execution rate or at an increased execution rate. As noted elsewhere herein, functional units 406 generally have sufficient resources to execute instructions at least at a normal execution rate. However, functional units 406 do not have sufficient resources to execute instructions at an increased execution rate in all situations. At least two resources sometimes limit execution to the normal execution rate: register file bandwidth and functional unit execution resources. With regard to register file bandwidth, it is possible that outstanding instructions 403 may be of a nature requiring more bandwidth than exists for the register files. In such a situation, SIMD unit 138 cannot operate at the increased execution rate and operates at the normal execution rate. With regard to functional unit execution resources, it is possible that there are too few execution resources present in the functional units 406 so that there are no outstanding instructions 403 that can be executed together. In such circumstances, SIMD unit 138 executes at its normal execution rate. If there are sufficient resources to execute at least two outstanding instructions 403 together, SIMD unit 138 executes at an increased execution rate. The term “execute together” means that functional units 406 perform operations on the same clock cycle(s) for multiple instructions that would otherwise be performed on different clock cycle(s). In the example, if the instructions are addition instructions, executing two instructions together means performing the addition operation for each instruction in a single clock cycle. In this example, executing two instructions together is contrasted with not executing two instructions together, where the addition operations for two different instructions are performed in different clock cycles. It should be noted that instructions are pipelined, which means that execution of an instruction includes multiple sub-operations, each of which is performed in one or more clock cycles. To execute instructions together, a lane of SIMD unit 138 may execute multiple sub-operations for different instructions in the same cycle(s).

As explained elsewhere herein, one reason SIMD units 138 can execute at higher execution rates is that many instructions are complex and require many “simple” functional units. This is because there are enough functional units 406 to execute them together. Thus, in some instances, executing instructions together means executing multiple instructions in the same clock cycle(s), where at least one such instruction is a "simple" functional unit used by a complex instruction in other cycles. It is a "simple" command that uses . In the example, two add instructions are executed together, and at least one such add instruction uses an adder that is used for a matrix multiply instruction that executes at a different time.

As mentioned above, the number of functional units sometimes limits whether multiple instructions can be executed together. More specifically, because the lanes do not contain redundant hardware to execute multiple instances of every instruction in an instruction set architecture, SIMD unit 138 cannot execute combinations of certain types of instructions together. Specifically, SIMD unit 138 cannot jointly execute two instances of an instruction for which two instances of the functional units required to execute the instruction are not present in functional units 406 . However, it is sometimes possible for one instruction of a very simple type and another instruction of a more complex type to be executed together if sufficient functional units exist to do so. In an example, add, multiply, and fuse-add-multiply instructions may be executed together. Other examples of simple instructions include simple math instructions (e.g., subtraction), bit manipulation instructions (e.g., bitwise AND, OR, XOR, shift, etc.), or other instructions. Some “co-executable complex instructions” are instructions for which sufficient hardware does not exist for multiple such instructions to be executed together. Accordingly, arbiter 404 does not select multiple such instructions for execution together. However, SIMD unit 138 has sufficient hardware to execute complex instructions that are co-executable together with simple instructions. Accordingly, the arbiter is allowed to select co-executable complex instructions for execution together with simple instructions. Finally, some “non-co-executable complex instructions” consume too many cycles or too many functional units 406 to be executable together with simple instructions. Accordingly, SIMD unit 138 never executes complex instructions that are non-co-executable with any other instructions.

In Figure 4, functional units 406 include simple functional units 408 and additional logic for complex instructions 410. Simple functional units 408 are functional units for executing simple instructions. In various examples, simple functional units 408 include adders, multipliers, bit-wise manipulation instructions, etc. Additional logic for complex instructions 410 includes hardware for co-executable complex instructions and for non-co-executable complex instructions. Simple functional units 408 are shared between simple instructions and at least some of the additional logic for complex instructions 410 . For example, adders and multipliers are used for addition and multiplication instructions, as well as more complex instructions that use multiplication and addition.

As mentioned above, arbiter 404 may select outstanding instructions 403 for execution in situations where the outstanding instructions 403 have register access requirements that fit within the register file bandwidth. In some examples, register file bandwidth refers to the bandwidth available at register file ports. A register file port is an interface that allows functional units 406 to access the contents of a register, and there is a limited number of register file ports. In the example, vector register file 412 has four ports, each of which can provide a certain number of bits per clock cycle, such as 64 bits. If the arbiter 404 cannot find a number of outstanding instructions 403 that fit within the bandwidth of the vector register file 412, the arbiter 404 determines that the SIMD unit 138 should execute at its normal execution rate. do. Multiple instructions "fit within" a bandwidth if there is sufficient bandwidth to access all operands of all instructions to be executed together.

Register file bandwidth is further expanded by the fact that, in some examples, there are different types of register files, each with its own independent bandwidth. In the example, SIMD unit 138 includes vector register file 412 and scalar register file 414. Vector register file 412 has vector register file ports, and scalar register file 414 has scalar register file ports. Vector register file ports are accessed independently from scalar register file ports. Accordingly, available bandwidth increases for sets of instructions that access different register files compared to sets of instructions that access the same register file. In the example, the first instruction accesses three vector register file registers, and the second instruction accesses two vector register file registers and one scalar register file register. A third instruction accesses three vector register file registers. First and second instructions are less likely to experience register file bandwidth conflicts than first and third or second and third instructions.

In some examples, instructions access operands within register files based on which “slot” the operands are in. More specifically, instructions reference registers in a specific order. The place of a register in this order defines a slot. In some implementations, registers for an instruction are accessed in different clock cycles, and the particular cycle in which registers are accessed depends on the register's slot. In some implementations, instructions executed together access operands in the same slot in the same clock cycle(s). In an example, two instructions executed together access registers in the first slot in the same clock cycle. Therefore, if there is sufficient bandwidth to access the registers of the two instructions' respective slots, those two instructions can be executed together, and if there is not enough bandwidth, the two instructions cannot be executed together. In the example, two instructions access two vector registers in their first slot. If vector register file 412 has sufficient bandwidth to satisfy these two accesses in the clock cycle for the first slot, arbiter 404 may schedule these two instructions to execute together; Otherwise, arbiter 404 cannot schedule these two instructions to be executed together.

There are a number of aspects that arbiter 404 considers when determining whether the register accesses for two instructions fit within the available register access bandwidth. One aspect involves a comparison between the amount of bandwidth required by instructions and whether such bandwidth is available. In one example, a certain number of register file ports are available for register access. Each port can provide access to a certain number of bits per clock cycle, and access to registers requires a certain number of such bits. In the example, a first instruction accesses three 32-bit vector registers, and a second instruction accesses one 64-bit vector register and two 32-bit registers. In this example, there are three ports to the vector register file 412, each providing 64 bits of bandwidth. In this case, the amount of bits required (5 x 32 + 64 = 224) is greater than the number of bits available (192), so those instructions cannot be executed together. In another example, a first instruction accesses two 32-bit registers and a second instruction accesses three 32-bit registers. In this example, the instructions can be executed together if no other conflicts exist.

In some implementations, ports may provide single or dual data rates. In these implementations, each “half” of the port can access a different bank of the register file. A bank of a vector register file is a portion of a register file that contains a set of mutually exclusive registers with registers assigned to different banks. A port can provide an improved rate of access to registers in a register file when data for the port is sourced from two different banks. In the example, the odd registers are in one bank and the even registers are in the other bank. Accordingly, when considering whether bandwidth is available for register accesses for instructions, arbiter 404 considers whether the registers being accessed are found in different banks, which may increase or limit the available bandwidth. You can.

Another aspect includes register file types of the registers being accessed. Specifically, as described above, different types of register files have their own independent bandwidth. In the example, vector register file 412 has three ports, each capable of accessing 64 bits, and scalar register file 414 also has three ports, each capable of accessing 64 bits. Accordingly, if two instructions have a mix of vector and scalar registers, arbiter 404 considers whether there is sufficient bandwidth across both vector register file 412 and scalar register file 414.

Another aspect includes whether register values can be accessed from entities other than the register file(s), to reduce the amount of bandwidth required to be accessed directly through the register file(s). Some such entities include transfer circuitry within the SIMD unit 138 execution pipeline, an operand cache, or other operands or register values used as instructions. The transmission circuit is a circuit that compensates for data risks occurring in the execution pipeline. A data hazard is a situation where one instruction in the pipeline writes to a register that is read by another instruction in the pipeline. If such another instruction fetches the value of a register from a register file, this may occur before the value generated by the first instruction is written to the file, and the other instruction may read the stale data. The transfer circuitry prevents this from happening by providing the command with the values to be written. This transfer circuitry does not take up the bandwidth of the register file. Therefore, when value transfers occur in this manner, the bandwidth of the register file is effectively increased. Instructions that are executed close together in program order (e.g., within the number of cycles it takes for the instruction to be executed by the pipeline) often use passed data. Operand caches cache register values. If an instruction can obtain values from those operand caches, this increases the effective register file bandwidth. Register values can be duplicated, meaning that a register value can be used more than once across one or more instructions. In this case, only one access consumes register file bandwidth.

In summary, the arbiter 404 can configure two (or more) instructions for execution in parallel in situations where there are sufficient functional units 406 to execute two instructions, and where there is sufficient register file bandwidth. ) You can select a command. There are enough functional units 406 if two instructions have instruction types such that each lane contains functional units 406 to execute both those types together. There is sufficient register file bandwidth if the operands accessed by two instructions fit within the available register file bandwidth.

Periodically (e.g., per clock cycle, or whenever arbiter 404 is ready to schedule instructions for execution), arbiter 404 determines which instruction(s) to execute and a number of instructions. A decision is made as to whether they can be implemented together. Arbiter 404 considers one or more combinations of pending instructions 403 and determines whether such one or more combinations meet the criteria set forth herein to be able to be executed together. Such combinations are referred to as “Eligible Combinations.” The arbiter 404 selects one such combination to be executed together in situations where such combinations exist, and causes such combinations of instructions to be executed together. Arbiter 404 does not cause any combination of instructions to be executed together if no such combination exists, in which case it causes one instruction to be executed. In various examples, arbiter 404 selects the eligible combination with the highest priority, as determined by a priority assignment operation. The priority is determined in any technically feasible manner, such as a priority that facilitates round robin execution, a priority that helps effect forward progress, or a priority determined in any other technically feasible manner. In some examples, if an instruction with the highest priority is not co-executable with any other instruction but does have priority, arbiter 404 selects that instruction for standalone execution.

5 is a flow diagram of a method 500 for executing instructions, according to an example. Although described with respect to the system of FIGS. 1-4 , those skilled in the art will recognize that any system configured to perform the steps of method 500 in any technically feasible order is within the scope of the present disclosure.

At step 502, arbiter 404 is operating at the first time to issue instructions for execution. It should be understood that the arbiter 404 selects instructions for execution at various times, such as per cycle, as long as resources are available for new instructions. Arbiter 404 identifies that sufficient processing resources exist to execute the first set of instructions together. In some examples, executing instructions together means that at least one operation for each of the instructions is performed on the same lane of SIMD unit 138 in the same cycle(s). In some examples, executing instructions together means that at least one operation of at least one of the instructions may be performed on the hardware used for complex instructions (e.g., one of the adders used for matrix multiplication or other more complex instructions). It means that it is carried out as one). In some examples, executing instructions together means that the execution rate of the instructions is increased beyond their “normal” execution rate. In some examples, the normal execution rate is the rate at which instructions can be executed even if there are conflicts over functional units and/or register bandwidth for each instruction. In some examples, the normal execution rate is the rate at which one lane of SIMD unit 138 has sufficient hardware to ensure execution, assuming there are no other reasons to stall (such as waiting for data to be fetched from memory). In some examples, the normal execution rate is a single rate execution rate and the increased execution rate is a dual execution rate. In some examples, instructions to be executed together are from different wavefronts, and in other examples, instructions to be executed together are from the same wavefront. In examples where the instructions to be executed together are from different wavefronts, executing these instructions together may allow operations on two wavefronts to proceed at the same time that operations from just one wavefront would proceed if the instructions could not be executed together. let it be In the example, if the instructions can be executed together, then one instruction for each of the two wavefronts can be completed per cycle (or every certain number of cycles), whereas if those instructions cannot be executed together, then only one instruction from one wavefront can be completed. The instructions may be completed every cycle (or every certain number of cycles).

Step 502 includes identifying that there are sufficient processing resources to execute the instructions together. In some examples, such processing resources include register file bandwidth and functional units. In some examples, each lane has enough functional units 406 to execute together certain combinations of instruction types, but not for other combinations of instruction types. In an example, complex instructions such as matrix multiplication require multiple types of functional units such as adders and multipliers. Moreover, since each lane can execute a respective instruction type, each lane contains at least one copy of these different functional units. Accordingly, each lane contains multiple copies of functional units for performing simpler instructions such as simple addition or multiplication operations. Thus, in one example, there are sufficient processing resources for two instructions if the instructions are of a type for which there is at least one copy of the functional units for executing those instructions.

In some examples, register file bandwidth includes bandwidth for one or more register files, such as vector register file 412 and scalar register file 414. In some examples, instructions consume register file bandwidth based on the specified operands. More specifically, instructions can specify operands by register names. Additionally, these operands implicitly or explicitly contain the amount of data that needs to be transferred. In one example, the operands are 32-bit operands or 64-bit operands. Arbiter 404 determines that the register file bandwidth is sufficient if the amount of data specified for a number of instructions is less than the available bandwidth for the specified registers. In various examples, vector register file 412 and scalar register file 414 have independent bandwidths, such that accesses to vector register file 412 do not consume bandwidth of scalar register file 414, and accesses to scalar register file 414 do not consume bandwidth of scalar register file 414. This means that access to file 414 does not consume bandwidth to vector register file 412. Additionally, it is possible that some register accesses do not consume bandwidth for other reasons, such as SIMD unit 138 may access operands from different locations, such as cache or forwarding circuitry. Operands that do not reference registers (such as literal values) do not consume register file bandwidth. In some examples, there are restrictions on register file access. For example, in some examples, to access the full bandwidth, accesses must be to two (or more) different banks of register files. In some examples, maximum bandwidth is accessible when all banks are used, and bandwidth is limited when register accesses are biased to specific banks. In the example, vector register file 412 includes two banks. Additionally, register accesses for two instructions utilize the full bandwidth and are evenly distributed between banks. In such a situation, there is sufficient bandwidth to execute the instructions together. In another example, register accesses for two instructions utilize the full bandwidth, but are heavily biased towards one bank. In such an example, there is not enough bandwidth to execute the instructions together.

Step 502 includes determining that there are sufficient functional units to execute the instructions together, as well as that there is sufficient register bandwidth to execute the instructions together. At step 504, in response to this determination, arbiter 404 issues a first set of instructions for execution together. It should be understood that in some implementations, during operation, arbiter 404 sometimes selects between two or more sets of instructions that are suitable for execution together. In some examples, this selection occurs based on a priority mechanism.

At step 506, arbiter 404 determines that there is no set of instructions that meet the criteria for execution together. In one example, this decision occurs because an instruction that cannot be executed with any other instruction (e.g., due to using too many functional units 406) has the highest priority (and therefore must be executed). . In other examples, this decision occurs because arbiter 404 cannot find any instruction set that meets the criteria described with respect to step 502. At step 508, in response to the decision of step 506, arbiter 404 issues one command instead of issuing multiple commands together.

It should be understood that arbiter 404 sequentially performs the steps of method 500 to issue instructions for execution. Additionally, method 500 includes one iteration in which arbiter 404 selects instructions for execution together, and one iteration in which arbiter 404 selects an instruction that is not intended for execution with any other instruction. However, it should be understood that this particular pattern is exemplary only, and that arbiter 404 is free to select instructions that may or may not be executed together as runtime circumstances warrant.

It should be understood that many modifications are possible based on the disclosure herein. Although features and elements are described above in specific combinations, each feature or element may be used alone, without other features and elements, or in various combinations with or without other features and elements.

Various functional units illustrated in the figures and/or described herein (processor 102, input driver 112, input devices 108, output driver 114, output devices 110, accelerated processing) including device 116, command processor 136, graphics processing pipeline 134, compute units 132, SIMD units 138, system 400, or register allocator 402, (without limitation) is implemented as a general-purpose computer, processor, or processor core, or as a program, software, or firmware, stored on a non-transitory computer-readable medium or other medium, and stored on a general-purpose computer, processor, or processor core. It may be feasible by The provided methods may be implemented in a general purpose computer, processor, or processor core. Suitable processors include, for example, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, or an Application Specific Integrated Circuit (ASIC). ), Field Programmable Gate Array (FPGA) circuits, any other type of integrated circuit (IC), and/or state machines. Such processors may be manufactured by constructing a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data, including a netlist (such instructions may be stored on a computer-readable medium). has exist). The result of such processing may be a maskwork that is then used in a semiconductor manufacturing process to fabricate a processor that implements the features of the present disclosure.

The methods or flow diagrams provided herein may be implemented as a computer program, software, or firmware incorporated into a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media. Includes media such as CD-ROM disks, and digital versatile disks (DVDs).

Claims (20)

As a method,
At a first time to issue instructions for execution, performing a first identification, including identifying that sufficient processing resources exist within a processing lane to jointly execute the first set of instructions;
In response to the first identification, executing the first set of instructions together;
At a second time for issuing instructions for execution, performing a second identification, including identifying that instructions for which there are sufficient processing resources for execution together within a processing lane are not available;
In response to the second identification, executing an instruction independently of any other instructions. 2. The method of claim 1, wherein identifying that sufficient processing resources exist comprises identifying that sufficient functional units exist in the processing lane to execute at least one operation of each instruction of the first set of instructions in a same clock cycle. A method comprising the steps of: 2. The method of claim 1, wherein identifying that sufficient processing resources exist comprises determining that each instruction of the first set of instructions comprises a simple instruction for which multiple copies of functional units exist within the processing lane. How to. 2. The method of claim 1, wherein identifying that sufficient processing resources exist comprises: wherein one of the first set of instructions comprises a complex instruction for which one copy of the functional units exists in the processing lane and the first set of instructions A method comprising determining that another instruction in the set includes a simple instruction for which there are multiple copies of functional units within the processing lane. The method of claim 1, wherein identifying that sufficient processing resources exist includes determining that sufficient bandwidth exists for operands of each instruction in the first set of instructions. 6. The method of claim 5, wherein sufficient bandwidth exists if the bandwidth consumed by the operands of the instructions in the first set of instructions is less than the bandwidth available in register files. 7. The method of claim 6, wherein operands available from sources other than a register file do not consume bandwidth of the register files. The method of claim 1, wherein the instructions in the first set of instructions are from the same wavefront. The method of claim 1, wherein the instructions in the first set of instructions are from different wavefronts. As a compute unit,
a memory configured to store instructions; and
A processing unit comprising:
At a first time to issue instructions obtained from the memory for execution, perform a first identification, including identifying that sufficient processing resources exist within a processing lane to jointly execute the first set of instructions;
In response to the first identification, jointly executing the first set of instructions;
At a second time to issue instructions obtained from the memory for execution, perform a second identification, including identifying that instructions for which there are sufficient processing resources for execution together within the processing lane are not available; ,
In response to the second identification, the compute unit is configured to execute an instruction independently of any other instructions. 11. The method of claim 10, wherein identifying that sufficient processing resources exist comprises identifying that sufficient functional units are present in the processing lane to execute at least one operation of each instruction of the first set of instructions in a same clock cycle. Compute unit, including: 11. The method of claim 10, wherein identifying that sufficient processing resources exist comprises determining that each instruction of the first set of instructions includes a simple instruction for which multiple copies of functional units exist within the processing lane. Compute unit. 11. The method of claim 10, wherein identifying that sufficient processing resources exist means that one of the first set of instructions comprises a complex instruction for which one copy of the functional units exists within the processing lane and and determining that another instruction in the compute unit includes a simple instruction for which there are multiple copies of functional units within the processing lane. 11. The compute unit of claim 10, wherein identifying that sufficient processing resources exist includes determining that sufficient bandwidth exists for operands of each instruction of the first set of instructions. 15. The compute unit of claim 14, wherein sufficient bandwidth exists if the bandwidth consumed by the operands of the instructions in the first set of instructions is less than the bandwidth available in register files. 16. The compute unit of claim 15, wherein operands available from sources other than a register file do not consume bandwidth of the register files. 11. The compute unit of claim 10, wherein instructions in the first set of instructions are from the same wavefront. 11. The compute unit of claim 10, wherein instructions in the first set of instructions are from different wavefronts. As an accelerated processing device,
It includes a plurality of compute units, each compute unit having:
a memory configured to store instructions, and
A processing unit comprising:
At a first time to issue instructions obtained from the memory for execution, perform a first identification, including identifying that sufficient processing resources exist within a processing lane to jointly execute the first set of instructions;
In response to the first identification, jointly executing the first set of instructions;
At a second time to issue instructions obtained from the memory for execution, perform a second identification, including identifying that instructions for which there are sufficient processing resources for execution together within the processing lane are not available; ,
In response to the second identification, the accelerated processing device is configured to execute an instruction independently of any other instructions. 20. The method of claim 19, wherein identifying that sufficient processing resources exist comprises identifying that sufficient functional units are present in the processing lane to execute at least one operation of each instruction of the first set of instructions in a same clock cycle. An accelerated processing device comprising:

Images