Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
  • G06F17/10—Complex mathematical operations
  • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
  • G06F1/26—Power supply means, e.g. regulation thereof
  • G06F1/32—Means for saving power
  • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
  • G06F1/3234—Power saving characterised by the action undertaken
  • G06F1/3243—Power saving in microcontroller unit
• • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
  • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
  • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
  • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

• a processor can include a graphics processing unit (GPU).
• the GPU includes specialized hardware to perform parallel processing for relatively large blocks of data. Accordingly, the GPU can support graphics applications, as well as other operations that require vector and matrix manipulation.
• a GPU can include dedicated hardware to perform designated types of matrix operations, including matrix multiplication.
• FIG. 1 is a block diagram of a GPU of a processor, the GPU configured to perform matrix multiplication by sequencing the application of submatrices to a matrix multiplier in accordance with some embodiments.
• FIG. 2 is a diagram illustrating example matrices for multiplication at the GPU of FIG. 1 in accordance with some embodiments.
• FIG. 3 is a diagram illustrating an example of sequencing the application of submatrices to the matrix multiplier of FIG. 1 in accordance with some embodiments.
• FIG. 4 is a block diagram of additional aspects of the GPU of FIG. 1 supporting submatrix sequencing in accordance with some embodiments.
• FIG. 5 is a flow diagram of a method of sequencing application of submatrices at a matrix multiplier of a GPU in accordance with some embodiments.
• FIGs. 1-5 illustrate techniques for reducing power consumption at a graphics processing unit (GPU) of a processor by sequencing the application of submatrices at a matrix multiplier to reduce the number of input changes at an input register of the matrix multiplier.
• the matrix multiplier is configured to perform a matrix multiplication for a relatively small matrix (e.g., a 4X4 matrix).
• a relatively small matrix e.g., a 4X4 matrix.
• the GPU decomposes the larger matrices into smaller submatrices and stores the submatrices at input registers of the matrix multiplier in a sequence, thereby calculating each column of a result matrix.
• FIG. 1 illustrates a GPU 100 of a processor that configured to perform matrix multiplication by sequencing the application of submatrices in accordance with some embodiments.
• the GPU 100 is part of a processor that is generally configured to execute sets of instructions in order to carry out operations on behalf of an electronic device.
• the GPU 100 is part of an electronic device such as a desktop or laptop computer, a server, a handheld electronic device such as a smartphone or tablet, a game console, and the like.
• the GPU 100 is generally configured to execute graphics and vector processing operations on behalf of the processor.
• a central processing unit (CPU, not shown at FIG. 1 ) of the processor provides the GPU with sets of operations for execution, whereby the sets of operations are associated with graphics or vector processing.
• CPU central processing unit
• the GPU 100 includes a plurality of Single-Instruction Multiple-Data (SIMD) processing units (e.g., SIMD units 102 and 104). It will be appreciated that the GPU 100 also includes additional modules to support the SIMD units, such as fetch and decode logic to fetch and decode instructions for the SIMD units, a register file to store operands for the SIMD units, and the like.
• SIMD units include additional modules to support the SIMD units, such as fetch and decode logic to fetch and decode instructions for the SIMD units, a register file to store operands for the SIMD units, and the like.
• each SIMD unit includes a matrix multiplier together with corresponding input registers and a corresponding output register.
• the SIMD unit 102 includes a matrix multiplier 1 10, input registers 106 and 107, and an output register 108.
• the term“register” refers to any storage module that is configured to store matrices (including submatrices).
• the matrix multiplier 1 10 is configured to multiply matrices stored at the registers 106 and 107 and store the resulting product at the register 108.
• the generation of a single product for matrices at the input registers 106 and 102 is referred to herein as a“multiplication cycle” for the matrix multiplier 1 10.
• the SIMD unit 102 is clocked by a clock signal (designated“CLK”) and a multiply cycle of the matrix multiplier 1 10 corresponds to a single clock cycle of the CLK clock signal. That is, for a single clock cycle of the CLK clock signal, the matrix multiplier 1 10 is configured to generate a product at the register 108 based on input operands stored at the input registers 106 and 107.
• each multiply cycle of the matrix multiplier 1 10 requires multiple cycles of the CLK clock signal.
• the matrix multiplier 1 10 is configured to generate a product for relatively small input matrices.
• the matrix multiplier 1 10 is a 4X4X4 multiplier, such that the matrix multiplier 1 10 is configured to multiply a 4X4 matrix stored at the input register 106 with a 4X4 matrix stored at the input register 107 to generate a 4X4 product (result) matrix at the output register 108.
• the CPU provides the GPU 100 with operations requiring multiplication of larger matrices, such as multiplication of 16X16 matrices.
• the SIMD is configured to decompose the larger matrices into multiple smaller
• the matrix multiplier 1 10 multiplies input matrices, designated matrix A, an MXK matrix, and matrix B, a KXN matrix, to calculate a result matrix R (an MXN) matrix.
• the matrices A and B are stored at the input registers 106 and 107, respectively, and the result matrix R is stored at the output register 108.
• the matrix multiplier 1 10 calculates the result matrix R by summing the K outer products of column k of the A matrix and row k of the B matrix, as set forth by the following formula:
• the SIMD 102 decomposes the input matrices into smaller submatrices that are the specified input size for by the matrix multiplier 1 10, multiplies the submatrices at the matrix multiplier 1 10 to generate a set of intermediate results, and combines the intermediate results to determine the final result matrix R.
• the matrix multiplier calculates the inner product
• the SIMD 102 decomposes the input matrices into smaller submatrices, determines the products of different sets of the submatrices based on dot products of the different sets, then calculates the outer product for the resulting dot products to determine the final result matrix.
• calculating the different intermediate results requires changing the submatrices stored at the input registers 106 and 107.
• each change in data stored at an input register consumes power at the GPU 102.
• each change in data at the inputs of the corresponding arithmetic logic units (ALUs) or other modules of the matrix multiplier 1 10 consumes additional power, relative to maintaining the input data in an unchanged state.
• the SIMD 102 sequences the storage of submatrices at the input registers 106 and 107 such that a submatrix is maintained at one of the input registers (e.g., register 107) for a plurality of successive multiply cycles, until that submatrix is no longer needed for calculation of the result matrix R. That is, the SIMD 102 sequences application of input submatrices at the input registers 106 and 107 to reduce the amount of input switching at one of the registers and, as a result, at one of the inputs of the matrix multiplier 1 10, thereby conserving power.
• FIG. 2 illustrates an example of two 16X16 matrices 220 and 222, designated matrix A and matrix B, respectively.
• Each of the matrices A and B include 16 4X4 submatrices (e.g., submatrix 221 of matrix A).
• the matrices A and B are multiplied at the GPU 102 to generate a result matrix 224, designated matrix R, which also includes a plurality of 4X4 submatrices.
• the matrix R can be viewed as a set of columns of submatrices.
• the first column of R is composed of submatrices R 0, o, Ri , o, R 2, o, and R 3, o.
• the GPU 100 calculates the matrix R by calculating each column of submatrices of R, then concatenates the different columns to form the R matrix.
• each column of submatrices of R is calculated concurrently at a different corresponding SIMD of the GPU 100, and one of the SIMDs then
• the corresponding SIMD To calculate a column of submatrices of R, the corresponding SIMD employs its matrix multiplier to determine a set of inner (dot) products for corresponding submatrices of the matrices A and B, then calculates outer products over the inner product results. For example, to generate the submatrix Ro , o, the SIMD 102 performs the following calculations:
• the SIMD 102 performs analogous calculations to generate the submatrices R 2, o and R 3 ,O.
• the SIMD 102 To perform each multiplication for calculating a corresponding submatrix, the SIMD 102 loads the corresponding submatrices of matrix A and matrix B into the input registers 106 and 107, respectively, and the matrix multiplier 1 10 performs the multiplication, storing the result at the output register 108.
• submatrices of the matrix B are reused to calculate different submatrices of the matrix R.
• the SIMD 102 is configured to sequence the multiplications, so that the submatrices of the matrix B, as stored at the input register 107, remain unchanged over a plurality of successive multiplication cycles of the matrix multiplier 1 10. The SIMD 102 thereby reduces the number of loads to the input register 107 and changes of the input of the matrix multiplier 1 10, thus reducing power consumption.
• FIG. 3 illustrates a set of successive multiplication cycles 301 -305 and the corresponding contents of each of the input registers 106 and 107.
• the SIMD 102 loads the submatrices Ao , o and Bo , o to the input registers 106 and 107, respectively.
• the matrix multiplier 1 10 multiplies the submatrices to calculate an intermediate result for the first column of the result matrix R.
• the SIMD 102 loads the submatrix Ai , o into the input register 106, but maintains the submatrix Bo , o at the input register 107.
• the matrix multiplier 1 10 multipliers the submatrices to calculate another intermediate result for the first column of the result matrix R.
• the SIMD 102 For the next multiplication cycle 303 the SIMD 102 loads the submatrix A 2, o into the input register 106, but maintains the submatrix Bo , o at the input register 107.
• the matrix multiplier 1 10 multipliers the submatrices to calculate still another intermediate result for the first column of the result matrix R.
• the SIMD 102 loads the submatrix A 3, o into the input register 106, but maintains the submatrix Bo , o at the input register 107.
• the matrix multiplier 1 10 multipliers the submatrices to calculate another intermediate result for the first column of the result matrix R.
• all calculations that require the submatrix Bo , o have been completed.
• the SIMD 102 loads the submatrix Ao ,i into the input register 106 and the submatrix Bi , o into the input register 107.
• the SIMD 102 maintains the submatrix Bo , o at the input register 107 for four consecutive
• the SIMD 102 continues executing multiplication operations at the matrix multiplier 1 10 and combining the resulting products to calculate the first column of the result matrix R.
• the sequence of multiplications (including corresponding input matrices loaded and maintained at the input registers 106 and 107) is as follows:
• the GPU 102 performs similar calculations to calculate the other columns of the result matrix R.
• the GPU 102 employs a different SIMD to concurrently calculate a corresponding column of the result matrix R, and employs one of the SIMDs, or other module, to concatenate the different columns into the final result matrix R.
• FIG. 4 illustrates additional aspects of the SIMD 102 of FIG. 1 to support sequencing of input submatrices for the matrix multiplier 1 10 in accordance with some embodiments.
• the SIMD 102 includes a data store 435 connected to a sequencer 430.
• the data store 435 is a buffer, cache, register file, or other memory structure configured to store submatrices (e.g., submatrix 433) for the matrix multiplier 1 10.
• the sequencer 430 is a hardware module configured to decompose the input matrices 105 (matrix A and matrix B) into corresponding submatrices and store the submatrices at the data store 435.
• the sequencer 430 is further configured to, for corresponding multiplication cycles, retrieve one or more submatrices from the data store 435 and load each retrieved submatrix to the corresponding input register 106 and 107.
• the sequencer 430 thus controls the sequencing of input submatrices at the matrix multiplier 1 10 to carry out a matrix multiplication of a relatively large matrix.
• FIG. 5 is a flow diagram of a method 500 of sequencing application of submatrices at a matrix multiplier of a GPU in accordance with some embodiments. For purposes of description, the method 500 is described with respect to an example implementation at the GPU 100 of FIG. 1.
• the sequencer 430 loads the initial submatrices (e.g.
• the matrix multiplier 1 10 multiplies the submatrices stored at the input registers 106 and 107 to generate a product and adds the result to the intermediate result for the corresponding column of the result matrix R, if any, as set forth above.
• the method flow moves to block 506 and the sequencer 430 determines if the input submatrix at the input register 106
• the method flow moves to block 508 and the sequencer 430 loads, to the input register 106, the submatrix of A corresponding to the current column (e.g., column 0) and the next row.
• the submatrix of B stored at the input register 107 is maintained, thereby conserving power.
• the method flow returns to block 504 and the matrix multiplier 1 10 executes the next multiply operation—that is, executes the next multiply cycle.
• the method flow moves to block 510 and the sequencer 430 determines if the input submatrix stored at the input register 510 corresponds to the last row of the matrix B. If not, the method flow moves to block 512 and the sequencer 430 loads to the input register 107 the submatrix of B corresponding to the column of R that is being calculated. In addition, the sequencer 430 loads to the input register 106 the submatrix of A corresponding to the initial row (e.g., row 0) and the next column. The method flow returns to block 504 and the matrix multiplier 1 10 executes the next multiply operation.
• the method flow moves to block 514 and the SIMD 102 stores the final result for the column of R.
• the GPU 100 combines each of the calculated columns to generate the result matrix R.
• the GPU 102 provides the result matrix R to a CPU for further processing.
• the GPU 100 employs the result matrix R to, for example, generate one or more objects in a display frame, and provides the display frame to a frame buffer for display at a display device.
• a method includes: for a first multiply cycle of a matrix multiplier of a graphics processing unit (GPU) multiplying a first matrix and a second matrix: multiplying a first submatrix of the first matrix stored at a first input register with a first submatrix of the second matrix stored at a second input register; for a second multiply cycle of the matrix multiplier, the second multiply cycle succeeding the first multiply cycle: multiplying the first submatrix of the first matrix stored at the first input register with a second submatrix of the second matrix stored at a second input register; and maintaining the first submatrix at the first input register for the first multiply cycle and the second multiply cycle.
• GPU graphics processing unit
• the method includes: for a third multiply cycle of the matrix multiplier, the third multiply cycle succeeding the second multiply cycle: multiplying the first submatrix of the first matrix stored at the first input register with a second submatrix of the second matrix stored at a second input register; and maintaining the first submatrix at the first input register for the first multiply cycle the second multiply cycle, and the third multiply cycle.
• the first submatrix includes at least one non-zero element.
• the method includes determining a product of the first matrix and the second matrix based on results of the first multiply cycle and the second multiply cycle, the product comprising a result matrix.
• determining the product includes: determining a submatrix of the result matrix based on results of the first multiply cycle and the second multiply cycle.
• the submatrix of the result matrix comprises one of a column and a row of the result matrix.
• determining the product includes: determining an outer product based on results of the first multiply cycle and the second multiply cycle.
• the method includes for a third multiply cycle of the matrix multiplier, the third multiply cycle succeeding the first multiply cycle: multiplying a second submatrix of the first matrix stored at the first input register with a second submatrix of the second matrix stored at the second input register; and changing the first submatrix of the first matrix to the second submatrix of the first matrix for the third multiply cycle.
• a method includes: multiplying submatrices of a first matrix with submatrices of a second matrix at a matrix multiplier of a graphics processing unit (GPU) to determine a matrix product, wherein the multiplying includes: maintaining a first submatrix at a first input register of the matrix multiplier over a first plurality of multiply cycles.
• the multiplying further includes: changing submatrices at a second input register of the matrix multiplier over the first plurality of multiply cycles.
• the multiplying further includes: maintaining a second submatrix at the second input register of the matrix multiplier over a second plurality of multiply cycles.
• at least one element of the first submatrix is a non-zero element.
• a graphics processing unit includes: a first input register; a second input register; a matrix multiplier to multiply a submatrix stored at the first input register with a submatrix stored at the second input register; and a sequencer to control submatrices stored at the first input register and the second input register, the sequencer configured to: for a first multiply cycle of the matrix multiplier store a first submatrix of the first matrix at the first input register and a first submatrix of a second matrix stored at the second input register; for a second multiply cycle of the matrix multiplier, maintain the first submatrix of the first matrix at the first input register and store a second submatrix of the second matrix stored at the second input register, the second multiply cycle succeeding the first multiply cycle.
• the sequencer if configured to: for a third multiply cycle of the matrix multiplier, the third multiply cycle succeeding the first multiply cycle: maintain the first matrix stored at the first input register and store a second submatrix of the second matrix stored at the second input register.
• the first submatrix includes at least one non-zero element.
• the GPU is configured to: determine a product of the first matrix and the second matrix based on results of the first multiply cycle and the second multiply cycle, the product comprising a result matrix.
• the GPU is configured to determine the product by: determining a submatrix of the result matrix based on results of the first multiply cycle and the second multiply cycle.
• the submatrix of the result matrix comprises one of a column and a row of the result matrix.
• he GPU is configured to determine the product by: determining an outer product based on results of the first multiply cycle and the second multiply cycle.
• the sequencer is configured to: for a third multiply cycle of the matrix multiplier, the third multiply cycle succeeding the first multiply cycle: store a second submatrix of the first matrix at the first input register and a second submatrix of the second matrix at the second input register.
• a computer readable storage medium may include any non-transitory storage medium, or combination of non-transitory storage media, accessible by a computer system during use to provide instructions and/or data to the computer system.
• Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc , magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media.
• optical media e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc
• magnetic media e.g., floppy disc , magnetic tape, or magnetic hard drive
• volatile memory e.g., random access memory (RAM) or cache
• non-volatile memory e.g., read-only memory (ROM) or Flash memory
• MEMS microelectromechanical systems
• the computer readable storage medium may be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).
• system RAM or ROM system RAM or ROM
• USB Universal Serial Bus
• NAS network accessible storage
• certain aspects of the techniques described above may implemented by one or more processors of a processing system executing software.
• the software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium.
• the software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above.
• the non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like.
• the executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Mathematical Physics (AREA)
• Theoretical Computer Science (AREA)
• Pure & Applied Mathematics (AREA)
• Mathematical Optimization (AREA)
• Mathematical Analysis (AREA)
• Data Mining & Analysis (AREA)
• Computational Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Computing Systems (AREA)
• Algebra (AREA)
• Databases & Information Systems (AREA)
• Software Systems (AREA)
• Advance Control (AREA)
• Complex Calculations (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=70327188

Family Applications (1)

Country Status (6)

Cited By (1)

Families Citing this family (9)

Citations (5)

Family Cites Families (14)

• 2018
  • 2018-10-31 US US16/176,449 patent/US11093580B2/en active Active
• 2019
  • 2019-06-18 WO PCT/US2019/037656 patent/WO2020091848A1/en not_active Ceased
  • 2019-06-18 CN CN201980077886.0A patent/CN113168430A/zh active Pending
  • 2019-06-18 JP JP2021523783A patent/JP7461945B2/ja active Active
  • 2019-06-18 KR KR1020217015589A patent/KR102586989B1/ko active Active
  • 2019-06-18 EP EP19880374.4A patent/EP3891626A4/en active Pending
• 2023
  • 2023-04-13 JP JP2023065959A patent/JP2023089161A/ja not_active Withdrawn

Patent Citations (5)

Also Published As

Similar Documents

Legal Events