JP2021060748A - Arithmetic processing device and arithmetic processing method - Google Patents

Abstract

To accomplish the acceleration of a matrix arithmetic processing.SOLUTION: An arithmetic processing device is accessible to a storage device that stores matrix data, and includes: a deciding unit which increases a partial row size that is a number of rows in a column direction of a partial matrix that is a dividing unit of the matrix data, obtains, when the partial row size after the increase becomes equal to or greater than a number of multiple multipliers, a partial column size that is a number of columns in a row direction of the partial matrix on the basis of the partial row size before the increase and of the number of multiple multipliers, and decides a shape of the partial matrix formed by the partial row size and by the partial column size; the plurality of multipliers; a plurality of adders; and a matrix arithmetic processing unit that executes a product-sum operation on a first partial matrix with the shape decided by the deciding unit among the pieces of first matrix data stored in the storage device, and a second partial matrix with the shape decided by the deciding unit among the pieces of second matrix data stored in the storage device.SELECTED DRAWING: Figure 2

Description

The present invention relates to an arithmetic unit and an arithmetic method for performing matrix operations.

In various applications, matrix operations are executed on a computer, but the amount of operations is large and becomes a bottleneck in the overall processing. Therefore, high speed has been pursued by various methods. For example, the information processing apparatus of Patent Document 1 divides a matrix into four in the row direction in units of multiples of the block size from the beginning in the row direction, and specifies four first submatrixes. The information processing apparatus divides the area other than the four first sub-matrix of the matrix into four and specifies the four second sub-matrix. The information processing apparatus assigns to each thread a matrix operation that generates a value of each element of each first submatrix and a matrix operation that generates a value of each element of each second submatrix.

JP-A-2018-197906

An offload device such as an FPGA (Field-Programmable Gate Array) has a certain amount of internal memory, and high-speed access to the internal memory reduces the load on the processor by performing high-speed calculation instead of the processor. When the method of Patent Document 1 is applied to an offload device, the internal memory of the offload device is larger than the register of the processor, so that the internal memory cannot be fully utilized.

Further, when it is configured by logic such as FPGA, it is inefficient because a circuit dedicated to a part where a part of the matrix is not allocated is required. The FPGA two-dimensionally allocates internal memory to a part of the matrix and executes multiplication of that part at high speed. However, in this method, the optimum shape of the submatrix for maximizing the calculation efficiency differs depending on the row of the original matrix and the shape of the matrix which is the size of the column.

When the FPGA performs multiplication on a fixed matrix size, it is possible to multiply on a fixed matrix size by tuning and fixing the size of the submatrix in advance. However, when matrix inputs of various sizes are expected, it becomes difficult to uniquely determine the size of the submatrix.

An object of the present invention is to speed up matrix operations.

The arithmetic device that is one aspect of the invention disclosed in the present application is an arithmetic device that can access a storage device that stores matrix data, and is the number of columns in the row direction of a submatrix that is a division unit of the matrix data. When the sub-column size is increased and the increased sub-column size is equal to or greater than the number of the plurality of multipliers, the sub-matrix of the sub-matrix is based on the sub-column size before the increase and the number of the plurality of multipliers. A determination unit that obtains a partial row size, which is the number of rows in the column direction, and determines the shape of the submatrix composed of the subcolumn size and the partial row size, and the plurality of multipliers and a plurality of adders. The first matrix data having a shape determined by the determination unit among the first matrix data stored in the storage device, and the determination unit among the second matrix data stored in the storage device. It is characterized by having a second submatrix having a shape determined by the above and a matrix calculation unit that executes a product-sum operation.

According to a typical embodiment of the present invention, the speed of matrix operation can be increased. Issues, configurations and effects other than those described above will be clarified by the description of the following examples.

FIG. 1 is an explanatory diagram showing an example of determining the shape of a submatrix. FIG. 2 is a block diagram showing a hardware configuration example of the arithmetic unit. FIG. 3 is a flowchart showing a higher-level algorithm for determining the submatrix size by the submatrix shape determining circuit. FIG. 4 is a flowchart showing a submatrix size determination algorithm by the submatrix shape determination circuit in steps S303 and S306. FIG. 5 is an explanatory diagram showing the relationship between the sense amplifier cache size, the column size, and the subsequence size. FIG. 6 is a flowchart showing a multiplication execution algorithm using a submatrix by the submatrix shape determination circuit. FIG. 7 is an explanatory diagram showing a processing example after the submatrix shape determination circuit. FIG. 8 is an explanatory diagram showing an operation example of the matrix operation array. FIG. 9 is a block diagram showing details of the register group. FIG. 10 is a flowchart showing an example of control processing by the control program. FIG. 11 is an explanatory diagram showing an example of control processing of the log output unit.

<Example of determining submatrix shape>
FIG. 1 is an explanatory diagram showing an example of determining the shape of a submatrix. The input matrix 101 is a matrix composed of the number of columns (COL) 102 in the row direction and the number of rows (ROW) 103 in the column direction. When the arithmetic unit that performs the matrix operation by the offload device acquires the input matrix, the input matrix 101 is divided into submatrixes prior to the matrix operation. The submatrix is a division unit of the input matrix 101.

Here, the first submatrix 111 is a matrix composed of the number of columns (COL) 112 and the number of rows (ROW) 113. As an example, the first submatrix 111 is a square matrix. The second submatrix 121 is a matrix composed of the number of columns (COL) 122 and the number of rows (ROW) 123. In the first sub-matrix 111 and the second sub-matrix 121, the number of columns (COL) 112 <the number of columns (COL) 122, the number of rows (ROW) 113> the number of rows (ROW) 123.

The first division example 110 is a division example in which the input matrix 101 is divided by the first submatrix 111. When the input matrix 101 is divided by the first submatrix 111, it is divided into eight first submatrix 111 by vertical V (2 pieces) × horizontal H (4 pieces).

The second division example 120 is a division example in which the input matrix 101 is divided by the second submatrix 121. When the input matrix 101 is divided by the second submatrix 121, it is divided into six second submatrix 121 by vertical V (3 pieces) × horizontal H (2 pieces).

The number of divisions of the first division example 110 is eight, and the number of divisions of the second division example 120 is six. By determining the shape of the submatrix to be the second submatrix 121, the arithmetic unit can reduce the number of operations as compared with the first submatrix 111, and can speed up the arithmetic processing.

<Hardware configuration example of arithmetic unit>
FIG. 2 is a block diagram showing a hardware configuration example of the arithmetic unit. The arithmetic unit 200 executes a matrix operation. The arithmetic unit 200 includes a processor 201, a memory 202, a general-purpose bus 203, an FPGA 204, and a DRAM (Dynamic Random Access Memory) 205. The arithmetic unit 200 may have at least FPGA 204.

Processor 201 controls memory 202 and FPGA 204. For example, the processor 201 sequentially reads the instruction sequence of the control program 206 on the memory 202 and executes the control program 206. When the control program 206 is executed, the processor 201 reads data from the memory 202 as a processing input thereof and writes the execution result to the memory 202. Further, the processor 201 exchanges and controls data with the FPGA 204 by accessing the FPGA 204 via the general-purpose bus 203.

The memory 202 stores the control program 206, the matrix A data 207, the matrix B data 208, the result C data 209, the log data 210, and various data (not shown). The control program 206 is a program for controlling the arithmetic unit 200, and is executed by the processor 201. The matrix A data 207 is data indicating one input matrix 101, and the matrix B data 208 is data indicating the other input matrix 101. The matrix A data 207 and the matrix B data 208 are transferred to the DRAM 205 connected to the FPGA 204 via the general-purpose bus 203 when the matrix operation is executed.

The result C data 209 is data showing the multiplication result of the matrix A data 207 and the matrix B data 208. The result C data 209 is stored in the memory 202 from the DRAM 205 via the FPGA 204 and the general-purpose bus 203. The log data 210 is data indicating a log of arithmetic processing of the FPGA 204, and is written to the memory 202 via the general-purpose bus 203. The general-purpose bus 203 connects the processor 201 and the FPGA 204 to transmit and receive data.

The FPGA 204 is an off-road device that programmablely configures and operates a logic circuit. FPGA 204 performs matrix operations. Further, the FPGA 204 is connected to the general-purpose bus 203 and transmits / receives data to / from the processor 201. The FPGA 204 also has an access function to the DRAM 205, and stores and reads data on the DRAM 205. Further, the FPGA 204 can also read / write data to / from the memory 202 via the general-purpose bus 203 by the DMA function.

The FPGA 204 includes an array size register 211, a sense amplifier cache size register 212, a threshold register 213, a matrix AB size register 214, a submatrix shape determination circuit 215, an addition control circuit 216, a read address generator 217, and the like. Read buffer 218, zero value 219, selector 220, matrix operation array 221, write address generation unit 222, write buffer 223, DMAC (Direct Memory Access Controller) 224, log output unit 225, and bus control. It has a part 226 and.

The array size register 211 holds the size of the matrix operation array 221. The sense amplifier cache size register 212 holds the size of the sense amplifier mounted on the DRAM 205. The threshold register 213 holds the number of hits of the sense amplifier cache when determining the shape of the submatrix in the matrix operation as the threshold. The matrix AB size register 214 stores the size of the matrix A data and the size of the matrix B data separately in rows and columns, respectively.

The submatrix shape determination circuit 215 uses the data stored in the array size register 211, the sense amplifier cache size register 212, the threshold register 213, and the matrix AB size register 214 to determine the size of the submatrix required to perform the matrix operation. Is calculated. This calculated value is output to the addition control circuit 216 and is used to perform the calculation with the optimum size of the submatrix. Further, a fraction of the matrix that does not match the submatrix (the margin portion of the first division example and the second division example in FIG. 1) is generated and output to the addition control circuit 216.

The addition control circuit 216 acquires the submatrix shape from the submatrix shape determination circuit 215 and controls the matrix calculation array 221. Further, the addition control circuit 216 controls the selector 220 to insert 0 as a fraction. The read address generation unit 217 generates the read address of the DRAM 205, and reads the matrix A data 207 and the matrix B data 208 from the DRAM 205. The read matrix A data 207 and matrix B data 208 are stored in the read buffer 218.

The read buffer 218 stores the matrix A data 207 and the matrix B data 208 read from the DRAM 205, and outputs the matrix A data 207 and the matrix B data 208 to the matrix operation array 221 via the selector 220. The zero value 2192219 is data indicating a zero value suitable for the data type used in the matrix operation. The selector 220 selects either the matrix A data 207 or the matrix B data 208 sent from the read buffer 218 and the zero value 219 according to the selection instruction from the addition control circuit 216 to the matrix operation array 221. Output.

The matrix operation array 221 performs a matrix operation, that is, a product-sum operation on the data from the selector 220. At this time, the matrix operation array 221 executes the product-sum operation according to the selected submatrix size under the control of the addition control circuit 216. This execution result is transferred to the write buffer 223.

The write address generation unit 222 generates an address for writing the result C data 209 stored in the write buffer 223 to the DRAM 205. The write buffer 223 holds the result C data 209 generated by the matrix operation array 221 and writes it to the DRAM 205. The DMAC224 controls data transmission / reception from the FPGA 204 to the memory 202. The data to be transmitted / received includes, for example, matrix A data 207, matrix B data 208, result C data 209, and log data 210, and the DMAC 224 controls the transmission / reception of internal data of the DRAM 205 or FPGA 204.

The log output unit 225 internally stores the operation of the submatrix shape determination circuit 215 as a log, and outputs the operation to the log data 210 in the memory 202 by using the DMAC224. The bus control unit 226 controls the general-purpose bus 203, and controls reading and writing to the array size register 211, the sense amplifier cache size register 212, the threshold register 213, and the matrix AB size register 214 by the processor 201.

The DRAM 205 is a storage device that holds data and transmits / receives data to / from FPGA 204. The DRAM 205 is used exclusively for the FPGA 204.

The operation of the arithmetic unit 200 is as follows. The arithmetic unit 200 causes the processor 201 to execute the control program 206. The arithmetic unit 200 sets the array size register 211, the sense amplifier cache size register 212, the threshold register 213, and the matrix AB size register 214 by the control program 206. Here, the arithmetic unit 200 starts the FPGA 204.

The FPGA 204 uses the DMAC224 to transfer the matrix A data 207 and the matrix B data 208 to the DRAM 205. When transferred, the submatrix shape determination circuit 215 is activated to calculate the optimum submatrix size for both the matrix A and the matrix B. Based on this calculation result, the read address generation unit 217 cuts out a submatrix size portion from the transferred matrix A and matrix B on the DRAM 205 and transfers the submatrix size portion to the read buffer 218.

The selector 220 transfers the transfer data to the matrix operation array 221, the matrix operation array 221 performs a submatrix operation, and writes the result C data 209 to the write buffer 223. The matrix operation array 221 writes the result C data 209 to the DRAM 205 according to the address generated by the write address generation unit 222. When all the matrix operations are completed, the FPGA 204 writes the result C data 209 to the memory 202 by the DMAC224. Further, the optimum submatrix size calculated by the submatrix shape determination circuit 215 is sent to the log output unit 225. The log output unit 225 generates the log data 210 and transfers it to the memory 202 using the DMAC224.

<Upper algorithm for determining submatrix size by submatrix shape determination circuit 215>
FIG. 3 is a flowchart showing a higher-level algorithm for determining the submatrix size by the submatrix shape determining circuit 215. The submatrix shape determination circuit 215 sets the matrix A column size in the column size register of the matrix AB size register 214 (step S301). The submatrix shape determination circuit 215 sets the matrix A row size in the row size register of the matrix AB size register 214 (step S302).

The submatrix shape determination circuit 215 determines the number of columns pCOL and the number of rows pROW of the submatrix of the matrix A, and the horizontal Ha and vertical Va of the matrix A (step S303). The number of columns pCOL of the matrix B is the number of columns in the row direction in the submatrix of the matrix A, and the number of rows pROW in the matrix B is the number of rows in the column direction in the submatrix of the matrix A. The horizontal Ha of the matrix A is the number of submatrixes in the horizontal H (row direction) of the matrix A divided by the submatrix, and the vertical Va of the matrix A is the vertical V (column) of the matrix A divided by the submatrix. The number of submatrixes in direction). Details of step S303 will be described later in FIG.

The submatrix shape determination circuit 215 sets the matrix B row size in the column size register of the matrix AB size register 214 (step S304). The submatrix shape determination circuit 215 sets the matrix B column size in the row size register of the matrix AB size register 214 (step S305). Since the matrix B is transposed and DA, in steps S304 and S305, unlike steps S301 and S302, the row size of the matrix B is set in the column size register and the column size of the matrix B is set in the row size register. To.

The submatrix shape determination circuit 215 determines the number of columns pCOL and the number of rows pROW of the submatrix of the matrix B, and the horizontal Hb and vertical Vb of the matrix B (step S306). The number of columns pCOL of the matrix B is the number of columns in the row direction in the submatrix of the matrix B, and the number of rows pROW in the matrix B is the number of rows in the column direction in the submatrix of the matrix B. The horizontal Hb of the matrix B is the number of submatrixes in the horizontal H (row direction) of the matrix B divided by the submatrix, and the vertical Vb of the matrix B is the vertical V (column) of the matrix B divided by the submatrix. The number of submatrixes in direction). Details of step S306 will be described later in FIG.

The submatrix shape determination circuit 215 replaces the number of columns pCOL and the number of rows pROW of the submatrix of the matrix B, and replaces the horizontal Hb and the vertical Vb (step S307). The submatrix shape determination circuit 215 compares the number of columns pCOL of the submatrix of the matrix A with the number of columns pCOL of the submatrix of the matrix B, and selects the larger one as the number of columns pCOL of the submatrix (step S308).

The submatrix shape determination circuit 215 selects the number of columns pROW of the submatrix of the matrix (A or B) for which the number of columns pCOL is selected (step S309). When the matrix A is selected, the submatrix shape determination circuit 215 recalculates the horizontal Hb and the vertical Vb using the pCOL selected in step S308 and the pROW selected in step S309, and the matrix B is selected. If so, the horizontal Ha and vertical Va are recalculated using the pCOL selected in step S308 and the pROW selected in step S309 (step S310). As a result, a series of processes is completed.

<Submatrix size determination algorithm of submatrix shape determination circuit 215>
FIG. 4 is a flowchart showing a submatrix size determination algorithm by the submatrix shape determination circuit 215 in steps S303 and S306. The submatrix shape determination circuit 215 sets the maximum value that the minimum number of accesses can take as the initial value of the minimum number of accesses to the DRAM 205 (step S401). The submatrix shape determination circuit 215 sets the initial value “1” for the subsequence size (step S402).

The submatrix shape determination circuit 215 shifts the subsequence size by 1 bit to double the value (step S403). The submatrix shape determination circuit 215 compares the subsequence size with the value of the array size register 211 (step S404). As a result of comparison, when the subsequence size becomes equal to or larger than the array size (step S404: Yes), since the subsequence size larger than that cannot be used, the process proceeds to the termination process (steps S415 to S418). Other than that (step S404: No), the process proceeds to step S405.

The submatrix shape determination circuit 215 calculates the number of sense amplifier cache hits based on the sense amplifier cache size register 212, the column size register of the target matrix, and the current submatrix size (step S405). The target matrix is the matrix A if the processing of this flowchart is the processing in step S303, and the target matrix is the matrix B if the processing of this flowchart is the processing in step S306.

FIG. 5 is an explanatory diagram showing the relationship between the sense amplifier cache size, the column size, and the subsequence size. Specifically, the submatrix shape determination circuit 215 divides the value of the sense amplifier cache size register 212 (sense amplifier cache size) by the column size of the target matrix, and truncates the remainder of the division result. Then, the submatrix shape determination circuit 215 multiplies the division result after truncation of the remainder and the submatrix size of the submatrix. The submatrix shape determination circuit 215 adds the remainder of the sense amplifier cache size divided by the column size and the submatrix size, whichever is smaller, to the multiplication result. The result of this addition is the number of sense amplifier cache hits.

Returning to FIG. 4, the submatrix shape determination circuit 215 compares the number of sense amplifier cache hits calculated in step S405 with the value of the threshold register 213 (step S406). If the number of cache hits is smaller than the threshold (step S406: No), this subsequence size is determined to be ineligible, and the process returns to step S403. If the number of cache hits is greater than or equal to the threshold (step S: Yes), the process proceeds to step S407.

The submatrix shape determination circuit 215 sets the number of columns pCOL of the submatrix of the target matrix to the submatrix size (step S407). The submatrix shape determination circuit 215 calculates the number of rows pROW of the submatrix of the target matrix by dividing the value of the array size register 211 by the number of columns pCOL set in step S407 (step S408).

The submatrix shape determination circuit 215 obtains the lateral H, which is the number of submatrixes in the lateral direction (step S409). Specifically, for example, the submatrix shape determination circuit 215 calculates the lateral H by dividing the value of the column size register of the target matrix by the number of columns pCOL and rounding up.

The submatrix shape determination circuit 215 obtains the vertical V, which is the number of submatrixes in the vertical direction (step S410). Specifically, for example, the submatrix shape determination circuit 215 calculates the vertical V by dividing the value of the row size register of the target matrix by the number of rows pROW and rounding up.

The submatrix shape determination circuit 215 multiplies the horizontal H calculated in step S409 by the vertical V calculated in step S410 to obtain the number N of submatrix covering the target matrix (step S411). The submatrix shape determination circuit 215 compares the number of submatrix N with the minimum number of accesses (step S412). If the number of submatrix N is larger than the minimum number of accesses (step S412: No), the process returns to step S403 as ineligible. In other cases (step S412: Yes), the current number of submatrix N is qualified and the process proceeds to step S413.

The submatrix shape determination circuit 215 substitutes the value of the current number of columns pCOL into the variable number of selected columns pCOL to obtain a tentative determination value (step S413). The submatrix shape determination circuit 215 substitutes the current number of submatrix N for the minimum number of accesses (step S414), and returns to step S403. By this loop, as shown in FIG. 1, the first submatrix 111 is initially selected as the submatrix, but it is updated to the second submatrix 121 in which the minimum number of accesses (N = V × H) becomes smaller. Will be.

As described above, when the subsequence size becomes equal to or larger than the array size in step S404 (step S404: Yes), since the subsequence size larger than that cannot be used, the process proceeds to the end process (steps S415 to S418). The current number of selected columns pCOL at the time of transition to the end processing is the optimum number of columns pCOL for the current matrix size.

The submatrix shape determination circuit 215 sets the current number of selected columns pCOL as the determined number of columns pCOL (step S415). The submatrix shape determination circuit 215 divides the value of the array size register 211 by the number of columns pCOL in step S415 to calculate the value of the number of rows pROW (step S416).

The submatrix shape determination circuit 215 divides the value of the column size register by the number of columns pCOL in step S415 and rounds up to calculate the lateral H, which is the number of submatrixes in the horizontal direction of the target matrix (step S417). The submatrix shape determination circuit 215 divides the value of the row size register by the number of rows pROW in step S416 and rounds up to calculate V, which is the number of submatrixes in the vertical direction of the target matrix (step S418). As a result, a series of processes is completed.

<Multiplication execution algorithm using submatrix by submatrix shape determination circuit 215>
FIG. 6 is a flowchart showing a multiplication execution algorithm using a submatrix by the submatrix shape determination circuit 215. The submatrix shape determination circuit 215 calculates a numerical value that specifies a region to be filled with zeros (step S601). Ca is a specified value of the zero-filled range in the column direction of the matrix A, and the remainder obtained by dividing the value of the column A size register in the matrix AB size register 214 by the number of columns pCOL of the matrix A determined in step S303 is set. To. The specified value of the zero-filled range is a value that specifies how far the submatrix at that time is zero-filled.

Ra is a specified value of the 0-filled range in the row direction of the matrix A, and the remainder obtained by dividing the value of the A row size register in the matrix AB size register 214 by the number of rows pROW of the matrix A determined in step S303 is set. To. Cb is a specified value of the 0-filled range in the column direction of the matrix B, and the remainder obtained by dividing the value of the B column size register in the matrix AB size register 214 by the number of columns pCOL of the matrix B after the replacement in step S307 is set. Will be done. Rb is a specified value of the 0-filled range in the row direction of the matrix B, and the remainder obtained by dividing the value of the B row size register in the matrix AB size register 214 by the number of rows pROW of the matrix B after the replacement in step S307 is set. Will be done.

The submatrix shape determination circuit 215 initially sets all the loop variables Ar, Bc, and Ac to 0 (step S602). The submatrix shape determination circuit 215 reads the data corresponding to the submatrix determined in the processes of FIGS. 3 and 4 from the matrix A and the matrix B, respectively, and transfers the data to the read buffer 218 (step S603). The submatrix shape determination circuit 215 compares the loop variables Ac and Ha-1, determines that they are the ends of the matrix A if they are the same (step S604: Yes), proceeds to step S605, and otherwise (steps to step S605). Step S604: No), the process proceeds to step S606.

The submatrix shape determination circuit 215 sets the Ca valid flag to ON, sets the Rb valid flag to ON (step S605), and proceeds to step S606. As a result, Ca is set in the zero-filled parameter of the column of the matrix A, and Rb is set in the zero-filled parameter of the row of the matrix B.

The submatrix shape determination circuit 215 compares the loop variables Bc and Hb-1, determines that they are the ends of the matrix B if they are the same (step S606: Yes), proceeds to step S607, and otherwise (steps to step S607). Step S606: No), the process proceeds to step S608. The submatrix shape determination circuit 215 sets the Cb valid flag to ON (step S607), and proceeds to step S608. As a result, Cb is set in the zero-filled parameter of the column of the matrix B.

The submatrix shape determination circuit 215 compares the loop variables Ar and Va-1, determines that they are the ends of the matrix A if they are the same (step S608: Yes), proceeds to step S609, and otherwise (steps to step S609). Step S608: No), the process proceeds to step S610. The submatrix shape determination circuit 215 sets the Ra valid flag to ON (step S609), and proceeds to step S610. As a result, Ra is set in the zero-filled parameter of the row of the matrix A.

The submatrix shape determination circuit 215 performs a process of starting the multiplication process of the submatrix (step S610). As a result, the matrix operation array 221 operates and multiplication for the submatrix is performed. The submatrix shape determination circuit 215 increments the value of the loop variable Ac (step S611). The submatrix shape determination circuit 215 compares the loop variable Ac with the vertical Va of the matrix A (step S612), and if they are the same (step S612: Yes), the process proceeds to step S613, and otherwise (step S612:). No), the process proceeds to step S614. In step S613, the submatrix shape determination circuit 215 increments the value of the loop variable Bc, sets the loop variable Ac to 0 (step S613), and proceeds to step S614.

The submatrix shape determination circuit 215 compares the loop variable Bc with the lateral Hb of the matrix B (step S614), and if they are the same (step S614: Yes), the process proceeds to step S615, and otherwise (step S614: No), the process proceeds to step S616. In step S615, the submatrix shape determination circuit 215 increments the loop variable Ar, sets the loop variable Bc to 0 (step S615), and proceeds to step S616.

The submatrix shape determination circuit 215 compares the loop variable Ar with the vertical Va of the matrix A (step S616), and if they are the same (step S616: Yes), all the processes are completed. Otherwise (step S616: No), the process returns to step S603 and the process continues.

<Processing after submatrix shape determination circuit 215>
FIG. 7 is an explanatory diagram showing a processing example after the submatrix shape determination circuit 215. The addition control circuit 216 receives inputs of Ca, Ra, Cb and Rb from the submatrix shape determination circuit 215. Further, the addition control circuit 216 receives inputs of the Ca valid flag 701, the Ra valid flag 702, the Cb valid flag 703, and the Rb valid flag 704 from the submatrix shape determination circuit 215. These valid flags are flags indicating whether or not the submatrix at that time has zero padding. Further, the addition control circuit 216 receives input from the submatrix shape determination circuit 215 of the number of columns pCOL of the submatrix selected in step S308 and the number of rows pROW of the submatrix selected in step S309.

The read buffer 218 reads the submatrix A00, A01, ... Of the matrix A data 207 from the DRAM 205, and holds the submatrix B00, B01, ... Of the matrix B data 208. There are as many submatrixes A00, A01, ... As there are submatrix sizes. Similarly, there are as many submatrixes B00, B01, ... As there are submatrix sizes. The read buffer 218 has a read port for each of all submatrixes and is configured to read all submatrixes at once.

The selector array 705 has a plurality of selectors 220. Each selector 220 selects either a submatrix from the read buffer 218 or a zero value. There are as many selectors 220 as there are submatrixes in the read buffer 218. For example, each selector 220 selects a 0 value if there is an instruction to fill in 0 from the addition control circuit 216, and selects a submatrix from the read buffer 218 if there is no instruction to fill in 0. Each selector 220 outputs the selected data to the multiplier 706 of the matrix operation array 221.

The matrix operation array 221 is composed of a plurality of multipliers 706 and a plurality of adders 707. The multiplier 706 performs multiplication in the matrix operation array 221. One multiplier 706 exists for a combination of one submatrix of the matrix A data 207 and one submatrix of the matrix B data 208, and outputs the multiplication result. The adder 707 aggregates the multiplication results of the multiplier 706. The plurality of adders 707 are arranged in a tree shape. Details will be described with reference to FIG.

The write buffer 223 holds submatrixes C00, C01, C02, C03, ... Of the result C data 209 from the matrix operation array 221. The selector 708 selects a submatrix of the result C data 209 stored in the write buffer 223 and writes it to the DRAM 205.

The operation after the submatrix shape determination circuit 215 is as follows. First, the addition control circuit 216 submatrixes Ca, Ra, Cb and Rb, Ca valid flag 701, Ra valid flag 702, Cb valid flag 703 and Rb valid flag 704, and the number of columns pCOL and the number of rows pROW. Obtained from the shape determination circuit 215. The addition control circuit 216 sets the select value of each selector 220 of the selector array 705 based on the acquired data.

Next, the selector array 705 reads all the sub-matrixes A00, A01, ... Of the matrix A and the sub-matrix B00, B01, ... Of the matrix B from the read buffer 218. The selector array 705 selects either a 0 value or a submatrix according to the select value, and outputs the selection result to the matrix operation array 221. At this point, zero padding of the data is completed.

Next, the matrix operation array 221 multiplies the submatrix of the matrix A and the submatrix of the matrix B by the multiplier 706 and adds them by the adder 707. The matrix operation array 221 writes the submatrix of the result C data 209, which is the output from the adder 707, to the write buffer 223. Then, the selector 708 selects and writes to the DRAM 205.

<Operation example of matrix operation array 221>
FIG. 8 is an explanatory diagram showing an operation example of the matrix operation array 221. In FIG. 8, for simplification of the description, a case where the array size is 16 (16 multipliers 706) will be described as an example, but if it is a power of 2, the array size is not limited to 16. In the matrix operation array 221, 16 sets of matrix A and matrix B elements having an array size are arranged, and a multiplier 706 is used to obtain a multiplication result for these pairs. After that, a tree of adder 707 that adds the adjacent multiplication results to each other is constructed. The matrix operation array 221 has selectors 801A to 801D at the uppermost level of the tree-shaped adder 707.

Selector 801A adds the addition result of adding the multiplication results from the two multipliers 706, the addition result of adding the multiplication results from the four multipliers 706, and the multiplication result from the eight multipliers 706. The added result and the added result obtained by adding the multiplication results from the 16 multipliers 706 are selectively written to the DRAM 205.

Selector 801B adds the addition result of adding the multiplication results from the two multipliers 706, the addition result of adding the multiplication results from the four multipliers 706, and the multiplication result from the eight multipliers 706. The added result and the result of the addition are selectively written to the DRAM 205.

The selectors 801C and 801D selectively write the addition result obtained by adding the multiplication results from the two multipliers 706 and the addition result obtained by adding the multiplication results from the four multipliers 706 to the DRAM 205. The addition result obtained by adding the multiplication results from the two multipliers 706 not selected by the selectors 801A to 801D is written from the adder 707 to the DRAM 205.

Reference numeral 816 is a data range selected when the number of columns pCOL of the submatrix selected in step S308 is 16, and reference numeral 808 is a case where the number of columns pCOL of the submatrix selected in step S308 is 8. Reference numeral 804 is a data range to be selected, reference numeral 804 is a data range selected when the number of columns pCOL of the submatrix selected in step S308 is 4, and reference numeral 802 is a submatrix selected in step S308. This is the data range selected when the number of columns pCOL is 2.

That is, the selectors 801A to 801D select the addition result according to the value of the number of columns pCOL. With this configuration, when the number of columns pCOL is 16, the matrix operation array 221 operates so as to output the addition result from the fourth-stage adder 707 by the selector 801A. When the number of columns pCOL is 8, the matrix operation array 221 operates so as to output two addition results from the two adders 707 in the third stage by the selectors 801A and 801B.

When the number of columns pCOL is 4, the matrix operation array 221 operates so as to output the four addition results from the four adders 707 in the second stage by the selectors 801A to 801D. When the number of columns pCOL is 2, the matrix operation array 221 operates so as to output eight addition results from the eight adders 707 in the first stage by the selectors 801A to 801D.

<Details of register group>
FIG. 9 is a block diagram showing details of the register group. The register group includes the array size register 211, the sense amplifier cache size register 212, the threshold register 213, the matrix AB size register 214, and the matrix operation start register 907 described above.

The address decoder 901 acquires the address from the processor 201 via the general-purpose bus 203 and outputs the write enable signal we to each register. The data latch 902 acquires data from the processor 201 via the general-purpose bus 203 and holds the data until it is written to a target register.

Specifically, for example, the address decoder 901 and the data latch 902 write data to the register corresponding to the address for the address-data pair from the processor 201. The address decoder 901 operates to take that correspondence, identifies the write to each register, and outputs a write enable signal we to each register.

The matrix AB size register 214 includes a matrix A column size register 903 that specifies the column size of the matrix A, a matrix A row size register 904 that specifies the row size of the matrix A, and a matrix B column that specifies the column size of the matrix B. It has a size register 905 and a matrix B row size register 906 that specifies the row size of the matrix B.

The matrix operation start register 907 is a register in which the start value is written as data. When the start value is written, the execution of a series of matrix operations is started. With this configuration, it is possible to write data from the processor 201 to each register, and it is possible to control the FPGA 204 from the processor 201.

<Control processing by control program 206>
FIG. 10 is a flowchart showing an example of control processing by the control program 206. The processor 201 sets the array size in the array size register 211 (step S1001). The processor 201 sets the sense amplifier cache size in the sense amplifier cache size register 212 (step S1002). The processor 201 sets the lower limit of the number of hits of the sense amplifier cache allowed in the threshold register 213 (step S1003).

Processor 201 waits for a call from a main program (eg, deep learning) that uses matrix operations (step S1004: No). When called (step S1004: Yes), the process proceeds to step S1005.

The processor 201 sets the column size of the matrix A specified by the main program in the matrix A column size register 903 (step S1005). The processor 201 sets the row size of the matrix A specified by the main program in the matrix A row size register 904 (step S1006). The processor 201 sets the column size of the matrix B specified by the main program in the matrix B column size register 905 (step S1007). The processor 201 sets the row size of the matrix B specified by the main program in the matrix B row size register 906 (step S1008).

The processor 201 sets the pointer of the matrix A data 207 to the DMAC224 (step S1009). The processor 201 sets the pointer of the matrix B data 208 to the DMAC224 (step S1010). The processor 201 writes a start value in the matrix operation start register 907 and causes the FPGA 204 to start the matrix operation (step S1011). At this time, first, the DMAC 224 transfers the matrix A data 207 to the DRAM 205. Next, the DMAC224 transfers the matrix B data 208 to the DRAM 205. At this time, the matrix B data 208 is transferred by transposing the columns and rows. After that, the submatrix shape determination circuit 215 is started and executed, and the setting of the number of columns pCOL and the like is calculated. Matrix operations are performed using that setting.

Processor 201 waits for the matrix operation in FPGA 204 to finish (step S1012). The processor 201 sets the pointer of the result C data 209 to DMAC224 (step S1013). The processor 201 activates the DMAC 224 and transfers the result C data 209 in the DRAM 205 to the memory 202 (step S1014). The processor 201 waits for the end of DMA, and when it finishes, returns the end of the matrix operation. This completes one matrix operation, returns to step S1004, and waits for the next call.

<Control processing of log output unit 225>
FIG. 11 is an explanatory diagram showing an example of control processing of the log output unit 225. The log data 210 includes an in-log time stamp 1101, an in-log matrix A size 1102, an in-log matrix B size 1103, and an in-log sub-matrix size 1104. The in-log time stamp indicates the time when the log was generated, that is, the time when the submatrix size was determined. The matrix A size in the log indicates the size of the matrix A used to determine the size of the submatrix. The in-log matrix B size indicates the size of the matrix B used to size the submatrix. The submatrix size in the log indicates the submatrix size determined at this point.

Further, the log output unit 225 has a log generation unit 1105 and a log temporary storage memory 1106. The log generation unit 1105 acquires, for example, the matrix A size, the matrix B size, and the submatrix size as necessary information from the submatrix shape determination circuit 215, and obtains the current log data 210 together with the time acquired from the arithmetic unit 200. Generate and transfer to the log temporary storage memory 1106. Further, the log generation unit 1105 activates the DMAC 224 for transferring the log data 210 to the memory 202. The log temporary storage memory 1106 stores the log data 210 generated by the log generation unit 1105 until the DMAC224 reads it.

The log output unit 225 operates as follows. At the timing when the submatrix shape determination circuit 215 determines the shape of the submatrix, the log generation unit 1105 generates log data 210 composed of time, matrix A size, matrix B size, and submatrix size, and temporarily stores the log. Store in 1106. At the same time, the log generation unit 1105 activates the DMAC 224 and transfers the temporarily saved log data 210 to the memory 202. As a result, it is recorded from what size of matrix A and matrix B the submatrix size determined for each matrix operation execution is generated.

With the above configuration, matrix operations using the optimum submatrix shape are executed on the offload device, and matrix multiplication of various sizes can be performed at high speed. That is, it is possible to perform calculations at higher speed and with higher power efficiency with the same circuit configuration.

Further, the above-mentioned arithmetic unit 200 can also be configured as described in (1) to (11) below.

(1) The arithmetic apparatus 200 that can access the DRAM 205 that stores the matrix data (matrix A data 207, matrix B data 208) has a partial matrix shape determination circuit 215 and a matrix arithmetic array 221.

The sub-matrix shape determination circuit 215 increases the sub-column size, which is the number of columns in the row direction of the sub-matrix, which is the division unit of the matrix data, and the increased sub-column size becomes equal to or more than the number of the plurality of multipliers 706. In the case, the subsequence size, which is the number of rows in the column direction of the submatrix, is obtained based on the subcolumn size before the increase and the number of the plurality of multipliers 706 (array size), and the subsequence size (pCOL) and the subsequence are obtained. The shape (pCOL, pROW) of the subsequence composed of the size (pROW) is determined.

The matrix calculation array 221 has a plurality of multipliers 706 and a plurality of adders 707, and is a first submatrix having a shape determined by the submatrix shape determination circuit 215 of the matrix A data 207 stored in the DRAM 205. (For example, A00) and the second submatrix (for example, B00) having a shape determined by the submatrix shape determination circuit 215 of the matrix B data 208 stored in the DRAM 205 are subjected to a multiply-accumulate operation.

Thereby, the matrix operation array 221 can be set to the ratio of the allocation in the row direction and the allocation in the column direction of the plurality of multipliers 706 and the plurality of adders 707 according to the shape of the submatrix. The speed can be increased.

(2) In the arithmetic unit 200 of the above (1), the submatrix shape determination circuit 215 determines the submatrix size of the first submatrix (for example, A00) determined for the matrix A data 207 and the matrix B data 208. The larger of the submatrix size of the second submatrix (for example, B00) is determined to be the shape of the submatrix (S308, S309).

Thereby, a common submatrix shape applicable to the matrix A data 207 and the matrix B data 208 can be obtained.

(3) In the arithmetic unit 200 of the above (1), the submatrix shape determination circuit 215 is a matrix with a plurality of submatrix based on the subcolumn size and the column size which is the number of columns in the row direction of the matrix data. The size of the area containing the data in the row direction (horizontal H) is determined, and the matrix data is included in a plurality of sub-matrix based on the partial row size and the row size which is the number of columns in the column direction of the matrix data. Determine the size (vertical V) of the area to be used in the column direction.

As a result, the matrix data can be filled with submatrix as much as possible, and the number of matrix operations can be reduced.

(4) The arithmetic unit 200 of the above (3) has an addition control circuit 216 that allocates a 0 value to a range in which a submatrix is not assigned, and the matrix operation array 221 has a 0 value by the addition control circuit 216. The product-sum operation is executed after the allocation of. As a result, the adverse effect on the matrix operation can be suppressed.

(5) The arithmetic unit 200 of the above (3) has a matrix AB size register 214 for setting the column size and the row size each time the matrix data is input, and the submatrix shape determination circuit 215 inputs the matrix data. Each time the submatrix is formed, the shape of the submatrix, the size of the area in the row direction (horizontal H), and the size of the area in the column direction (vertical V) are determined.

As a result, it is possible to perform the matrix operation dynamically and efficiently by optimally allocating the plurality of multipliers 706 and the plurality of adders 707 for each input matrix data according to the shape of the input matrix data. it can.

(6) In the arithmetic unit 200 of the above (1), the submatrix shape determination circuit 215 has a cache size of the sense amplifier of the DRAM 205, a column size which is the number of columns in the row direction of the matrix data, a subcolumn size, and the like. The number of cache hits of the sense amplifier is obtained based on the above, and if the number of cache hits is not equal to or greater than the threshold value (value of threshold register 213), the submatrix size is increased (S405, S406: No, S403).

The access time of the DRAM 205 changes depending on the conditions. The data in the same row in the same sense amplifier cache can be accessed at high speed, while the data in different rows is written back to the data currently in the sense amplifier cache and the data in the target row is read out. Therefore, the access time is up to 10 times longer. As described above, since the access time varies greatly depending on the access pattern, the shape of the submatrix can be changed according to the sense amplifier cache size by calculating the number of cache hits as the lower limit of the shape of the submatrix.

(7) In the arithmetic unit 200 of the above (6), when the number of cache hits is equal to or greater than the threshold value, the subsequence shape determination circuit 215 has the subsequence size (pCOL) before the increase and the number of the plurality of multipliers 706. Based on (array size), the subsequence size, which is the number of rows in the column direction of the subsequence, is obtained, and the shape of the subsequence is tentatively determined by the subsequence size (pCOL) and the subsequence size (pLOW) (step). S407, S408), when the tentatively determined subsequence size is increased and the increased subsequence size is equal to or greater than the number (array size) of a plurality of multipliers 706, the tentatively determined subsequence size (pCOL) Based on the above and the number of multipliers 706 (array size), the subsequence size, which is the number of rows in the column direction of the subsequence, is obtained, and the shape of the subsequence is determined by the subsequence size and the subsequence size (). Steps S415 and S416).

As a result, the submatrix shape determination circuit 215 can search for a subsequence size such that the number of cache hits is equal to or greater than the threshold and the number of accesses to the DRAM 205 is minimized until the subsequence size becomes equal to or greater than the array size. .. Therefore, the number of matrix operations can be reduced.

(8) In the arithmetic unit 200 of the above (7), the submatrix shape determination circuit 215 is a matrix with a plurality of submatrix based on the subcolumn size and the column size which is the number of columns in the row direction of the matrix data. The size of the area containing the data in the row direction (horizontal H) is tentatively determined, and the matrix data is divided into a plurality of sub-matrix based on the partial row size and the row size which is the number of columns in the column direction of the matrix data. The size of the area to be included in the column direction (vertical V) is tentatively determined, and when the multiplication result N of both tentatively determined sizes (horizontal H and vertical V) is equal to or less than the minimum number of accesses to the matrix data, the minimum number of accesses. Is updated to the multiplication result N (steps S409 to S414).

As a result, the submatrix shape determination circuit 215 has a row size (horizontal H) of the region in which the number of cache hits is equal to or greater than the threshold and the number of accesses to the DRAM 205 is minimized until the subsequence size becomes equal to or greater than the array size. , Vertical V) can be searched. Therefore, the area to be filled with 0 can be reduced with respect to the size of the matrix data, and the reading efficiency from the DRAM 205 can be improved.

(9) The arithmetic unit 200 of the above (1) is FPGA 204. As a result, the load on the processor 201 can be reduced.

(10) The arithmetic unit 200 of the above (1) includes an FPGA 204 including a submatrix shape determination circuit 215 and a matrix operation array 221, a processor 201 that executes a control program 206 for controlling the FPGA 204, a control program 206, and a matrix. It has a memory 202 for storing a matrix operation result (result C data) by the operation array 221. As a result, the FPGA 204 can be controlled.

(11) In the arithmetic unit 200 of the above (10), the FPGA 204 includes a log generation unit 1105 that generates log data 210 indicating the execution contents of the submatrix shape determination circuit 215, and the memory 202 is generated by the log generation unit 1105. The log data 210 that has been created is stored. Thereby, the execution contents of the submatrix shape determination circuit 215 can be confirmed.

The present invention is not limited to the above-described examples, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the configurations described. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration.

In addition, the control lines and information lines indicate those that are considered necessary for explanation, and do not necessarily indicate all the control lines and information lines necessary for implementation. In practice, it can be considered that almost all configurations are interconnected.

200 Arithmetic Logic Unit 201 Processor 202 Memory 205 DRAM
206 Control program 207 Matrix A data 208 Matrix B data 209 Result C data 210 Log data 211 Array size register 212 Sense amplifier cache size register 213 Threshold register 214 Matrix AB size register 215 Partial matrix shape determination circuit 216 Adder control circuit 221 Adder operation array 225 Log output unit 706 Multiplier 707 Adder

Claims (12)

An arithmetic unit that can access a storage device that stores matrix data.
When the subcolumn size, which is the number of columns in the row direction of the submatrix that is the division unit of the matrix data, is increased and the subcolumn size after the increase becomes equal to or greater than the number of a plurality of multipliers, the subcolumn before the increase Based on the size and the number of the plurality of multipliers, the partial row size, which is the number of rows in the column direction of the submatrix, is obtained, and the shape of the submatrix composed of the submatrix size and the partial row size. And the decision-making part that decides
The first submatrix having the shape determined by the determination unit among the first matrix data stored in the storage device and having the plurality of multipliers and the plurality of adders, and stored in the storage device. Of the second matrix data, the matrix calculation unit that executes the product-sum operation of the second submatrix of the shape determined by the determination unit, and
An arithmetic unit characterized by having. The arithmetic unit according to claim 1.
The determination unit has the larger shape of the submatrix size of the first submatrix determined for the first matrix data and the submatrix size of the second submatrix determined for the second matrix data. To the shape of the submatrix,
An arithmetic unit characterized by that. The arithmetic unit according to claim 1.
Based on the subcolumn size and the column size which is the number of columns in the row direction of the matrix data, the determination unit determines the size of the area including the matrix data in the plurality of the submatrix in the row direction. Based on the partial row size and the row size which is the number of columns of the matrix data in the column direction, the size of the region including the matrix data in the plurality of the sub-matrix is determined in the column direction. decide,
An arithmetic unit characterized by that. The arithmetic unit according to claim 3.
It has a control unit that assigns a 0 value to a range of the area to which the submatrix is not assigned.
The matrix calculation unit executes the product-sum operation after the zero value is assigned by the control unit.
An arithmetic unit characterized by that. The arithmetic unit according to claim 3.
It has a setting unit for setting the column size and the row size each time the matrix data is input.
Each time the matrix data is input, the determination unit determines the shape of the submatrix, the size of the region in the row direction, and the size of the region in the column direction.
An arithmetic unit characterized by that. The arithmetic unit according to claim 1.
The determination unit determines the number of cache hits of the sense amplifier based on the cache size of the sense amplifier of the storage device, the column size which is the number of columns in the row direction of the matrix data, and the subsequence size. If the number of cache hits is not equal to or greater than the threshold value, the subsequence size is increased.
An arithmetic unit characterized by that. The arithmetic unit according to claim 6.
When the number of cache hits is equal to or greater than the threshold value, the determination unit is the number of rows in the column direction of the submatrix based on the subsequence size before the increase and the number of the plurality of multipliers. The submatrix size is obtained, and the shape of the submatrix is tentatively determined by the substring size and the submatrix size.
When the tentatively determined subsequence size is increased and the increased subsequence size becomes equal to or greater than the number of a plurality of multipliers, it is based on the tentatively determined subsequence size and the number of the plurality of multipliers. The subsequence size, which is the number of rows in the column direction of the subsequence, is obtained, and the shape of the subsequence is determined by the subsequence size and the subsequence size.
An arithmetic unit characterized by that. The arithmetic unit according to claim 7.
The determination unit is based on the subcolumn size and the column size which is the number of columns of the matrix data in the row direction, and the size of the region including the matrix data in the plurality of the submatrix in the row direction. Is tentatively determined, and based on the partial row size and the row size which is the number of columns in the column direction of the matrix data, the size of the region including the matrix data in the plurality of the sub-matrix in the column direction. Is tentatively determined, and when the tentatively determined multiplication result of both sizes is equal to or less than the minimum number of accesses to the matrix data, the minimum number of accesses is updated to the multiplication result.
An arithmetic unit characterized by that. The arithmetic unit according to claim 1.
The arithmetic unit is an offload device.
An arithmetic unit characterized by that. The arithmetic unit according to claim 1.
An offload device including the determination unit and the matrix calculation unit, and
A processor that executes a control program for controlling the offload device, and
A memory for storing the control program and the matrix calculation result by the matrix calculation unit, and
An arithmetic unit characterized by having. The arithmetic unit according to claim 10.
The offload device includes a generation unit that generates log data indicating the execution content of the determination unit.
The memory stores log data generated by the generation unit.
An arithmetic unit characterized by that. An arithmetic method performed by an arithmetic unit that has access to a storage device that stores matrix data.
The arithmetic unit
When the subcolumn size, which is the number of columns in the row direction of the submatrix that is the division unit of the matrix data, is increased and the subcolumn size after the increase becomes equal to or greater than the number of a plurality of multipliers, the subcolumn before the increase Based on the size and the number of the plurality of multipliers, the partial row size, which is the number of rows in the column direction of the submatrix, is obtained, and the shape of the submatrix composed of the submatrix size and the partial row size. Decide,
A first submatrix having the determined shape among the first matrix data stored in the storage device, which has the plurality of multipliers and the plurality of adders, and a first submatrix stored in the storage device. The product-sum operation of the second submatrix of the determined shape of the two matrix data is executed.
A calculation method characterized by that.

Images