JP7241397B2 - Arithmetic Device, Arithmetic Method, and Arithmetic Program - Google Patents

Description

The present invention relates to an arithmetic device, an arithmetic method, and an arithmetic program.

In various applications such as numerical computation and deep learning, matrix-matrix multiplication (hereinafter referred to as "matrix multiplication") and matrix-vector multiplication occupy most of the computational complexity. Therefore, an arithmetic device and an arithmetic method for efficiently executing such matrix arithmetic have been developed (see Patent Documents 1 to 3). Processors have also been developed that are capable of performing matrix operations.
[Prior art documents]
[Patent Literature]
[Patent Document 1] International Publication No. 2018/207926 [Patent Document 2] JP-A-2018-139045 [Patent Document 3] JP-A-2018-197906

The matrix-vector product of an n-dimensional square matrix and an n-dimensional vector involves n ² multiplications and approximately n ² additions, resulting in a computational complexity of approximately 2n ² . Therefore, when the n-dimensional square matrix is fixed, the amount of computation of the matrix-vector product becomes n ² orders for the input of the n-dimensional vector. Therefore, if the matrix size is increased to increase the size of the matrix calculator, the ratio of the amount of data load to the amount of calculation can be reduced. However, if the matrix calculator is made larger, the load/store capability of the register file or the like becomes relatively low, and the processing performance of small-sized matrix operations and operations other than matrices becomes relatively low.

In order to solve the above problems, a first aspect of the present invention provides an arithmetic device. The arithmetic device may include a vector storage unit that stores at least the first partial vector among the first plurality of partial vectors obtained by dividing the first vector. The arithmetic unit has a matrix storage unit that stores at least a first submatrix by which the first partial vector is to be multiplied, among a plurality of first submatrices obtained by dividing the first matrix by which the first vector is to be multiplied in the row direction and the column direction. Be prepared. The arithmetic unit includes a pipeline arithmetic unit capable of executing an arithmetic operation of adding an intermediate vector to the matrix-vector product of the partial matrix stored in the matrix storage unit and the partial vector stored in the vector storage unit. you can In the arithmetic device, the pipeline operation unit performs another matrix-vector product operation using the first partial vector or the first partial matrix during the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector. An arithmetic control unit may be provided for instructing execution to the pipeline arithmetic unit.

The vector storage unit may further store the second partial vector among the first plurality of partial vectors. The matrix storage unit may further store a second partial matrix by which the second partial vector is to be multiplied, among the first plurality of partial matrices. After the cycle in which the operation result of the matrix-vector product of the first partial matrix and the first partial vector becomes available without delay, the operation control unit performs the matrix-vector product of the second partial matrix and the second partial vector as the first partial matrix. The pipeline operation unit may be instructed to perform an operation to be added to the operation result of the matrix-vector product of the matrix and the first partial vector.

The vector storage unit may further store a third partial vector by which the first partial matrix is to be multiplied, among a plurality of second partial vectors obtained by dividing the second vector by which the first matrix is to be multiplied. During the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector, the operation control unit performs the operation of the matrix-vector product of the first partial matrix and the third partial vector as another matrix-vector product operation. Execution may be instructed to the pipeline operation unit.

The first vector and the second vector may be column vectors contained in the second matrix to be multiplied by the first matrix.

The vector storage unit may store a plurality of second vectors included in the second matrix. The operation control unit controls each cycle from the start of the pipeline operation of the matrix-vector product of the first submatrix and the first partial vector until the operation result becomes available without delay for the first submatrix and the plurality of submatrices. may be filled with a matrix-vector product operation of the third subvector from each of the second vectors of .

The matrix storage unit may further store a third submatrix by which the first subvector is to be multiplied, among the plurality of first submatrices. During the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector, the operation control unit performs the operation of the matrix-vector product of the third partial matrix and the first partial vector as another matrix-vector product operation. Execution may be instructed to the pipeline operation unit.

The matrix storage unit may store a plurality of third submatrices. The operation control unit controls each cycle from the start of the pipeline operation of the matrix-vector product of the first submatrix and the first partial vector until the operation result becomes available without delay for the plurality of third submatrices. and a matrix-vector product operation of each of and the first subvector.

In a second aspect of the invention, a computation method is provided. The computing method may include storing at least the first partial vector among the first plurality of partial vectors obtained by dividing the first vector in the vector storage unit. In the calculation method, the matrix storage unit stores at least a first submatrix by which the first subvector is to be multiplied, among a plurality of submatrices obtained by dividing the first matrix by which the first vector is to be multiplied in the row direction and the column direction. be prepared to do so. The computation method includes a pipeline computation unit capable of executing a computation of adding an intermediate vector to a matrix-vector product of a partial matrix stored in the matrix storage unit and a partial vector stored in the vector storage unit, During the pipeline operation of the first sub-matrix and the matrix-vector product of the first sub-vector, it may comprise initiating execution of another matrix-vector product operation with the first sub-vector or the first sub-matrix.

A third aspect of the present invention provides a computing program executed by a computing device. The arithmetic device may include a vector storage unit that stores at least the first partial vector among the first plurality of partial vectors obtained by dividing the first vector. The arithmetic unit stores at least a first submatrix by which the first partial vector is to be multiplied, among a plurality of first submatrices obtained by dividing the first matrix by which the first vector is to be multiplied, in the row direction and the column direction. good. The arithmetic unit includes a pipeline arithmetic unit capable of executing an arithmetic operation of adding an intermediate vector to the matrix-vector product of the partial matrix stored in the matrix storage unit and the partial vector stored in the vector storage unit. you can The computing program causes the computing device to perform another matrix-vector product operation using the first partial vector or the first submatrix during the pipeline operation of the first submatrix and the matrix-vector product of the first partial vector. It can be for getting started.

It should be noted that the above summary of the invention does not list all the necessary features of the invention. Subcombinations of these feature groups can also be inventions.

4 shows an example of matrix computation according to the present embodiment. An example of a calculation formula in which the matrix operation according to the present embodiment is decomposed into a matrix-vector product of partial matrices and partial vectors is shown. 3 shows the configuration of an arithmetic device 300 according to the present embodiment. A first example of pipeline processing by the arithmetic device 300 according to the present embodiment is shown. A second example of pipeline processing by the arithmetic device 300 according to the present embodiment is shown. A third example of pipeline processing by the arithmetic device 300 according to the present embodiment is shown. A fourth example of pipeline processing by the arithmetic device 300 according to the present embodiment is shown. FIG. 8 illustrates an example computer 2200 in which aspects of the invention may be implemented in whole or in part.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution of the invention.

FIG. 1 shows an example of matrix computation according to this embodiment. This figure shows the matrix operation C=A×B that computes the matrix product of matrix A and matrix B and substitutes into matrix C. Matrices A, B, and C are square matrices with 8 rows and 8 columns.

a _ij (i=1, 2, . . . 8, j=1, 2, . . . 8) are the elements of matrix A (also referred to as “elements”). b _ij (i=1, 2, . . . 8, j=1, 2, . . . 8) are the elements of matrix B; c _ij (i=1, 2, . . . 8, j=1, 2, . . . 8) are the elements of matrix C; For the range of j≧3, illustration of each component of b _ij and c _ij is omitted.

Further, the column vector of the j column of the matrix B, that is, the vector having the components b _ij (i=1, 2, . . . 8) of the j column of the matrix B is the vector vbj, Denote vector vcj. That is, vector vbj=( ^b _1j , b _2j , . . . , b _8j ) ^T and vector vcj= ₍ c _1j , c _2j , . Then vector vcj can be calculated by matrix-vector product vcj=A*vbj of matrix A and vector vbj.

Here, for example, when using an arithmetic unit capable of executing a matrix-vector product of a matrix of 4 rows and 4 columns and a vector of 4 elements as a unit of operation, the arithmetic unit calculates the matrix operation C=A×B at one time. Divide into units that can be done. In the figure, submatrices obtained by dividing the matrix A into two in the row direction and the column direction are indicated by A11, A21, A12, and A22. Submatrix Amn (m = 1, 2, n = 1, 2) is a submatrix containing the components of the m-th row range divided in the row direction and the n-th column range divided in the column direction in the matrix A. be a matrix element. Partial vectors obtained by dividing the vector vbj into two in the row direction are denoted by vb1j and vb2j. vbmj (m=1, 2) is the component of the m-th row range obtained by dividing the vector vbj in the row direction as the component of the partial vector. Partial vectors obtained by dividing the vector vcj into two in the row direction are denoted by vc1j and vc2j. For vcmj (m=1, 2), the components of the m-th row range divided in the row direction in the vector vcj are the components of the partial vector.

In this figure, matrix multiplication is shown as an example of matrix operation, and matrix-vector multiplication is described as being included in part of matrix multiplication. For matrix-vector products that are not included in the matrix-product operation, it is similar to the matrix-vector product vc1=A×vb1 for the first columns of matrix B and matrix C, and so on. In this embodiment, the matrices A, B, and C have power-of-2 elements in the row and column directions, and the matrix A is divided into power-of-2 elements in the row and column directions. An example is given for the case. Alternatively, matrices A, B, and C may have a number of elements other than a power of 2 in at least one of the row-wise or column-wise directions, and matrix A may have at least Each may be divided into numbers other than powers of 2 (eg, 3×3, 5×5, 9×9, 3×5, 5×9, etc.).

FIG. 2 shows an example of a calculation formula in which the matrix operation according to this embodiment is decomposed into a submatrix and a matrix-vector product of the subvectors. The matrix-vector product vcj=A*vbj of matrix A and vector vbj is vc1j=(A11 A12)*vbj=A11*vb1j+A12*vb2j to compute subvector vc1j and vc2j=(A21 A22) to compute subvector vc2j. *vb _j =A21*vb1j+A22*vb2j. That is, when j=1, vc11=A11*vb11+A12*vb21 and vc21=A21*vb11+A22*vb21. When j=2, vc12=A11*vb12+A12*vb22 and vc22=A21*vb12+A22*vb22. The same applies to j=3, . . . , 8 below.

In this way, when the matrix A is divided into d pieces in the row direction and the column direction, and the vector vb is divided into d pieces, the matrix-vector product of the matrix A and the vector vb is the matrix-vector product of the submatrix and the subvector. d×d pieces are included. If the arithmetic unit has only a register capable of storing a single submatrix, the arithmetic unit performs the matrix operations shown in FIG. end up

FIG. 3 shows the configuration of an arithmetic device 300 according to this embodiment. Arithmetic device 300 can execute a matrix-vector product of a matrix up to the number of rows and columns specified in the specification and a vector up to the number of rows specified in the specification as one unit of operation by pipeline operation. Arithmetic unit 300 divides a matrix-vector product of a matrix and a vector larger than the size that can be processed in one unit of computation into a plurality of sets of submatrices and matrix-vector products of the partial vectors that can be processed in one unit of computation. calculate.

Arithmetic device 300 includes vector storage unit 310 , matrix storage unit 320 , pipeline operation unit 330 , result storage unit 340 , operation control unit 350 , main memory 360 , and memory control unit 370 . Vector storage unit 310 stores at least the first partial vector among the first plurality of partial vectors obtained by dividing the first vector. In this embodiment, the vector storage unit 310 is a register as an example. Alternatively, the vector storage unit 310 may be another storage device such as a cache memory that can pipeline partial vectors to the pipeline operation unit 330 .

Here, the first vector is a target vector for which at least one submatrix is multiplied by one matrix stored in the matrix storage unit 320 . The first vector is larger than the size that can be processed by the arithmetic unit 300 in one unit of arithmetic. The first plurality of partial vectors are obtained by dividing the first vector into sizes that can be processed in one unit of operation. In the matrix operation of FIG. 1, the first vector corresponds to one of the vectors vbj (eg vb1). The first plurality of partial vectors correspond to partial vectors vbij (i=1, 2) obtained by dividing the first vector vbj. If the first vector is larger, the first vector may be split into three or more partial vectors.

Moreover, the vector storage unit 310 may have a sufficient storage area to further store the second partial vector and other partial vectors among the first plurality of partial vectors. For example, in the matrix operation of FIG. 1, vector storage section 310 may store partial vector vb1j and partial vector vb2j.

Matrix storage unit 320 stores at least the first submatrix by which the first vector is to be multiplied, among a plurality of first submatrices obtained by dividing the first matrix by which the first vector is to be multiplied in the row direction and the column direction. In this embodiment, the matrix storage unit 320 is a register as an example. Alternatively, the matrix storage unit 320 may be another storage device such as a cache memory that can pipeline submatrices to the pipeline operation unit 330 .

Here, the first matrix is a target matrix by which at least one partial vector is multiplied by the first vector stored in the vector storage unit 310 . The first matrix is larger than the size that can be processed by the arithmetic unit 300 in one unit of arithmetic. The first plurality of submatrices are obtained by dividing the first matrix into sizes that can be processed by the arithmetic unit 300 in one unit of arithmetic. The first matrix corresponds to the matrix A in the matrix operation of FIG. The first plurality of submatrices correspond to submatrices Aij (i=1, 2, j=1, 2) obtained by dividing the first matrix A in the row direction and the column direction. If the first matrix is even larger, the first matrix may be divided into three or more in each of the row direction and column direction.

Further, the matrix storage unit 320 may have a sufficient storage area to further store the second partial matrix by which the second partial vector is to be multiplied and other partial matrices among the first plurality of partial matrices. good. For example, in the matrix operation of FIG. 1, the matrix storage unit 320 may store a submatrix A11 by which the subvector vb1j should be multiplied and a submatrix A12 by which the subvector vb2j should be multiplied. Here, the first partial vector and the second partial vector are located in different row ranges in the first vector. Therefore, the first submatrix and the second submatrix are located in different column ranges in the target matrix. Note that the first submatrix and the second submatrix may be located in the same row range in the target matrix.

Pipeline operation unit 330 is connected to vector storage unit 310 and matrix storage unit 320, receives from vector storage unit 310 the partial vector to be operated on stored in vector storage unit 310, and performs the operation stored in matrix storage unit 320. A submatrix of interest is received from the matrix storage unit 320 . The pipeline operation unit 330 can perform an operation of adding an intermediate vector to the matrix-vector product of the submatrix and the partial vector to be operated by the pipeline operation. In this embodiment, the pipeline operation unit 330 calculates a matrix-vector product of a 4-row, 4-column submatrix and a 4-row partial vector, and adds a 4-row intermediate vector to the matrix-vector product to obtain the operation result. An operation for calculating a partial vector (also referred to as a "result vector") can be executed as a unit operation.

Here, "executable as one unit of operation" means that the pipeline operation unit 330, in response to an instruction from the outside or a request such as the execution of an instruction, converts the matrix-vector product of the partial matrix and the partial vector to be operated into It means to collectively execute the operation of adding intermediate vectors and output the result. The pipeline operation unit 330 may have a large number of arithmetic units so that all basic operations included in this operation (for example, multiplication and addition of values) are performed by separate arithmetic units. may be performed by the same calculator.

Further, when the pipeline operation unit 330 performs the pipeline operation, it means that the pipeline operation unit 330 outputs the result through processing in a plurality of stages after starting the operation, and each stage can operate in parallel. do. That is, pipeline operation unit 330 can sequentially start other operations in each cycle from the start of a certain operation to the output of the result if there is no particular obstacle in execution.

For example, the pipeline operation unit 330 inputs a submatrix and a partial vector in the first cycle, multiplies corresponding elements of the submatrix and the partial vector in the second cycle, and converts the result vector in the third cycle. The products calculated in the second cycle for each element to be included may be summed, and in the fourth cycle, a partial vector of the operation result may be output. The pipeline operation unit 330 can have a pipeline structure with any number of stages as required.

Result storage unit 340 is connected to pipeline operation unit 330 . The result storage unit 340 receives and stores the result vector output by the pipeline operation unit 330 . The resulting vectors are, for example, vc11 and vc21 in FIG. In this embodiment, the result storage unit 340 is a register as an example. Alternatively, the result storage unit 340 may be another storage device such as a cache memory that can store partial vectors from the pipeline operation unit 330 in a pipeline manner. Vector storage unit 310, matrix storage unit 320, and result storage unit 340 may be implemented as the same storage device.

Operation control unit 350 is connected to vector storage unit 310 , matrix storage unit 320 , pipeline operation unit 330 , and result storage unit 340 . The arithmetic control unit 350 receives an instruction from outside the arithmetic device 300, or decodes a matrix arithmetic instruction during program execution in the arithmetic device 300. In response to the execution request of the matrix arithmetic, the request It controls the vector storage unit 310, the matrix storage unit 320, the pipeline operation unit 330, and the result storage unit 340 in order to execute the calculated matrix operation.

The main memory 360 stores matrices to be subjected to matrix calculations and calculation results. Memory control unit 370 is connected between vector storage unit 310 , matrix storage unit 320 , result storage unit 340 and main memory 360 . Memory control unit 370 operates between vector storage unit 310, matrix storage unit 320, and result storage unit 340 and main memory 360 in response to an external instruction or a memory access instruction during program execution in arithmetic unit 300. data transfer.

For example, in response to a vector load request from the main memory 360 to the vector storage unit 310, the memory control unit 370 loads the partial vector specified by the vector load out of the partial vectors stored in the main memory 360 into the main memory. It reads out from the memory 360 and stores it in the vector storage unit 310 . In addition, in response to a request to load the matrix from the main memory 360 to the matrix storage unit 320, the memory control unit 370 loads the submatrix specified by the matrix load out of the submatrices stored in the main memory 360 into the main memory. It reads out from the memory 360 and stores it in the matrix storage unit 320 . In addition, in response to a request from the result storage unit 340 to store the matrix or vector in the main memory 360, the memory control unit 370 reads out the matrix or vector of the operation result stored in the result storage unit 340, Store in main memory 360 . Depending on the design of the arithmetic unit 300, the main memory 360 may not be provided in addition to the vector storage unit 310 and the matrix storage unit 320, and a relatively large memory functioning as the vector storage unit 310 and the matrix storage unit 320 may be provided. Partial vectors and partial matrices may be supplied to the pipeline operation unit 330 in a direct pipeline manner from the memory.

In the configuration described above, the pipeline operation unit 330 executes matrix-vector multiplication operations through pipeline processing. For example, when performing the calculation of vc11=A11×vb11+A12×vb21 shown in FIG. requires multiple cycles. Therefore, in the cycle following the cycle in which the first operation for calculating the matrix-vector product of the first partial matrix A11 and the first partial vector vb11 is started, the pipeline operation unit 330 performs Even if a second operation is introduced that adds the matrix-vector product of vector vb21 to the result of the first operation, the execution of the second operation fails (pipeline hazard) and the result of the first operation becomes available. It becomes necessary to make the processing of the second operation wait until .

Depending on the design of the pipeline, the waiting time for processing the second operation may be reduced to some extent by supplying the result of the first operation to the second operation without waiting for it to be written to the register (bypass or forwarding). can be done. However, as long as there is a dependency between the first operation and the second operation, it is difficult to completely eliminate the vacancies that arise in the pipeline of pipeline operation section 330 due to pipeline hazards.

Therefore, the operation control unit 350 causes the pipeline operation unit 330 to perform the pipeline operation of the matrix-vector product (for example, A11×vb11) of the first submatrix and the first partial vector. The pipeline operation unit 330 is instructed to perform another matrix-vector product operation using . Here, "another matrix-vector product" is an operation that does not use the result of the operation of the matrix-vector product of the first submatrix and the first partial vector, and includes an operation that includes a matrix-vector product, i.e., for example, the matrix-vector product has the first part The calculation may be such that a calculation result other than the matrix-vector product of the matrix and the first partial vector is added. As a result, the operation control unit 350 executes one or more operations that do not depend on the operation result of the first operation after the pipeline operation unit 330 waits for the operation result of the first operation and before it starts executing the second operation. Other matrix-vector products can be injected into the pipeline operation unit 330, thereby increasing the utilization efficiency of the pipeline operation unit 330. FIG.

Further, after the cycle in which the calculation result of the matrix-vector product of the first partial matrix and the first partial vector becomes available without delay, the operation control unit 350 calculates the matrix-vector product of the second partial matrix and the second partial vector as follows: The pipeline operation unit 330 may be instructed to perform an operation to be added to the operation result of the matrix-vector product of the first partial matrix and the first partial vector. As a result, the operation control unit 350 can prevent pipeline hazards from occurring in the second operation, and can insert another matrix-vector product operation between the first operation and the second operation. .

FIG. 4 shows a first example of pipeline processing by the arithmetic device 300 according to this embodiment. In the calculation shown as cycle 0, the calculation control unit 350 instructs the vector storage unit 310 to read vb11, which is an example of the first partial vector, and reads A11, which is an example of the first partial matrix, to the matrix storage unit 320. , the matrix-vector product of the first partial matrix A11 and the first partial vector vb11 is calculated, and the intermediate vector in the middle of the calculation is stored in the intermediate register (temporary register) vctmp1 of the pipeline operation unit 330. to the pipeline operation unit 330 .

By the start of execution of cycle 1, the vector storage unit 310 stores vb11, which is an example of the first partial vector, and vb21, which is an example of the second partial vector, as well as a second vector to be multiplied by the first matrix A (as an example, A third partial vector (vb12 as an example) to be multiplied by the first submatrix A11 is further stored among the second plurality of partial vectors vbi2 obtained by dividing vb2). In this example, the first vector and the second vector are column vectors contained in the second matrix B to be multiplied by the first matrix A, for example, the first vector is vb1 and the second vector is vb2. The third partial vector is vb12 to be multiplied by the first partial matrix A11 among the second plurality of partial vectors vbi2 obtained by dividing the second vector vb2. Alternatively, the first vector and the second vector may each be separate vectors to be multiplied by the matrix A.

In the operation shown as cycle 1, during the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector, the operation control unit 350 instructs the vector storage unit 310 to read the third partial vector vb12, In addition to instructing the matrix storage unit 320 to read out the first submatrix A11, as another matrix-vector product operation that does not cause a pipeline hazard, execute the matrix-vector product operation of the first submatrix and the third partial vector. The pipeline calculation unit 330 is instructed. In response to this, pipeline operation section 330 executes an operation of storing the matrix-vector product of the first partial matrix and the third partial vector in intermediate register vctmp2 of pipeline operation section 330 as an intermediate vector during calculation. . This operation corresponds to the first matrix-vector product operation in the third row of FIG. 2, and the matrix-vector products of cycles 0 and 1 are reflected in different result vectors vc11 and vc12. Therefore, since there is no dependency between these operations, the pipeline operation unit 330 can execute these operations without causing pipeline hazards.

In the calculation shown as cycle 2, the calculation control unit 350 instructs the vector storage unit 310 to read vb21, which is an example of the second partial vector, and reads A12, which is an example of the second partial matrix, to the matrix storage unit 320. , and instructs the pipeline operation unit 330 to perform the operation of calculating the matrix-vector product of the second partial matrix A12 and the second partial vector vb21 and adding the operation result vctmp1 of the operation in cycle 0, and the operation result The result storage unit 350 is instructed to store the obtained partial vector vc11. Here, since the calculation of cycle 2 depends on the calculation of cycle 0, the calculation control unit 350 inserts the calculation of cycle 1, which does not depend on the calculation of cycle 0, between the calculations of cycle 0 and cycle 2. The utilization efficiency of the pipeline of the pipeline operation unit 330 can be increased.

By the start of execution of cycle 3, the vector storage unit 310 may further store a fourth partial vector to be multiplied by the second partial matrix A12 among the second plurality of partial vectors. In the calculation shown as cycle 3, the calculation control unit 350 instructs the vector storage unit 310 to read vb22, which is an example of the fourth partial vector, and reads A12, which is an example of the second partial vector, to the matrix storage unit 320. , and instructs the pipeline operation unit 330 to perform the operation of calculating the matrix-vector product of the second partial matrix A12 and the fourth partial vector vb22 and adding the operation result vctmp2 of the operation in cycle 1, and the operation result The main memory 360 is instructed to store the resulting partial vector vc12. Here, since the calculation of cycle 3 depends on the calculation of cycle 1, the calculation control unit 350 inserts the calculation of cycle 2, which does not depend on the calculation of cycle 1, between the calculations of cycle 1 and cycle 3. The utilization efficiency of the pipeline of the pipeline operation unit 330 can be increased.

In the example of this figure, the partial vectors (vc11, vc12) of the first row range (1st to 4th rows) in the multiple column vectors (vc1, vc2) of the matrix C are calculated in cycles 0 to 3. 7, the subvectors (vc21, vc22) of the second row range (5th to 8th rows) in the column vectors (vc1, vc2) of the matrix C are calculated. The calculations in cycles 4 to 7 are the same except that submatrices A21 and A22 are used in place of submatrices A11 and A12, and subvectors vc21 and vc22 are used in place of subvectors vc11 and vc12.

In this example, the operation control unit 350 uses the first submatrix between the first operation of the matrix-vector product of the first submatrix and the first partial vector and the second operation using the result of the operation. Insert another matrix-vector product operation, in this example the matrix-vector product of the first sub-matrix and the third sub-vector. As a result, the arithmetic control unit 350 can utilize one empty cycle required between the first arithmetic operation and the second arithmetic operation.

When a plurality of empty cycles occur between the first computation and the second computation, computation control section 350 calculates the matrix-vector product of each of the first submatrix and the plurality of third subvectors in the first computation and the second computation. can be inserted in between. For example, the vector storage unit 310 further stores a plurality of second vectors vb2, vb3, . Operation control section 350 controls each cycle from the start of the pipeline operation of the matrix-vector product of first submatrix A11 and first subvector vb11 to before the operation result becomes available without delay as the first part Matrix-vector products A11×vb12, A11×vb13, . . . of matrix A11 and third partial vectors vb12, vb13, . The first vector and the plurality of second vectors may be arranged in the column order of the second matrix or in the reverse order of the columns. can be

FIG. 5 shows a second example of pipeline processing by the arithmetic device 300 according to this embodiment. When pipeline operation unit 330 has more intermediate registers, or when operation results become available after being temporarily stored in main memory 360, operation control unit 350 performs The operations may be controlled to occur before the operations of cycles 2-3. In this case, the operation control unit 350 performs another matrix vector Cycle 1 in FIG. 5, which is a product operation, and Cycle 2 in FIG. 5, which is another matrix-vector product operation using the first partial vector, are executed by pipeline operation section 330 . Further, the calculation control unit 350 performs the calculation of the cycle 3, which is the calculation of the matrix-vector product of the second submatrix A21 used in the calculation of the cycle 2 and the second partial vector vb12 used in the calculation of the cycle 1, to the It may be executed between the first operation and the second operation. This allows the arithmetic control unit 350 to further fill the empty cycles between the first arithmetic operation and the second arithmetic operation. Note that the execution order of the operations in cycles 0-3 may be arbitrary, and the execution order of the operations in cycles 4-7 may be determined according to the execution order of the corresponding operations in cycles 0-3.

FIG. 6 shows a third example of pipeline processing by the arithmetic device 300 according to this embodiment. In the calculation shown as cycle 0, the calculation control section 350 performs the same control as in cycle 0 of FIG.

By the start of execution of cycle 1, the matrix storage unit 320 has divided the first matrix A in the row direction and the column direction in addition to A11 as an example of the first submatrix and A12 as an example of the second submatrix. Among the first plurality of submatrices Aij, a third submatrix (A21 as an example) to be multiplied by the first subvector vb11 is further stored. Alternatively, the third submatrix may be a submatrix included in a matrix other than the first matrix A.

In the operation shown as cycle 1, during the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector, the operation control unit 350 instructs the vector storage unit 310 to read the first partial vector vb11, In addition to instructing the matrix storage unit 320 to read out the third partial matrix A21, as another matrix-vector product operation that does not cause pipeline hazards, the matrix-vector product operation of the third partial matrix A21 and the first partial vector vb11 is performed. It instructs the pipeline operation unit 330 to execute. In response to this, pipeline operation unit 330 performs an operation to store the matrix-vector product of third submatrix A21 and first subvector vb11 in intermediate register vctmp2 of pipeline operation unit 330 as an intermediate vector during calculation. Execute. This operation corresponds to the first matrix-vector product operation in the second row of FIG. 2, and the matrix-vector products of cycles 0 and 1 are reflected in different result vectors vc11 and vc21. Therefore, since there is no dependency between these operations, the pipeline operation unit 330 can execute these operations without causing pipeline hazards.

In the calculation shown as cycle 2, the calculation control section 350 performs the same control as in cycle 2 of FIG. Here, since the calculation of cycle 2 depends on the calculation of cycle 0, the calculation control unit 350 inserts the calculation of cycle 1, which does not depend on the calculation of cycle 0, between the calculations of cycle 0 and cycle 2. The utilization efficiency of the pipeline of the pipeline operation unit 330 can be increased.

By the start of execution of cycle 3, the vector storage unit 310 may further store a fourth submatrix by which the second subvector vb21 is to be multiplied, among the first submatrices Aij. In the calculation shown as cycle 3, the calculation control unit 350 instructs the vector storage unit 310 to read vb21, which is an example of the second partial vector, and reads A22, which is an example of the fourth partial vector, to the matrix storage unit 320. , and instructs the pipeline operation unit 330 to perform the operation of calculating the matrix-vector product of the fourth partial matrix A22 and the second partial vector vb21 and adding the operation result vctmp2 of the operation in cycle 1, and the operation result The main memory 360 is instructed to store the resulting partial vector vc21. Here, since the calculation of cycle 3 depends on the calculation of cycle 1, the calculation control unit 350 inserts the calculation of cycle 2, which does not depend on the calculation of cycle 1, between the calculations of cycle 1 and cycle 3. The utilization efficiency of the pipeline of the pipeline operation unit 330 can be increased.

In the example of this figure, two partial vectors vc11, vc21 contained in one column vector vc1 of matrix C are calculated in cycles 0-3, and two partial vectors vc11, vc21 contained in another column vector vc2 of matrix C are calculated in cycles 4-7. Compute two partial vectors vc12, vc22. Calculations in cycles 4 to 7 are the same except that partial vectors vb12 and vb22 are used instead of partial vectors vb11 and vb21, and partial vectors vc12 and vc22 are used instead of partial vectors vc11 and vc21.

In this example, the calculation control unit 350 uses the first partial vector between the first calculation of the matrix-vector product of the first partial matrix and the first partial vector and the second calculation using the calculation result. Insert another matrix-vector product operation, in this example the matrix-vector product of the third sub-matrix and the first sub-vector. As a result, the arithmetic control unit 350 can utilize one empty cycle required between the first arithmetic operation and the second arithmetic operation.

When a plurality of idle cycles occur between the first and second computations, the computation control unit 350 calculates the matrix-vector product of each of the plurality of third partial matrices and the first partial vectors for the first and second computations. can be inserted in between. For example, the matrix storage unit 320 stores a plurality of third submatrices A21, A31, . The operation control unit 350 controls each cycle from the start of the pipeline operation of the matrix-vector product of the first submatrix A11 and the first subvector vb11 to the time the operation result becomes available without delay as a plurality of second Fill with a matrix-vector product operation of each of the three sub-matrices A21, A31, . . . and the first sub-vector vb11. In addition, the first submatrix and the plurality of third submatrices may be arranged in the same row range of the first matrix in the column order or in reverse order of the column order, and are not arranged in the column order of the second matrix, Each can be a submatrix of any column range.

FIG. 7 shows a fourth example of pipeline processing by the arithmetic device 300 according to this embodiment. When pipeline operation unit 330 has more intermediate registers, or when operation results become available after being temporarily stored in main memory 360, operation control unit 350 performs The operations may be controlled to occur before the operations of cycles 2-3. In this case, the operation control unit 350, during the execution of the pipeline operation of the first operation for calculating the matrix-vector product of the first partial matrix A11 and the first partial vector vb11, calculates another matrix vector using the first partial vector Cycle 1 operation in FIG. 7, which is the product operation, and cycle 2 operation in FIG. Further, the calculation control unit 350 performs the calculation of the cycle 3, which is the calculation of the matrix-vector product of the third partial matrix A21 used for the calculation of the cycle 1 and the second partial vector vb12 used for the calculation of the cycle 2, to the It may be executed between the first operation and the second operation. This allows the arithmetic control unit 350 to further fill the empty cycles between the first arithmetic operation and the second arithmetic operation. Note that the execution order of the operations in cycles 0-3 may be arbitrary, and the execution order of the operations in cycles 4-7 may be determined according to the execution order of the corresponding operations in cycles 0-3. Here, the pipeline processing of FIG. 7 is substantially the same as the pipeline processing of FIG. 5 with the operations of cycles 1 and 2 interchanged and the operations of cycles 5 and 6 interchanged.

In any pipeline processing including the first to fourth examples described above, the operation control unit 350 allows the pipeline operation unit 330 to obtain partial vectors and partial matrices that the pipeline operation unit 330 uses. The memory control unit 370 may be instructed to transfer the data from the main memory 360 to the vector storage unit 310 and the matrix storage unit 320 earlier. For example, in the example of FIG. 4, main memory 360 transfers sub-vectors vb11, vb12, vb21, vb22 to vector storage 310 and sub-matrices A11 and A12 to matrix storage 320 before cycle 0. may be transferred. Alternatively, main memory 360 transfers partial vectors vb11 and vb12 to vector storage 310 before cycle 0, transfers submatrix A11 to matrix storage 320, and prior to cycle 2: Partial vectors vb21 and vb22 may be transferred to vector storage section 310 and submatrix A12 may be transferred to matrix storage section 320 .

In the case of the pipeline processing shown in the first and second examples, the pipeline operation unit 330 uses different partial vectors vb11, vb12, vb21, vb22 for each cycle. is used every two cycles. Therefore, the matrix storage unit 320 only needs to have a throughput capable of outputting one submatrix every two cycles, and the power consumption and circuit scale of the matrix storage unit 320 can be reduced.

In the case of the pipeline processing shown in the third and fourth examples, the pipeline operation unit 330 uses different submatrices A11, A12, A21, and A22 for each cycle. is used every two cycles. Therefore, the matrix storage unit 320 only needs to have a throughput capable of outputting one partial vector every two cycles, and the power consumption and circuit scale of the vector storage unit 310 can be reduced.

A designer of the arithmetic device 300 or a user of the arithmetic device 300 may decide the execution order of the pipeline processing so that the circuit scale of the arithmetic device 300 can be made smaller or the power consumption of the arithmetic device 300 can be made smaller. can be selected.

Various embodiments of the invention may be described with reference to flowchart illustrations and block diagrams, where blocks refer to (1) steps in a process in which operations are performed or (2) devices responsible for performing the operations. may represent a section of Certain steps and sections are implemented either by dedicated circuitry, by programmable circuitry provided with computer readable instructions stored on a computer readable medium, or by processors provided with computer readable instructions stored on a computer readable medium. may be Dedicated circuitry may include both digital and analog hardware circuitry, and may include both integrated circuits (ICs) and discrete circuitry. Programmable circuits include logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc. and the like.

Computer-readable media may include any tangible device capable of storing instructions to be executed by a suitable device, such that computer-readable media having instructions stored thereon may be designated in flowcharts or block diagrams. It will comprise an article of manufacture containing instructions that can be executed to create means for performing the operations described above. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), Blu-ray (RTM) Disc, Memory Stick, Integration Circuit cards and the like may be included.

The computer readable instructions may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or object oriented programming such as Smalltalk, JAVA, C++, etc. language, and any combination of one or more programming languages, including conventional procedural programming languages, such as the "C" programming language or similar programming languages. good.

Computer readable instructions may be transferred to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing apparatus, either locally or over a wide area network (WAN), such as a local area network (LAN), the Internet, or the like. ) and may be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

FIG. 8 illustrates an example computer 2200 in which aspects of the invention may be implemented in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as one or more sections of or operations associated with an apparatus according to embodiments of the present invention. One or more sections may be executed and the computer 2200 may be capable of executing a process or steps of such processes according to embodiments of the present invention. Such programs may be executed by CPU 2212 to cause computer 2200 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

Computer 2200 according to this embodiment includes CPU 2212 , RAM 2214 , graphics controller 2216 , and display device 2218 , which are interconnected by host controller 2210 . Computer 2200 also includes input/output units such as communication interface 2222 , hard disk drive 2224 , DVD-ROM drive 2226 , and IC card drive, which are connected to host controller 2210 via input/output controller 2220 . The computer also includes legacy input/output units such as ROM 2230 and keyboard 2242 , which are connected to input/output controller 2220 through input/output chip 2240 .

CPU 2212 operates according to programs stored in ROM 2230 and RAM 2214, thereby controlling each unit. Graphics controller 2216 retrieves image data generated by CPU 2212 into itself, such as a frame buffer provided in RAM 2214 , and causes the image data to be displayed on display device 2218 .

Communication interface 2222 communicates with other electronic devices over a network. Hard disk drive 2224 stores programs and data used by CPU 2212 within computer 2200 . DVD-ROM drive 2226 reads programs or data from DVD-ROM 2201 and provides programs or data to hard disk drive 2224 via RAM 2214 . The IC card drive reads programs and data from IC cards and writes programs and data to IC cards.

ROM 2230 stores therein any programs that are dependent on the hardware of computer 2200, such as a boot program that is executed by computer 2200 upon activation. Input/output chip 2240 may also connect various input/output units to input/output controller 2220 via parallel ports, serial ports, keyboard ports, mouse ports, and the like.

A program is provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The program is read from a computer-readable medium, installed in hard disk drive 2224 , RAM 2214 , or ROM 2230 , which are also examples of computer-readable medium, and executed by CPU 2212 . The information processing described within these programs is read by computer 2200 to provide coordination between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing the manipulation or processing of information in accordance with the use of computer 2200 .

For example, when communication is performed between the computer 2200 and an external device, the CPU 2212 executes a communication program loaded into the RAM 2214 and sends communication processing to the communication interface 2222 based on the processing described in the communication program. you can command. The communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in a recording medium such as the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or an IC card under the control of the CPU 2212, and transmits the read transmission data. Data is transmitted to the network, or received data received from the network is written to a receive buffer processing area or the like provided on the recording medium.

In addition, the CPU 2212 causes the RAM 2214 to read all or necessary portions of files or databases stored in external recording media such as a hard disk drive 2224, a DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. Various types of processing may be performed on the data in RAM 2214 . CPU 2212 then writes back the processed data to the external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on recording media and subjected to information processing. CPU 2212 performs various types of operations on data read from RAM 2214, information processing, conditional decision making, conditional branching, unconditional branching, and information retrieval, as specified throughout this disclosure and by instruction sequences of programs. Various types of processing may be performed, including any of , permutation, etc., and the results written back to RAM 2214 . In addition, the CPU 2212 may search for information in a file in a recording medium, a database, or the like. For example, if a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 2212 determines that the attribute value of the first attribute is specified. search the plurality of entries for an entry that matches the condition, read the attribute value of the second attribute stored in the entry, and thereby associate it with the first attribute that satisfies the predetermined condition. an attribute value of the second attribute obtained.

The programs or software modules described above may be stored in a computer readable medium on or near computer 2200 . Also, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the program to the computer 2200 via the network. do.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the scope of claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The execution order of each process such as actions, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly "before", "before etc., and it should be noted that they can be implemented in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if the description is made using "first," "next," etc. for the sake of convenience, it means that it is essential to carry out in this order. not a thing

300 arithmetic unit 310 vector storage unit 320 matrix storage unit 330 pipeline operation unit 340 result storage unit 350 operation control unit 360 main memory 370 memory control unit 2200 computer 2201 DVD-ROM
2210 host controller 2212 CPU
2214 RAM
2216 graphic controller 2218 display device 2220 input/output controller 2222 communication interface 2224 hard disk drive 2226 DVD-ROM drive 2230 ROM
2240 input/output chip 2242 keyboard

Claims (9)

a vector storage unit that stores at least a first partial vector among a plurality of first partial vectors obtained by dividing the first vector;
a matrix storage unit that stores at least a first submatrix by which the first partial vector is to be multiplied, among a plurality of first submatrices obtained by dividing the first matrix by which the first vector is to be multiplied in the row direction and the column direction;
a pipeline operation unit capable of executing an operation of adding an intermediate vector to the matrix-vector product of the partial matrix stored in the matrix storage unit and the partial vector stored in the vector storage unit by pipeline operation;
The pipeline operation unit performs another matrix-vector product operation using the first partial vector or the first partial matrix during the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector. and an arithmetic control unit that instructs the pipeline arithmetic unit to execute the above. the vector storage unit further stores a second partial vector among the first plurality of partial vectors;
the matrix storage unit further stores a second submatrix to be multiplied by the second subvector among the first plurality of submatrices;
After a cycle in which the calculation result of the matrix-vector product of the first submatrix and the first partial vector becomes available without delay, the operation control unit controls the matrix-vector product of the second partial matrix and the second partial vector to an operation result of the matrix-vector product of the first submatrix and the first partial vector. the vector storage unit further stores a third partial vector by which the first submatrix is to be multiplied, among a plurality of second partial vectors obtained by dividing a second vector by which the first matrix is to be multiplied;
The operation control unit, during the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector, performs the calculation of the other matrix-vector product of the first partial matrix and the third partial vector. 3. The arithmetic unit according to claim 1, wherein the pipeline operation unit is instructed to execute a matrix-vector multiplication operation. 4. The arithmetic unit according to claim 3, wherein said first vector and said second vector are column vectors included in a second matrix by which said first matrix is to be multiplied. the vector storage unit stores a plurality of the second vectors included in the second matrix;
The operation control unit controls each cycle from the start of the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector until the operation result becomes available without delay to the first 5. The arithmetic unit of claim 4, wherein the arithmetic unit fills with a matrix-vector product operation of the third partial vector from each of the submatrices and the plurality of second vectors. the matrix storage unit further stores a third submatrix by which the first subvector is to be multiplied, among the first plurality of submatrices;
The operation control unit performs the operation of the third partial matrix and the first partial vector as the operation of the other matrix-vector product during the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector. 3. The arithmetic unit according to claim 1, wherein the pipeline operation unit is instructed to execute a matrix-vector multiplication operation. The matrix storage unit stores a plurality of the third submatrices,
The operation control unit controls each cycle from the start of the pipeline operation of the matrix-vector product of the first partial matrix and the first partial vector until the operation result becomes available without delay to the plurality of 7. The arithmetic unit of claim 6, wherein padding with a matrix-vector product operation of each of the third sub-matrix and the first sub-vector. the vector storage unit stores at least the first partial vector among the first plurality of partial vectors obtained by dividing the first vector;
a matrix storage unit that stores at least a first submatrix by which the first partial vector is to be multiplied, among a plurality of first submatrices obtained by dividing the first matrix by which the first vector is to be multiplied in the row direction and the column direction;
a pipeline operation unit capable of executing an operation of adding an intermediate vector to a matrix-vector product of a submatrix stored in the matrix storage unit and a partial vector stored in the vector storage unit by pipeline operation; during a pipeline operation of a matrix-vector product of a sub-matrix and said first sub-vector, beginning execution of another matrix-vector product operation using said first sub-vector or said first sub-matrix. An arithmetic program executed by an arithmetic device,
The computing device is
a vector storage unit that stores at least a first partial vector among a plurality of first partial vectors obtained by dividing the first vector;
a matrix storage unit that stores at least a first submatrix by which the first partial vector is to be multiplied, among a plurality of first submatrices obtained by dividing the first matrix by which the first vector is to be multiplied in the row direction and the column direction;
a pipeline operation unit capable of executing an operation of adding an intermediate vector to the matrix-vector product of the partial matrix stored in the matrix storage unit and the partial vector stored in the vector storage unit by pipeline operation,
The computing program causes the computing device to generate another matrix vector using the first partial vector or the first partial matrix during the pipeline operation of the matrix-vector product of the first partial matrix and the first partial matrix. A calculation program that initiates the execution of the product calculation.

Images