JP3542184B2 - Linear calculation method - Google Patents

Description

[0001]
[Industrial applications]
The present invention relates to a method for performing high-speed linear calculations, such as solving simultaneous linear equations and calculating eigenvalues, using a parallel computer.
[0002]
[Prior art]
In scientific and technical calculations such as structural analysis and fluid calculation, linear calculations such as solution of simultaneous linear equations using large-scale matrices ranging from tens of thousands to millions of dimensions as coefficient matrices and eigenvalue calculations are required. As a means for performing such calculations at high speed, a parallel computer is effective. A parallel computer is a system in which a large number of high-speed processors, from tens to tens of thousands, are connected via a network. Problems are divided appropriately, assigned to each processor, and processed at the same time. The execution speed can be dramatically improved compared to execution.
[0003]
In dividing the problem and assigning partial problems to the processors, (1) the performance of a single processor must be exhibited, and (2) parallelization efficiency (ideal performance improvement when the number of processors is increased from one to p). It is necessary to consider two factors: (a ratio of how much performance improvement was actually achieved with respect to p times). Regarding the single performance of (1), a processor used in a high-performance parallel computer is often a RISC-type processor having a cache, so that a partitioning method that can effectively use the cache is required. Regarding the parallelization efficiency of (2), the number of inter-processor communications and the amount of data are reduced to reduce communication time, and the load between processors is equalized to reduce the waiting time of the processor due to uneven load. It is necessary to do.
[0004]
In the case of the Gaussian elimination method (LU decomposition method) used to solve a system of linear equations, conventionally, to solve these two problems, a parallelized block Gaussian method (for example, Intel Corporation: "Large-scale simultaneous processing with a parallel processing computer") A method called "Method of Solving a Linear Equation" (see Japanese Patent Application Laid-Open No. 5-250401 (filed on Dec. 20, 1991)) has been used. This is done by dividing the coefficient matrix 4 of FIG. 2 into blocks 5, 6, 7 and the like having a size of LB × LB (LB: block size), treating each block as a real number, and applying a normal Gaussian elimination method. Method. In parallelization, each block is allocated to 16 processors 0 to 15 as shown in FIG. 2, and a processor in charge performs an erasure operation on each block.
[0005]
The advantages of the parallelized block Gaussian method compared to the normal Gaussian elimination method are as follows. First, multiplication between real numbers in the erasure operation is multiplication between LB × LB matrices in the parallelized block Gaussian method. Generally, in the matrix multiplication, since the operation is performed many times while the data is in the cache, the cache can be effectively used, and the single performance on the RISC processor can be improved. Furthermore, since data communication between processors is performed in block units, the number of times of communication is reduced as compared with the normal Gaussian elimination method, and the parallelization efficiency can be improved by reducing the communication time. Due to these advantages, the parallelized block Gaussian method is the most common method for solving simultaneous linear equations by Gaussian elimination on a parallel computer.
[0006]
[Problems to be solved by the invention]
In the parallelized block Gaussian method, the block size LB is determined from two different requirements. First, (1) from the standpoint of improving the performance of a single unit, in a matrix multiplication of the form C ← C−A × B that appears in the erasure operation, LB under the condition that all of the matrices A, B, and C enter the cache Is desirably as large as possible. This is because the larger the LB is, the more operations are performed while the data is in the cache, and the more effective use of the cache is achieved. On the other hand, (2) there is an optimum value of LB also from the viewpoint of improving parallelization efficiency. As the LB increases, the block serving as a unit of data communication increases, and the communication time decreases due to the decrease in the number of times of communication. On the other hand, the number of blocks handled by each processor increases, and the waiting time due to uneven load increases. . As the LB decreases, the number of data communications increases, but the load on each processor becomes equal. Therefore, the LB that maximizes the parallelization efficiency is determined by these two balances. Generally, (1) the value of LB that maximizes the single unit performance and (2) the value of LB that maximizes the parallelization efficiency do not match. As an example, FIG. 3 shows a case where the parallelized block Gaussian method is executed by 16 processors. In this example, as a result of determining the LB so as to maximize the single unit performance, the assigned block 5 of the processor 0 and the assigned block 7 of the processor 15 greatly differ from each other. Therefore, high parallelization efficiency cannot be expected. As described above, in the conventional parallelized block Gaussian method, it is necessary to determine the LB so as to achieve the highest overall performance while sacrificing one or both of the unit performance and the parallelization efficiency. There was a problem that it was not possible to bring out the performance that it had.
[0007]
An object of the present invention is to solve this problem by improving the parallelized block Gaussian method, and to fully exploit the performance of a parallel computer in linear computation by increasing both the unit performance and the parallelization efficiency. I do.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, the present invention improves the parallelized block Gaussian method and introduces two different block sizes. The reason that the conventional parallelized block Gaussian method cannot simultaneously increase the simplex performance and the parallelization efficiency is that the optimum block size differs for each of these two purposes. Therefore, in the present invention, first, an optimal block size LB is determined from the viewpoint of parallelization efficiency, a matrix is divided into blocks based on this size, and each block is assigned to a processor. This makes it possible to optimize the parallelization efficiency. Next, in the erasure operation, instead of erasing the entire matrix each time at each block stage, block pivot rows and block pivot columns for a certain number of stages k are created, and once in k stages, they are collectively collected. Perform erasure. As a result, the erasure operation can be processed as a matrix multiplication of the size of MB = k * LB. By appropriately setting k, the performance of a single device can be optimized.
[0009]
[Action]
Taking the case where the Gaussian elimination method is executed by 16 processors as an example, how the present invention achieves high parallelization efficiency and high unit performance simultaneously will be described.
[0010]
First, FIG. 4A shows the erasure operation of the entire matrix. The matrix is divided into blocks, and the size of each block is LB × LB. The shaded blocks in the matrix indicate the blocks in charge of the processor 0. By making the LB small enough, the number of blocks in charge of each processor becomes substantially equal, and the parallelization efficiency can be increased. FIG. 4A shows a case where block pivot columns and block pivot rows for four stages are created, and erase for four stages is collectively performed. In this case, the processor 0 performs the erasure operation using the shaded block of the block pivot row / column. FIG. 4B shows only the operation performed by the processor 0, which is displayed. As is clear from the figure, by erasing four blocks at a time, the erasing operation in the processor 0 can be processed as a multiplication between matrices of size MB = 4 * LB. As a result, the size of matrix multiplication can be increased as compared with the conventional parallelized block Gaussian method, and the unit performance can be improved by effective use of the cache.
[0011]
As described above, according to the method of the present invention, it is possible to simultaneously improve the parallelization efficiency and the single-unit performance.
[0012]
【Example】
(Example 1)
Hereinafter, the principle and embodiments of the present invention will be described in detail with reference to the drawings. Here, as an embodiment, a method of solving simultaneous linear equations by a Gaussian elimination method using a parallel computer is described. As shown in FIG. 1, a parallel computer system to which the present method is applied is an input device for inputting data such as matrix data and a right-hand side vector, and a system of linear equations having p processors 31 each having a cache memory. It comprises a solving device 2 and an output device 3 for outputting a solution. FIG. 5 shows the processing in this embodiment. First, a coefficient matrix A, a right side vector b, a matrix dimension N, two block sizes LB and MB are input from an input device (process 13), and matrix data and a right side vector are assigned to a processor (process 14), and K An erasure operation is performed for each block stage from = 1 to K = N / LB to obtain an LU decomposition of the coefficient matrix (process 15). Next, using the obtained LU factorization, a forward elimination operation (process 22) and a backward substitution operation (process 23) are performed to obtain a solution, and finally, solutions stored in a distributed manner among the processors are collected ( Process 24) and output (Process 25). The most significant features of the present invention are the data allocation part of the processing 14 and the erasure calculation part of the processing 15, and these parts will be described below.
[0013]
(1) Using the block size LB for processor allocation input in the data allocation input step to the processor (process 13), the coefficient matrix is divided into blocks of size LB × LB as shown in FIG. To the processor. 2 indicate blocks assigned to the processors 0, 1, and 15, respectively. When the LB is large, the unit of communication becomes large, so that the number of communication times is reduced and the communication overhead is reduced. However, the number of blocks in charge of each processor increases, and the idle time increases due to uneven load between the processors. On the other hand, if the LB is small, the load becomes equal, but the number of times of communication increases and the communication overhead increases. In the present invention, the LB is determined in advance so as to maximize the parallelization efficiency based on a balance between these two.
[0014]
(2) Erasing Operation The erasing operation according to the present invention is characterized in that the erasing operation for the entire matrix is not performed for each block stage, but k block pivot columns / rows are created in advance and then erasing for k block stages is performed. Thus, the erasure operation is processed as an operation of a large matrix having a size of MB = k * LB to effectively use the cache. Therefore, each process of the erasure calculation will be described with reference to processes 15 to 21 in FIG.
[0015]
First, at the beginning of the K-th block, LU decomposition of the (K, K) -th block is performed (process 16). This is the same as the conventional parallelized block Gaussian method. Next, a K-th block pivot column is created by applying the inverse matrix of U just created to the K-th block column from the right (process 17), and the inverse matrix of L is applied to the K-th block row from the left by applying the inverse matrix to the K-th block row. A K block pivot row is created (process 18). Then, an erase operation is performed using the K-th block pivot column and the K-th block pivot row (process 19). However, the erasure is not performed on the entire matrix, but is performed only on a portion to be the next k block pivot columns / rows. This range suffices to create k columns of block pivot columns and rows. Thereafter, if K is not a multiple of k = MB / LB, the process proceeds to the (K + 1) th block stage (process 20). If K is a multiple of k = MB / LB, the entire matrix is erased using the k block pivot columns and rows created so far (process 21). FIG. 4A shows the processing in the erase operation. In this example, k = 4, and the erasure calculation is performed using four block pivot columns 8 and four block pivot rows 10. FIG. 4A shows an operation on the entire coefficient matrix 4, and FIG. 4B shows an extracted result of the operation on the block 5 in charge (each size is LB × LB) of the processor 0. As is apparent from the figure, the erasure operation is performed using the part 9 used in the operation of the processor 0 in the block pivot row and the part 11 used in the operation of the processor 0 in the block pivot row, and the size MB = 4. * Can be processed as a multiplication of a matrix of LB. As a result, if k is appropriately selected and three MB × MB matrices are just entered into the cache, the cache can be used to the maximum, and the performance of a single unit can be improved. On the other hand, in the conventional parallelized block Gaussian method, the erasure operation of the entire matrix is performed for each block stage, so the erasure operation is a matrix multiplication of LB × LB, but the range of LB balances the optimization of the parallelization efficiency. And it is difficult to make full use of single performance. In the present invention, this point is greatly improved.
[0016]
(Example 2)
In the first embodiment, the case of the Gaussian elimination method, which is one of the linear calculations, has been described. However, in addition to the above, the linear calculation in which the matrix is deleted using the pivot columns and the pivot rows is used in the present embodiment. The invention is applicable. Examples of such linear calculations include QR decomposition of matrices used in the least squares method and the like, and Householder transform used in eigenvalue calculations. In this embodiment, an application example of the present invention to the least squares method will be described.
[0017]
As an example of the problem of the least squares method, m sets of input data and output data (x1, y1), (x2, y2),. . . , (Xm, ym), consider the problem of approximating the relationship between the input x and the output y with an (n−1) -order expression y = f (x). What we want to find is the 0th, 1st,. . . , N-1 order terms a1, a2,. . . , An. To find this, first, a matrix A having the (i, j) -th element as the (j−1) th element of the i-th input data xi is obtained by a standard procedure of the least squares method, and this is expressed as A = QR. After the QR decomposition, a simultaneous linear equation in which R is a coefficient matrix and (transposed matrix of Q) × (y1, y2,..., Ym) is a right-hand side vector may be solved. Of this series of processing, the largest calculation amount is the QR decomposition part of A, and the present invention can be used when this is executed on a parallel machine.
[0018]
FIG. 6 shows a process for executing this QR decomposition on a parallel machine using the method of the present invention. Also in the case of the QR decomposition, the matrix is divided into blocks according to the block size LB selected so as to maximize the parallelization efficiency, and the portion to be assigned to the processor (process 27) is the same as the Gaussian elimination method of the first embodiment. However, in the next erasure operation, the pre-processing operation for creating the K-th block pivot row / column is performed from the LU decomposition process (process 16 in FIG. 5) to the QR decomposition process (process 28 in FIG. 6) when the Gaussian elimination method is used. ). The details of the creation of the K-th block pivot row and column (steps 16 and 17 in FIG. 6) are also different from those of the Gaussian elimination method. However, the method (process 21) of the present invention in which several block pivot rows and columns are created in advance and the erasing operation is performed once in the MB / LB stage can be applied in exactly the same manner. The effect obtained by this method is the same as that of the Gaussian elimination method.
[0019]
In the present embodiment, the QR decomposition used in the least squares method and the like is shown as an application example to another linear operation. However, this method can be similarly applied to other linear calculation algorithms such as the Householder transform. .
[0020]
【The invention's effect】
As described above, according to the present invention, in a linear calculation such as a Gaussian elimination method on a parallel computer, both the stand-alone performance and the parallelization efficiency can be improved, and the calculation that fully exploits the performance of the parallel computer can be achieved. Becomes possible.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an entire linear calculation system using a parallel computer.
FIG. 2 is a diagram showing processor assignment to blocks in a conventional parallelized block Gaussian method.
FIG. 3 is a diagram showing that, in the conventional parallelized block Gaussian method, as a result of increasing the block size in order to improve the performance of a single unit, the number of blocks in charge of each processor becomes uneven.
FIG. 4 is a diagram showing that, according to the present invention, a large block size for an erasure operation can be obtained even if a small block size is used for processor allocation.
FIG. 5 is a diagram showing an entire procedure when the present invention is applied to a simultaneous linear equation solution method by the Gaussian elimination method.
FIG. 6 is a diagram showing an overall procedure when the present invention is applied to a QR decomposition of a matrix.
[Explanation of symbols]
1: input device, 2: simultaneous linear equation solving device, 3: output device, 4: entire coefficient matrix, 5: block assigned to processor 0, 6: block assigned to processor 1, 7: assigned to processor 15. Block, 8: whole block pivot column for 4 stages, 9: part of block pivot column used for calculation by processor 0, 10: whole block pivot line for 4 stages, 11: processor among block pivot lines Part used for calculation of 0, 12: Start, 13: Data input, 14: Assignment of coefficient matrix and right side vector to processor, 15: Outermost repetition loop, 16: LU of (K, K) block Decomposition, 17: Creation of K-th block pivot column, 18: Creation of K-th block pivot row, 19: Part by K-th block pivot column / row Erasure operation, 20: determination of whether K is a multiple of MB / LB, 21: simultaneous erasure operation for K stages, 22: forward erasure operation, 23: backward substitution operation, 24: collection of solution vectors, 25: Output of solution vector, 26: end, 27: assignment of matrix data to processor, 28: QR decomposition of (K, K) th block, 29: collection of solution matrix, 30: output of solution matrix, 31: processor, 32: cache memory.

Claims (4)

A method for performing a linear calculation on an N-dimensional matrix A using a parallel computer including an input device, a processing device including a plurality of processors each including a cache memory, and an output device,
Input the matrix A to be calculated and two different block sizes LB, MB,
The coefficient matrix A is divided into a plurality of blocks of size LB × LB, and the divided blocks are distributed and assigned to the plurality of processors.
The linear operation is sequentially performed for each block stage of the divided block,
The calculation of the K-th stage (1 ≦ K ≦ N / LB) of the sequential execution is as follows .
The processor assigned to each block of the block stage is in charge of each operation, and
a. Preprocessing operations required to create the Kth block axis column and the Kth block axis;
b. Creation of the K-th block axis row,
c. Creation of the K-th block axis line,
d. A part of the matrix A to be updated by using the K-th block axis row and the K-th block axis row, the elimination operation of the range necessary to create the next several stages of block axis rows and block axis rows, ,
It is determined whether or not K is a multiple of MB / LB, and when K is a multiple, in addition to the above,
e. Simultaneous update operations for MB / LB stages by the (K-MB / LB + 1), (K-MB / LB + 2),..., K-th block axis rows and block axis rows of the part to be updated of the matrix A Execute,
A linear calculation method characterized by performing parallel execution by the above method. 2. The linear calculation method according to claim 1, wherein the input processing is performed when MB × MB ≦ W ≦ 4 × MB ×, where the size of the cache memory measured in units of data used for calculation is W words. A linear calculation method including a process of determining the MB so as to satisfy the MB. 2. The linear calculation method according to claim 1, wherein the preprocessing operation is an operation for performing an LU decomposition of the (K, K) block of the coefficient matrix A in the N-dimensional simultaneous linear equation Ax = b. 2. The linear calculation method according to claim 1, wherein the preprocessing operation is an operation for performing a QR decomposition of the (K, K) block of the N-dimensional matrix A.

Images