JPH0962656A - Parallel computers - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accelerate the product operation of large sized matrices by calculating the element of a matrix product by transferring the column vector or row vector of a matrix between adjacent processor elements. SOLUTION: A controller 3 distributes first and second matrix data through a communication route 21 to respective PEs 1-N as the data of the column vector and row vector. The respective PEs calculate the inner product of the vector from the respectively distributed two pieces of data and return the calculated result through the communication route 21 to the controller 3 as one element of the matrix product. Also, the respective PEs 1-N transfer data of the row vector stored in a storage means 1b to one of the logically adjacent PEs connected by the communication route 22 provided separately from the communication route 21 so as to perform inner product calculation by the set of the different column and how vectors. Then, the respective PEs successively repeat the inner product operation, the transfer of the row vector and the return of the inner product calculated result. By these operations, all the elements of the matrix product are gathered in the controller 3 and the matrix product is appropriately obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel computer for processing matrix products at high speed among matrix calculations frequently appearing in scientific and technological calculations.

[0002]

2. Description of the Related Art Most of the scientific and technological calculations currently performed are linear calculations. This includes solving large-scale simultaneous linear equations, matrix eigenvalues, eigenvector problems, etc.
With regard to these typical problems, not only software studies such as numerical analysis, but also very detailed studies have been accumulated from the hardware aspect of vector computers.

The most basic operation in linear calculation is matrix operation. Matrix operations mean arithmetic operations between matrices, but the matrix multiplication is particularly important in scientific and technological calculations. Now, as an N × N dimensional matrix, A = (a _ij ), B =
(B _ij ), C = (c _ij ) as an example, the matrix product A = B · C
Consider computing. Here, the matrix product A = B
In actual calculation, C is calculated from the equation (1).

[0004]

[Equation 1]

The calculation expressed by the above equation (1) is simply N ³ product-sum operations, more precisely N ³ multiplications and (N-
1) N ² additions are required. That is, assuming that the total operation amount is O (N ^m ) for linear calculation, m = 3 in the calculation of the matrix product. However, it is known that there is a scheme for setting m <3 according to a very detailed analysis of matrix product calculation (V. Strassen, Numer. M).
ath. 13 354-356 (1969)). This is very interesting as a study of the algorithm, but in reality, the algorithm is complicated, and the dependence on the matrix size N is O (N ^m ): m <
Even if the number is 3, a relatively complicated procedure that does not depend on the size of the matrix is required, so that there is not much merit in practical use. Therefore, it is generally better to calculate the matrix product as defined.

By the way, if the matrix product is calculated using an ordinary vector type computer, there are some points that must be taken into consideration. Firstly, it is necessary to lengthen the vector length in order to use the vector register as effectively as possible. Secondly, the computing time of the computing unit is usually much faster than the memory access speed. Preventing continuous access to memory, ie avoiding bank conflicts. FIG.
An example of an algorithm usually adopted under such consideration is described by using the program description language FORTRAN.

Here, the matrix product arithmetic method as shown in FIG. 4 is called an intermediate product type (JJ Dongarra et.a.
l. , SIAM Rev. 26 (1) 91-112 (1
984)). This intermediate product type operation is known as the method with the fastest memory access. However, since the number of operations itself is N ³ times, if the number of vector registers is M, one operator needs to perform N ³ / M times of calculations. Usually, the size of a matrix may be in the range of tens of thousands to millions, but the number of vector registers is on the order of hundreds at most, such as 32 or 64, so that each computing unit requires a huge number of times. You have to calculate. Furthermore, depending on the size of the matrix, bank conflict may occur even if the intermediate product type is used, which is an unavoidable problem.

Therefore, the arithmetic unit in a general-purpose vector type computer has been devised so that such a huge amount of calculation can be processed at extremely high speed, and recently, it is possible to perform one calculation in the order of 1 nanosecond. However, in order to realize this, extremely expensive semiconductor elements and extremely advanced packaging technology are required.

However, matrix calculation is the basis of linear calculation and appears in a great many situations in scientific and technological calculation, but the calculation required for that is basically only the product-sum operation. Therefore, if only the adder and the multiplier are installed, the calculation can be performed and a complicated operation mechanism is not required. That is, in a vector computer, a matrix product is calculated with an unnecessarily large-scale equipment in order to process a very frequent calculation such as a matrix product at high speed. This was an unavoidable problem for the purpose of versatility.

On the other hand, since the matrix multiplication is a repetition of simple calculation, it is expected that it can be processed at high speed if it is executed by a recent massively parallel computer. For example, in a distributed memory type massively parallel computer, each PE (processor element) is usually equipped with a memory of about several megabytes, and data is appropriately distributed over the entire distributed memory. Each PE is connected by a communication path,
Data is exchanged between the PEs via the communication path as needed. Regarding the connection between the PEs, various networks such as an omega network have been proposed in order to exchange data between the PEs as fast as possible. In other words, network configuration technology is an important factor in determining the performance of massively parallel computers.

In a massively parallel computer, the key to high speed operation is to improve the locality of data due to its characteristics. In other words, no matter how good the network is, the communication time is much slower than the calculation time. That is, since the calculation is performed inside the microprocessor, it can be calculated in the order of 10 nanoseconds, but it is difficult to exchange data between PEs in the order of 10 nanoseconds. This is because it becomes important for efficient use of massively parallel computers.

Here, for example, when solving a simultaneous linear equation in which the coefficient matrix is a sparse matrix (sparse matrix), it is possible to allocate the data to the distributed memory while maintaining the locality. However, in the matrix multiplication operation, there is no data locality from the definition. Therefore, when trying to raise the parallelism to the limit by using such a massively parallel computer, in the simplest case, all PEs have all the data of matrices B and C. For example, PE (ij) is the equation (2). In principle, it is possible to process the calculation time of order N by performing the calculation shown and collecting all the matrix elements at the end.

[0013]

[Equation 2]

However, considering that N is in the order of tens of thousands to millions, it is very difficult to implement N ² PEs, and in order to access data from any PE at high speed. , A complex network is required in terms of electronic circuits. Moreover, there is a waste that only a part of the data held by each PE is used.

By the way, the distributed memory type massively parallel computer must have a very sophisticated network in order to ensure versatility as a computer. Here, if the problem itself that the computer is trying to solve has a low degree of parallelism, it cannot be improved. On the other hand, a high-level network is required to process a problem to be solved by a computer, which has a high degree of parallelism but has poor locality of data, at high speed. However, the matrix multiplication operation is simple, and it can be said that it is unnecessary to provide a general-purpose network such as an Omega network.

[0016]

As described above, a vector type computer or a massively parallel computer can be considered as a computer for high-speed processing of matrix product operations that frequently appear in scientific and technological calculations. However, as a computer for performing operations that occur very frequently, such as matrix multiplication,
There is a problem that the vector computer requires expensive semiconductor elements and the like, and the massively parallel computer requires a complicated network.

The present invention has been made in consideration of the above circumstances, and does not require expensive semiconductor elements and does not require a complicated network. Its purpose is to provide a parallel computer that runs at a much higher speed than the.

[0018]

The present invention (Claim 1) includes:
A plurality of processor elements, a controller that distributes data to each processor element and collects a calculation result, a first communication path that connects the controller and each processor element, and a processor element that is logically adjacent A parallel computer comprising a second communication path provided separately from the first communication path, wherein the control device comprises:
The first communication path is the first source of matrix multiplication operation.
A plurality of first vectors formed by grouping the elements of the matrix in one of the vertical direction and the horizontal direction are exclusively distributed to each of the processor elements, and Each of the processor elements includes means for exclusively distributing a plurality of second vectors formed by grouping elements in a direction different from the first vector in the vertical direction and the horizontal direction, to each of the processor elements. Is a first storage means for storing the first vector, a second storage means for storing the second vector, and the first storage means stored in the first storage means. A multiplier that sequentially multiplies each element of the vector and each element of the second vector stored in the second storage unit, and cumulatively adds the multiplication results of the multiplier. An adder and the transferred first vector are stored in the first storage means, the transferred second vector are stored in the second storage means, and a cumulative addition result of the adder is stored. The element is transferred to the control device via the first communication path as an element of a matrix product of the first matrix and the second matrix, and the second vector is logically transferred via the second communication path. And a control means for controlling transfer to one of the adjacent processor elements.

According to the present invention (claim 2), in the above invention (claim 1), the second storage means is configured by using a dual port memory having a first port and a second port. The second communication path includes, for each of the processor elements, a first port of a dual port memory of one processor element and a dual port memory of one of the processor elements logically adjacent to the one processor element. A plurality of dedicated communication paths provided so as to connect to a second port, wherein the control means includes one of the first port and the second port of the dual port memory and the second vector. Is read out and transferred to one of the processor elements that are logically adjacent by the dedicated communication path, Control of writing the second vector transferred from the other processor element logically adjacent to the dedicated communication path of the second port from the other of the first port and the second port of the dual port memory. It is characterized by

According to the present invention (claim 3), in the above invention (claim 2), each of the processor elements is
A first register for storing the cumulative addition result of the adder, a second register provided between one input end of the multiplier and the first storage means, and the second storage means. And a third register connected to one port of the dual-port memory that constitutes the second storage means, a fourth register connected to the other port of the dual-port memory that constitutes the second storage means, and the multiplication. Connected to the other input end of the multiplier, the third register, and the fourth register of the adjacent processor element via the second communication path, and the other input end of the multiplier and the third register And a selector for selectively connecting the third register with the fourth register of the processor element adjacent to the third register. The multiplication of the first vector data and the second vector data stored in the storage means and the second storage means via the second register and the third register and the selector, respectively. To the controller, transfer the cumulative addition result of the adder to the controller via the first register via the first communication path, and store the second vector stored in the second storage means. Transfer to the one processor element logically adjacent to the dedicated communication path via the fourth register and one of the third register and the selector, and logically via the other dedicated communication path. The second vector transferred from the other adjacent processor element is converted into the fourth vector.
And the third register and the other of the selectors are used to control the storage in the second storage means.

According to the present invention (claim 4), in the above invention (claim 1), the second communication path is configured by using a bus which logically connects the plurality of processor elements in at least one direction. The control means reads out the second vector from the second storage means and logically adjoins the second communication path according to a predetermined order for the plurality of processor elements. To the processor element of the second
The second vector transferred from the other processor element logically adjacent to the second vector is controlled to be written in the second storage means.

According to the present invention (Claim 5), in the above invention (Claim 4), each of the processor elements is
The control means further includes a register for storing the cumulative addition result of the adder, and the control means performs control of transferring the cumulative addition result of the adder to the control device via the register via the first communication path. Is characterized by.

(Operation) In the present invention (Claim 1), first, the control device sets the data of the first matrix and the second matrix for calculating the matrix product as vector data in two directions different from each other, that is, a vertical vector. The data of the lateral vector is distributed to each PE (processor element) via the first communication path.
When the first matrix is on the multiplication side and the second matrix is on the multiplied side, the first vector based on the first matrix is the horizontal vector and the second vector based on the second matrix is the vertical vector. Vector. On the other hand, when the second matrix is on the multiplication side and the first matrix is on the multiplied side, the second vector based on the second matrix is the lateral vector and the first vector based on the first matrix. Is the vertical vector.

Each PE calculates the inner product of the vector from the two distributed data, and the calculated result is used as the first
As an element of the matrix product of the matrix and the second matrix
It returns to the control device via the communication path of.

In addition, each PE uses a second vector between each PE in order to perform inner product calculation with different first and second vector sets (that is, in order to obtain other elements of the matrix product).
Swap the vector of. That is, each PE uses the first vector stored in the second storage means as the first vector.
Data is transferred to one of the logically adjacent PEs connected by a second communication path provided separately from the communication path.

Thus, each PE sequentially repeats the above inner product calculation and transfer of the second vector, and the return of the inner product calculation result. By the above operation, the inner product calculation result by all the pairs of the first and second vectors, that is, all the elements of the matrix product to be obtained are collected in the control device, and the control device appropriately configures the calculation result and calculates the matrix product. You can ask.

In the inner product calculation performed by each PE, all the data stored in each PE are independent, and the time required for multiplication and addition is the same in each PE. That is, multiplication and addition can be performed at the same time in all PEs. In other words, the degree of parallelization is 1
It is 00%.

In addition, since it is not necessary to synchronize the end of the inner product calculation, the arbitration procedure regarding the right of use of the first communication path becomes unnecessary, and each PE immediately transfers the data when the inner product calculation is completed. You can get started. That is, unlike the conventional massively parallel computers, the data locality is completely maintained.

Further, when a certain PE is transferring data, all PEs are transferring data, and the parallelization degree is 100% from the viewpoint of data transfer. As described above, according to the present invention, the process in the matrix product calculation can be performed at a much higher speed than the conventional one.

Further, each PE may be provided with an adder, a multiplier and a memory, and in particular, the memory has the number of digits of the matrix in the matrix calculation, for example, in the matrix product calculation of N × N matrix, the length is N. Since it suffices to have a memory capable of holding two vectors of the above, very expensive semiconductor elements and extremely advanced mounting technology unlike the conventional vector computer are not required.

The second communication path may have a function for sending data to PEs which are logically adjacent to each other, so that it can be used for any PEs used in a conventional massively parallel computer. There is no need to use complicated networks to send data to and from it.

Further, since the calculation procedure is fixed, the high speed of calculation can be maintained without providing a cache with a high speed storage element. According to the present invention (Claims 2 and 3), regarding the data transfer via the second communication path in each PE, the memory in which the data transfer is performed is a dual port memory. Therefore, in the data transfer of each PE, it is possible to omit the hardware required for using the single port memory, such as a register.

Further, by using the dual port memory, the second communication path does not have to be common to each PE. That is, the second communication path can be configured such that the logically adjacent PEs are directly connected in a chain.

As a result, regarding the data transfer in each PE, when the data in the dual port memory is sequentially written into the dual port memory in the adjacent PE which is logically adjacent, the data in the dual port memory in another adjacent PE is almost simultaneously written. Can be loaded into your own dual-port memory. However, in this case, in order to prevent access to the same memory cell, it is necessary to write the data immediately after reading the data.

As a result of the above, since data transfer is simultaneously performed in all PEs, the processing speed can be improved. In the present invention (Claim 4), the second communication path is configured by using a bus that logically connects the plurality of PEs in at least one direction logically, but no arbitration procedure is required. , Transfer can be performed according to a predetermined order for the plurality of PEs.

In the present invention (claims 3 and 5), each P
The result of the inner product calculation of the vector performed at E is not stored directly in the control unit but is temporarily stored in a register, and the control unit receives the calculation result stored in the first register. The inner product calculation and data transfer in the above, and the calculation result return can be performed separately and independently.

In this way, the time required to transfer the calculation result is hidden behind the inner product calculation processing and does not affect the actual processing time. Generally, the time required for communication is much slower than the time required for calculation,
According to the present invention, the processing time can be improved because the time required for returning the calculation result can be ignored.

[0038]

Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) First, a first embodiment of the present invention will be described.

FIG. 1 shows the system configuration of a parallel computer according to this embodiment. As shown in FIG. 1, the parallel computer includes a controller 3, N (N is an integer of 2 or more) PEs.
1 to PEN, a first communication path 21 that connects the control device 3 and PEs 1 to PEN, and a second communication path 22 that is used for each PE to transfer data to a logically adjacent PE.

The control unit 3 controls the PE1 to PEN and the data of two matrices for matrix product calculation (N × M matrix X on the multiplication side and M × N matrix Y on the multiplied side). And a memory 32 for storing the calculation result of the inner product returned from PE1 to PEN. Based on the data of the matrices X and Y stored in the memory 32, the control unit 31 sets the vector data of the number M of elements (N horizontal vectors of the matrix X and N vertical vectors of the matrix Y) PE1 to PE1. The matrix product is constructed from the results of the inner product calculation (N × N matrix Z = each element of XY) which is exclusively distributed to PEN and returned from PE1 to PEN. It is preferable that the control unit 31 explicitly gives an arithmetic command to each PE when distributing the vector data.

PE1 which is one of the plurality of PEs includes a first distributed memory 1a, a second distributed memory 1b, a multiplier 1c,
It has an adder 1d and a register 1e. Distributed memory 1a,
Reference numeral 1b is a memory that stores vector data for inner product calculation, and is connected to the input side of the multiplier 1c. The output of the multiplier 1c is connected to one input of the adder 1d. Further, the output of the adder 1d itself is connected to the other input of the adder 1d by the loop circuit 1L, whereby the adder 1d calculates the total sum of the output data of the multiplier 1c. The output of the adder 1d is connected to the register 1e. The multiplier 1c and the adder 1d are 32 bits to 6 bits depending on the application.
It consists of about 4 bits.

Other PE2 provided in this parallel computer
N has the same configuration as PE1 described above, and has distributed memories 2a to Na, 2b to Nb, multipliers 2c to Nc, adders 2d to Nd, and registers 2e to Ne, respectively.

The first communication path 21 is for connecting the PE1 to PEN and the control device 3, and transfers data for calculating the matrix product from the control device 3 to PE1 to PEN and from PE1 to PEN. It is used for transferring the calculation result of the returned inner product.

The second communication path 22 is provided for each PE to transfer data to a logically adjacent PE. In FIG. 1, for example, the m-th PEm is the m-th PE-.
It is assumed to be logically adjacent to the first PEm-1 and the m + 1th PEm + 1. However, PE1 is PE2
And PEN and PEN are logically adjacent to PE1 and PEN-1.

The communication path 22 only needs to have a function of sending data from each PE to a logically adjacent PE, and any P
It does not have to be a network capable of transferring data to E. Further, the communication path 22 may have a function of bidirectionally transmitting data from each PE to a logically adjacent PE, but only a function of transmitting data from each PE to one logically adjacent PE. May have.

The operation in each PE is controlled by an in-PE control unit (not shown) provided in each PE. The PE control unit starts the control in response to an arithmetic command from the control device 3 and passes the last (Nth) arithmetic result to the control device 3 to end the control.

Here, the basic operation in PEm (m = 1 to N) will be briefly described. First, two vectors for inner product calculation transferred from the control device 3 and stored in the distributed memories ma and mb, respectively. -The respective elements of the data are sequentially multiplied by the multiplier mc. The result of the multiplication is successively given to the adder md provided with the loop circuit mL,
Here, cumulative addition is performed to obtain the inner product of the above two vector data.

The result of the inner product calculation is temporarily stored in the register me and then returned to the control device 3. At the same time, the PEm passes one of the two vector data to the adjacent PEm + 1 (or PEm-1).

Then, each PE calculates the inner product with respect to the new two vector data as described above, and returns the result to the control device 3, and at the same time,
Pass the data to the adjacent PE.

As described above, in each PE, the inner product calculation of two vector data and the control device 3 for calculating the calculation result are performed.
To N and one vector data to the neighboring PE are repeated (N times), and finally all the elements of the matrix product are aligned in the controller 3.

Next, the operation of obtaining the matrix product by the parallel computer of this embodiment will be described with reference to the flow chart shown in FIG. In the following, a case where the matrix product A = B · C is obtained for an N × N dimensional square matrix A = (a _ij ), B = (b _ij ), C = (c _ij ) will be described as an example.

First, the elements of the matrices B and C for matrix product calculation are stored in the memory 32 of the controller 3 (step S1). Next, the control device 3 distributes the data stored in the memory 32 to the PE1 to PEN, for example, as follows via the communication path 21 (step S2).

The horizontal vector {b _1k } of the matrix B and the vertical vector {c _k1 } of the matrix C are stored in the first distributed memory 1a and the second distributed memory 1b of the PE1, respectively. PE
A horizontal vector {b _2k } and a vertical vector {c _k2 } are stored in the first distributed memory 2a and the second distributed memory 2b, respectively. First distributed memory 3a of PE3, second
The horizontal vector {b _3k } and the vertical vector {c _k3 } are stored in the distributed memory 3b of FIG. The horizontal vector {b _Nk } and the vertical vector {c _kN } are stored in the first distributed memory Na and the second distributed memory Nb of the PEN, respectively. That is, the horizontal vector {b _mk } and the vertical vector {c _km } are stored in the first distributed memory ma and the second distributed memory mb of PEm (m is 1 to N), respectively. Here, k = 1, 2, ..., N.

Contrary to the above, PEm (m is 1 to
N) of the first distributed memory ma, the vertical vector of the matrix C {c
_{It is also possible} to store _km } and store the lateral vector {b _mk } in the second distributed memory mb.

The distribution of the horizontal vector and the vertical vector to each PE need not be regular as described above as long as they are exclusive. No matter which horizontal vector or vertical vector is distributed to which PE, all elements of the matrix product can be finally obtained.

After the storage is completed as described above, the PE control unit of each PE initializes the variable CNT for recording the number of calculations to 0 (step S3). Next, the in-PE control unit of each PE compares the variable CNT with the number of data N (step S4), and when CNT ≧ N is not satisfied, the following steps S5 to S9 are executed. By comparing the above and performing the following operations while the conditions are satisfied,
The elements forming the matrix product A can be sequentially obtained in PE1 to PEN. Then, in step S4, CN
When T ≧ N is satisfied, all the elements of the matrix product A are arranged in the control device 3, and the process ends. It should be noted that using the variable CNT for the end determination is an example, and other methods may be used.

If CNT ≧ N is not satisfied in step S4, vector data {b _1k } to {b _Nk }, {c _k1 } to {c _kN } distributed in PE1 to PEN, respectively.
The inner product of the vector is calculated on the basis of (step S5).
That is, PEm (m = 1 to N) is distributed memory ma, m
The inner product obtained by the equation (3) is calculated from the two vectors {b _mk } and {c _km } stored in b.

[0058]

(Equation 3)

For example, the operation will be described focusing on PE1. First, b ₁₁ stored in the distributed memories 1a and 1b.
And the product of c ₁₁ are multiplied by the multiplier 1c and sent to the adder 1d. Next, the product of b ₁₂ and c ₂₁ is calculated in the same manner and sent to the adder 1d. The adder 1d cumulatively adds the present result to the previous result. Similarly, the distributed memories 1a, _b1N and _cN1
All vectors stored in 1b are multiplied and cumulatively added to the adder 1d. Distributed memory 1a,
When the calculation is completed for all the data stored in 1b, the adder 1d stores the calculation result in the register 1e. The calculation result stored in the register 1e represents the element a ₁₁ in the matrix product A.

The inner product calculation is performed in all PEs as in PE1 described above. The register 1e~Ne in PE1~PEN elements a ₁₁ ~a _NN is stored. These elements indicate the diagonal elements of the matrix product A.

Since the time required for multiplication and addition is the same in all PEs, the diagonal elements of the matrix product A are obtained at the same time when all PEs operate in parallel at the same time. In that case, the diagonal elements of the matrix product A can be obtained in N clocks.

The elements a _{11 to} a _NN stored in the registers 1e to Ne as described above are returned to the control device 3 via the communication path 21 (step S6). Control device 3
Is the diagonal elements a _{11 to} a _NN of the returned matrix product A,
The elements a _{11 to} a _NN of the matrix product A in the memory 32 are stored at appropriate addresses. (Step S7).

Here, in this embodiment, the execution of the above steps S6 and S7 is performed by the steps S4 and S5.
And the steps S8 and S9 in the subsequent rounds can be performed independently.

Now, when the inner product calculation is completed for one set of vector combinations as described above, PE1 to PE1
The PEN uses one of the data recorded in the two distributed memories 1a to Na and 1b to Nb for the communication path 2
The data is transferred to the PEs that are logically adjacent to each other via the P. Here, the distributed memories 1a to Na, 1b.
It is assumed that the data in the distributed memories 1b to Nb among the two data stored in .about.Nb are transferred. In this case, for example, each PE operates as follows (step S8).

The PE ₁ holds the horizontal vector {b _1k } of the distributed memory 1a as it is and transfers the vertical vector {c _k1 } of the distributed memory 1b to the PEN. The PE2 holds the horizontal vector {b _2k } of the distributed memory 2a as it is and transfers the vertical vector {c _k2 } of the distributed memory 2b to the PE1. PE
m holds the horizontal vector {b _mk } of the distributed memory ma as it is and the vertical vector {c _km } of the distributed memory mb PEm.
Transfer to -1. The PEN holds the horizontal vector {b _Nk } of the distributed memory Na as it is and transfers the vertical vector {c _kN } of the distributed memory Nb to PEN-1.

That is, PEm (m is 1 to N) holds the horizontal vector {b _mk } of the distributed memory ma as it is and transfers the vertical vector {c _km } of the distributed memory mb to PEm-1. If the communication path 22 transfers data sequentially, the transfer can be performed in two clocks.

In the above example, the PEm data is PEm-
It was decided to transfer to 1, but the opposite direction, that is, P
The data of Em + 1 may be transferred to PEm. In this case, the PEN data is transferred to PE1.

Next, the in-PE control unit of each PE that has transferred the data assumes that the inner product calculation has been completed for a certain set of vector combinations for the data transferred from the control device 3, and sets the variable CNT to 1 respectively. Only (step S9).

As described above, each PE repeats the above steps S5 to S9 sequentially until CNT ≧ N is satisfied in step S4. As a result, the control device 3
The inner product calculation is executed for all the combinations of the horizontal vector and the vertical vector in the data transferred from.

That is, in the first round, the element a _mm (m = 1 to N) in the matrix product A is obtained, and the adjacent P
After the data transfer to E, in the second round PEm (m = 1 to N), the calculation represented by the equation (4) is performed. However, m + 1 = (m + 1)
mod N.

[0071]

(Equation 4)

The result of this calculation is the element a in the matrix product A.
_{mm + 1} (that is, a ₁₂ , a ₂₃ , ...). Similarly, in each round, the elements a _{mm + 2} , a _{mm + 3} , ..., A _{mm + N-1} in the matrix product A are obtained.

Since the calculation results are sequentially transferred to the control device 3, if CNT ≧ N is satisfied in step S4, it means that all the elements in the matrix A have been stored in the memory 32 of the control device 3. Become.

The above is an example of the operation for obtaining the matrix product by the parallel computer of this embodiment. Next, consider the time required for the matrix product calculation of the N × N matrix in this embodiment.

Generally, in the matrix product calculation of N × N matrix, the sum of products calculation itself is N ³ times of multiplication and (N−
1) N requires ^two additions but, according to this embodiment, it is possible to perform N ² times and multiplying the (N-1) N additions simultaneously. That is, a parallelization degree of 100% can be obtained. Further, the locality of data is completely maintained, and all PEs are transferring data when transferring data to the closest PEs, so the parallelization degree is 100% from the viewpoint of data transfer.

The multiplications and additions, which are the basic operations of the matrix product, are always completed with a fixed number of clocks. It is assumed that multiplication and addition can be processed in 2 clocks each. This is because the addition itself can be executed in one clock, and when the multiplier is provided in the parallel computer, the multiplication itself can be executed in one clock, but at least two clocks are required to perform the data fetch. As described above, in this embodiment, N
^{Since two} multiplications and (N-1) N additions are performed,
The number of clocks required for multiplication and addition is N ² × 2 + (N-
1) N × 2 clocks.

It is assumed that the time required for data transfer in each PE is 2 clocks because each PE writes and reads. Here, since the time required for multiplication and addition in each PE is fixed, even if the end of the inner product calculation in each PE is not synchronized, the control unit 31 of the control device 3 issues an instruction to start the calculation. P
If the PEs are given to E at the same time and the inner product operations are started simultaneously at each PE, each PE may start data transfer at the time when the inner product calculation is completed.

By the way, in order to return the inner product calculated by each PE to the control device 3, it goes through the communication path 21. that time,
Since all these data are independent, it is possible in principle to transmit data at the same time, but if each data is 32 bits or 64 bits, P
Given that the E-number is on the order of 10 ⁴ , it is actually difficult to implement so that this can be done simultaneously. Therefore, in this embodiment, the same configuration as that of a normal bus is adopted, but since the processing to be performed by each PE is constant, the arbitration procedure for the right to use the communication path 21 is not necessary. That is, in order from a certain PE, for example, DM
Data transfer may be performed in the same way as A (Direct Memory Access) transfer. In this case, it takes two clocks in total for the address and the data to fetch the data from one PE into the control device 3 via the communication path 21. After all, since there are N pieces of data, 2N clocks are required. However, since this data transfer has nothing to do with the operation of the multiplier or adder of each PE, the registers 1e to Ne are provided in each PE to transfer the data while each PE processes the inner product. It is possible. Therefore, this time is actually hidden behind the processing of the inner product operation and does not affect the processing time.

On the other hand, to send data to the adjacent PE,
It is possible with two clocks. Since data transfer to the adjacent PE is performed N times, 2N clocks are required in total. In this embodiment, the communication path 22 has the same configuration as that of a normal bus, but the communication path 21
As with, no arbitration procedure for the right of use is required. That is, data may be transferred sequentially from a certain PE.

As a result of the above, N × in this embodiment.
The number of clocks required for the matrix multiplication of N matrices is (multiplication and addition) + (data transfer) = N ² × 2 + (N−1) N × 2 +
2 × N = 4N ² clocks.

The clock frequency of recent standard microprocessors is often about 100 MHz. Therefore, assuming that the clock frequency is 100 MH, that is, the clock time is 10 nanoseconds in this embodiment, N = 10.
It takes 400 seconds to obtain the matrix product of ⁵ . If this is recalculated to the number of floating point operations per unit time, 5
It becomes TFLOPS (5 × 10 ¹² FLOPS). That is, according to this embodiment, while the fastest existing supercomputer exhibits an execution performance of about 10 GFLOPS, it is possible to realize a processing speed 100 times or more that higher.

Further, according to this embodiment, each P
Since only basic elements such as a memory element, a multiplier, and an adder need to be provided in E, extremely expensive semiconductor elements and extremely high-level mounting technology unlike conventional vector computers are not required. .

The distributed memory for storing the vector data has the size of the matrix in the matrix calculation, for example, in the matrix product calculation of the N × N matrix, it is sufficient to hold the vector of length N. There is no need for a large capacity device like.

The communication path for transferring the vector data may have a function for sending the data to the logically adjacent PE, so that it can be connected to any PE as used in the conventional massively parallel computer. There is no need to use complicated networks to send data to and from it.

Since the operation procedure is fixed, the high speed operation can be maintained even if the cache of the high speed storage element is not provided. (Second Embodiment) Next, referring to FIG. 3, a second embodiment of the present invention will be described.
The embodiment will be described. In this embodiment, parts corresponding to those in FIG. 1 are designated by the same reference numerals, and differences from the first embodiment will be mainly described.

FIG. 3 shows the system configuration of the parallel computer according to this embodiment. As shown in FIG. 3, in this parallel computer, one of the two distributed memories in the parallel computer of the first embodiment (memory 1b to Nb used for data transfer to an adjacent PE) is a dual port memory (FIG. 3). 1b2 to Nb2), and the adjacent PEs are directly connected.

Further, in this embodiment, each of PE1 to N is
, The first distributed memories 1a to Na and the multiplier 1c to
Second registers 1f to Nf are connected to Nc, and third registers 1g to Ng and a selector 1h are provided between one port of the second distributed memories 1b2 to Nb2 and the multipliers 1c to Nc. ~ Nh are connected. Further, the fourth registers 1i to Ni are connected to the other ports of the second distributed memories 1b2 to Nb2.

At the time of inner product calculation, the second registers 1f to N
f and the third registers 1g to Ng store the data of the first distributed memories 1a to Na and the second distributed memories 1b2 to Nb2, respectively, and transfer the data to the multipliers 1c to Nc via them. are doing. In addition, the third registers 1g to Ng and the fourth registers 1i to Ni
Temporarily stores the data read from the second distributed memories 1b2 to Nb2 and the data to be written to the second distributed memories 1b2 to Nb2 at the time of data transfer between the adjacent PEs.

The selectors 1h to Nh are switching-controlled by a PE control unit (not shown) in order to control the flow of data in each PE. That is, the selector m of each PEm
When the inner product is calculated in step S5 in FIG. 2, h connects the multiplier mc and the register mg, and in the data transfer to the adjacent PE in step S8 in FIG.
Register m-1i of PEm-1 logically adjacent to Em
Switch to connect and.

Here, an example of the data transfer to PE1 to the adjacent PE will be described. First, the second distributed memory 1
Data is read from b2 and once transferred to transfer register 1i
Stored in. Then, it is transferred to the PE 2 via the communication path 22. Further, the data recorded in the transfer register Ni of PEN is transferred to PE1 via the communication path 22. This data is sent from the selector 1h to the second register 1g. Then, the data is written in the second distributed memory 1b2. Here, transfer of data to PE2 and PEN
The transfer of data from is done at the same time. The data transfer as described above is simultaneously performed in all other PEs.

The data transfer may be performed in the opposite direction. In this case, the functions of the register mg and the register mi in the data transfer to the adjacent PE are reversed. On the other hand, when calculating the inner product of vector data,
Distributed memories 1a to Na and second distributed memory 1b2
-Nb2 from first register 1f to Nf, second register 1g to Ng and selector 1h to N, respectively.
Data is sent to the multipliers 1c to Nc via h.

The parallel computer of the second embodiment as described above performs the same processing as in FIG. 2 and stores the calculation result of the matrix product in the memory 32 of the control device 3. With such a configuration, in the data transfer of each PE, it is possible to omit the hardware necessary for data exchange when using the single-port memory as in the first embodiment, such as a register.

Further, by using the dual port memory as the second distributed memories 1b2 to Nb2, it becomes possible to directly connect the PEs logically adjacent to each other in a chain. As a result, the second distributed memory 1
The data of b2 to Nb2 is read and transferred to the adjacent PE that is logically adjacent to the other, and the other adjacent P
The data transferred from E is stored in its second distributed memory 1b.
A series of operations for writing 2 to Nb2 can be simultaneously performed for all PEs.

Furthermore, the second distributed memories 1b2 to Nb2
Data is stored in the registers 1i to Ni in advance, and the data stored in the registers 1i to Ni is transferred to the logically adjacent PEs and temporarily stored in the registers 1g to Ng. Therefore, the data reading and the data writing can be performed almost simultaneously in the second distributed memory of each PE. Of course, in order to prevent access to the same memory cell of the second distributed memory,
It goes without saying that writing is performed immediately after reading the data.

As described above, according to this embodiment, data transfer between adjacent PEs can be performed at a higher speed,
It is possible to further improve the processing speed of the matrix product calculation. By the way, in each of the above-described embodiments, N × N
An example of obtaining the matrix product of two-dimensional square matrices has been described, but of course, the present invention, like the matrix product of N × M matrix and M × N matrix, is a matrix that can be multiplied by the number of lateral vectors of the matrix to be multiplied. Can be applied to the matrix product calculation of non-square matrices as long as the condition that the number of vertical vectors of is equal and the number of vertical vectors of the matrix to be multiplied and the number of horizontal vectors of the matrix to be multiplied are equal .

Further, in each of the above-mentioned embodiments, the data transfer in each PE is performed using the communication path 22 without passing through the arithmetic unit including the multiplier and the adder of each PE. It is also possible to perform data transfer via the ordinary bus 21 via the arithmetic unit of each PE without providing the communication path 22. However, in this case, two clocks are required to replace the data once, and since this data is the original data for obtaining the inner product, this time cannot be hidden behind the operation, so the matrix product is used. The total calculation time is 6N ² -2N clocks. That is, the time required for communication occupies a little over 1/3 of the total calculation time.

Further, in each of the above-mentioned embodiments, the PE is constructed as a dedicated hardware. However, it is sufficient for the PE to have a logically independent memory capable of performing multiplication and addition and holding two vectors. Therefore, for example, it is possible to regard a commercially available personal computer or EWS (engineering workstation) as each PE and configure hardware dedicated to matrix multiplication. The present invention is not limited to the above-described embodiment, and can be implemented with various modifications within the technical scope.

[0098]

According to the present invention, in order to perform a matrix product calculation that frequently appears in scientific and technological calculations, each processor element is provided with an adder and a multiplier having a relatively simple structure, and each processor element has a matrix. Since either the vertical vector or the horizontal vector of the matrix that is the source of the product calculation is carried around by transferring between adjacent processor elements, the elements of the matrix product are calculated in a distributed and sequential manner. , Without using expensive semiconductor devices such as vector computers, and without using networks that require complicated and sophisticated mounting technology such as massively parallel computers, matrix product calculation is much faster than before. Can be done.

[Brief description of drawings]

FIG. 1 is a diagram showing a system configuration of a parallel computer according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an operation flowchart of the parallel computer according to the first embodiment of the present invention.

FIG. 3 is a diagram showing a system configuration of a parallel computer according to a second embodiment of the present invention.

FIG. 4 is a diagram showing an example of a matrix product calculation algorithm in a conventional vector computer.

[Explanation of symbols]

1-N ... PE (processor element) 1a-Na, 1b-Nb ... Distributed memory 1b2-Nb2 ... Dual port memory 1c-Nc ... Multiplier 1d-Nd ... Adder 1L-NL ... Loop circuit 1e-Ne ... Register, 1f to Nf, 1g to Ng, 1i
～Ni ... register 1H～Nh ... selector 21, 22, 23 ₁ ~ 23 _N ... communication path (bus) 3 ... controller 31 ... controller 32 ... memory

Claims (5)

[Claims] 1. A processor is logically adjacent to a plurality of processor elements, a control device for distributing data to each processor element and collecting operation results, and a first communication path connecting the control device and each processor element. A parallel computer having a second communication path provided separately from the first communication path connecting a processor element, wherein the control device uses the first communication path to perform matrix product calculation A plurality of first vectors obtained by grouping the elements of the first matrix, which are arranged in one of the vertical direction and the horizontal direction, to each of the processor elements exclusively, A plurality of second vectors formed by grouping the elements of the two matrices in the vertical direction and the horizontal direction in a direction different from the first vector; Each of the processor elements has a first storage means for storing the first vector, and a second storage means for storing the second vector. Storage means, and a multiplier for sequentially multiplying each element of the first vector stored in the first storage means and each element of the second vector stored in the second storage means, An adder for accumulatively adding multiplication results by the multiplier, storing the transferred first vector in the first storage means, and storing the transferred second vector in the second storage means And the cumulative addition result of the adder is the first
Of the matrix and the second matrix are transferred to the control device through the first communication path as one element of the matrix product, and the second vector is logically adjacent to the other one side through the second communication path. And a control means for controlling the transfer to the processor element. 2. The second storage means is configured by using a dual port memory having a first port and a second port, and the second communication path is one for each processor element. A plurality of dedicated units provided to connect a first port of a dual port memory of a processor element and a second port of a dual port memory of one of the processor elements logically adjacent to the one processor element. A communication path, and the control means includes the first port of the dual port memory.
The second vector from one of the second port and the second port, transfers the second vector to one of the processor elements that is logically adjacent to the dedicated communication path, and logically adjacent to the other dedicated communication path. 2. The control for writing the second vector transferred from the other processor element of the dual port memory from the other one of the first port and the second port of the dual port memory. Parallel computer. 3. The processor elements each include a first register for storing a cumulative addition result of the adder, and a first register provided between one input end of the multiplier and the first storage means. A second register, a third register connected to one port of the dual port memory forming the second storage means, and a third port connected to the other port of the dual port memory forming the second storage means. Connected to a fourth register connected thereto, the other input end of the multiplier, the third register and the fourth register of the adjacent processor element via the second communication path, Selectively connecting between the other input terminal of the multiplier and the third register and between the third register and the fourth register of the adjacent processor element; A selector, and the control means stores the first vector data and the second vector data stored in the first storage means and the second storage means, respectively, in the second register. And the cumulative addition result of the adder is given to the multiplier via the third register and the selector, respectively, and transferred to the control device via the first communication path via the first register, The second vector stored in the second storage means is stored in the fourth register;
And via one of the third register and the selector to the one processor element that is logically adjacent to the dedicated communication path, and the other of the processor elements that is logically adjacent to the other dedicated communication path. A control for storing the second vector transferred from the processor element in the second storage means via the fourth register and the other of the third register and the selector is performed. The parallel computer according to Item 2. 4. The second communication path is configured by using a bus that logically annularly connects the plurality of processor elements in at least one direction, and the control means is the second storage. Means for reading the second vector, transferring the second vector to the one processor element logically adjacent to the second communication path according to a predetermined order for the plurality of processor elements, and logically transferring the second vector to the processor element. 2. The parallel computer according to claim 1, wherein control is performed to write the second vector transferred from the other processor element that is physically adjacent to the second storage means. 5. Each of the processor elements further comprises a register for storing a cumulative addition result of the adder, and the control means controls the cumulative addition result of the adder via the register to the first communication path. 5. The parallel computer according to claim 4, wherein the control is performed to transfer to the control device.