JPH01147767A - Double cascaded parallel processing system - Google Patents

Abstract

PURPOSE:To improve the operation rate of an element processor PE by carrying forward plural summing calculations concurrently in a case that sequential operational processing is performed by a cascade system. CONSTITUTION:A host computer 1 performs the loading of a program, the transfer of data, a scalar operational processing and the progress control of an operation to/of the PE 3 through an array controller 2. The PEs 3 are arrayed by L sets in a lateral direction and M sets in a longitudinal direction. IN order to obtain a sum total, at first, a cascade sum is taken toward right direction for every row, and the sum total Xsum is obtained on the uppermost PE(L,M). Simultaneously with this sum total calculation, the sum total Ysum of data (y) is obtained. By using this calculation, since the PEs necessitated for the sum total calculation of xi do not overlap the PEs necessitated for the sum total calculation of yi, the calculation to obtain Xsum and the calculation to obtain Ysum can be executed in parallel. Accordingly, the operation rate of the PE is improved doubly by using this calculation system.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an arithmetic processing method using an electronic computer, and particularly relates to an arithmetic processing method using an electronic computer, and in particular, to
In a parallel computer consisting of
An operation that refers to data distributed and stored in E.

It relates to efficient processing methods.

[Conventional technology]

The inter-processor coupling methods of a parallel computer consisting of a large number of element processors include: - Lattice coupling method - Hypercube (hypercubic lattice) coupling method - Matrix crossbar switch coupling method (for example, Japanese Patent Application No. 61-26
9655 Sanfu) etc. When discretizing partial differential equations and solving them on a parallel computer, the space to be analyzed is divided into multiple subspaces,
It is common to have each PE process grid points contained in a subspace (which includes one or more grid points). Therefore, it is natural to store data (values of variables) little by little in a distributed manner in storage devices attached to each PE. Such data allocation often occurs not only in solving partial differential equations, but also in problems that involve processing a large amount of data, such as image processing.

By the way, in the process of calculation, data distributed to all or some PEs is referred to, so even though the calculation order is commutative, calculations that must be processed sequentially often occur.9 Typical Examples As,・Sum calculation Xsum=Σi
X 1・Inner product calculation <x, y>=Σ1x1yt・Maximum value search Xmax=max(xih=1y2+−-)
・Maximum value search Xmin=mix(x * l i=
l e 2 +...). However, Σ1 is a sum regarding i. Here, these are collectively referred to as "summation type calculations."

Cascade Sum is a typical method for performing summation calculations on parallel computers.
"Parallel Computer" (Kyoritsu Shuppan (1984), p.

203-207). This is a method of calculating the summation using the following steps. That is, (1) Take the partial sum of each two pieces of data. Next, calculate the partial sums again for each two partial sums.

Hereafter, by repeating the operation (■), the summation can be obtained.

According to cascade sum, the sum of N data is n
It can be processed by step operations (n = l
1ogzN l↑, 1...l↑ is the symbol for rounding up,
A step is an operation that takes a pair of partial sums).

Therefore, compared to the case where everything is processed sequentially, the calculation time is n
It is efficient because it can be shortened to /N (/ is the symbol for division).

For example, when N=210=1024 units.

This means that the calculation time has been reduced to 0.98%.

[Problem that the invention seeks to solve]

However, since the cascade sum repeats the operation of taking two partial sums, many unused PEs occur.

That is, in the first step, 1/2 of the PEs do not work,
In the second step, 3/4 of the PEs do not work, and in the third step, 7/8 of the PEs do not work.
This situation arises, and the operating rate of the PE becomes (N-1)/(nxN) (xx is the symbol for multiplication). For example, N=210=
When there are 1024 PEs, the PE operating rate is only 9.99%.

In numerical simulations, summation-type calculations often appear, so it is desirable to improve the utilization rate.

[Means and operations for solving problems] In real problems, multiple summation-type calculations are often required at the same time.

The present invention aims to improve the operating rate of PE by simultaneously performing a plurality of summation-type calculations.

For example, when calculating the sum of two sets X 811111 :ΣrXi and Ysum=Σiyt, pair the data of neighboring PEs and use the PE with the larger number to calculate Xsum=
Find the partial sum of Σ1x1. On the other hand, start with the data of the same neighboring PEs as a pair and always use the PE with the smaller number to find Ysum=Σtyt. If you do this, the calculation to find X sum and the calculation to find Y sum will be different. can proceed simultaneously.

〔Example〕

An embodiment of the present invention will be described below with reference to FIGS. 1 to 3. The third rM is a configuration diagram of a matrix crossbar switch coupling type parallel computer to which the processing method of the present invention is applied.

The host computer 1, via the array controller 2,
It loads programs to PE3, transfers data, performs scalar calculation processing, and manages the progress of calculations. In addition to the above functions, the array controller 2 controls data transfer from the PE 3 to the host computer 1 and data transfer from the PE 3 to the peripheral device 7 (for example, writing to a magnetic disk). The PE3 is an element processor that performs numerical calculations and transmits and receives data to the PE, and also has a built-in storage device. PE is horizontally L = 2AQ level, vertically M = 2A
m units, total N = L umbrellas M units are lined up (' is the symbol for cross power,
Q and m are positive integers). FIG. 3 shows an example of the configuration of a parallel computer consisting of 16 PEs with Q=m=2. Each PE is identified by a two-dimensionally assigned number (it j).

However, i=1.2.・・・・・・e L, j=1−+
2...

M. The loss bar switch 4 is a data transfer path between PEs arranged in the horizontal direction, has an input port and an output port for each channel, and is highly parallel. That is, L PEs belonging to the same row can each receive data at the same time.

However, one PE cannot receive data from two or more power points at the same time. The column crossbar switch 5 is a data transfer path between PEs arranged in the vertical direction, and has a degree of parallelism M. That is, M PEs belonging to the same column can each receive and receive data at the same time. The cluster memory 6 is an external storage device shared by each vertically arranged P'E3. The peripheral device 7 is an input/output device, an external storage device, etc.

The parallel computer is used to refer to the data stored in the storage device associated with each PE. We will explain how to perform a pair of summation calculations in parallel. The data stored in the storage device associated with PR(+, a) is x(t*JLy(ita)
It is written as Each PE has at most 1 XpV. If P
If E has multiple X or y, for each PE,
Find the partial sum of the data that your PE owns, and do it again! (1, J).

Just put it as y(i, a). Also, P E (iyJ)
If there is no data, it is assumed that there is data with a value of O.

Xsum=:ΣtJx(tta)=ΣJ(Σ,X(,,
, )), first calculate the sum for i and firstly calculate the sum for j. ΣIJ is a double cloth regarding i and j. The sum for i is the sum of the horizontal rows of PEs and is called the "rent". The sum with respect to j is called the "death phase".

To collect rent, data is first transferred horizontally. PEs with odd number i send data X to the PE on the right, and PEs with even number i send data X to the PE on the left. That is, P E
(x, J) becomes P E (z, -)! (1, J), P E (sea) sends X (Jl
, J). P E (z, J) is PE (1e
Send y (aea) to a), P E (4? J) is P
Send y(aea) to E (8,)). However, j covers all of 1, 2, 3, and 4.

■First step in the horizontal direction...The PE whose i is an even number receives and takes the PE data X on the left, adds it to its own PE's own X, and sends the data to the PR two places on the right. send. The PE whose i is an odd number receives and takes y from the PE on the right, adds it to y that it owns, and sends the data to the PE two places on the left.

That is, P E (xtJ) is! (1#J) is received,
Calculate x (zea) + x (zwJ) and assign the value to x (z, a). PE(ztJ) is! (21
Send the value of J) to PE (4hJ) PE with i=3.4
The same goes for.

PE(1,J) takes y(2,)) and -y(1,J
)+y(zta) is calculated and the value is assigned to y(tea).

PE(3,J) is the value of X(IIIJ) PE(teJ)
send to

The same goes for PE with i=3.4. However, j is 1.
Covers all 2, 3, and 4.

■Second horizontal step...Similar to (■) above, X is sent to the right PE and added, and y is sent to the left PE and added. That is, PE(4,J) becomes X(2,a)+X(
4. Calculate J) and find the value! Substitute into (4, J).

P E (xtJ) is y (1, a) + y (ay
Calculate a) and assign the value to y(LtJ). However, j covers all of 1, 2 degrees and 3.4 degrees.

In this way, the rent related to X is found on the PE with i=4, and the rent related to y is found on the PE with i=1. Next, i=4. By taking the death phase of the data of i=1,
In order to determine um and Ysum, data is first transferred.

X(4,1) to P E (4tz), X(4,11)
to P E (414), and y(t, z) to P E (t
t), and sends y(tta) to P E (z, a).

The following two steps are necessary to find the phase of death.6 ■Vertical row 1st step...P E (a e
z) receives X(4,1) x (a,z) + x
(a, z) is calculated and the value is assigned to x(a, z). far. P E (494) receives X (number, 8) and calculates x (a, -a) + x (494),
Assign that value to x(ttt). P E (4+z)
sends x(apt) to P E (a, a).

Similarly, PE (engineering, i) and PE (x, s) are also y
P R(zta) is P
Send y(aya) to E(t,z).

■Vertical row 2nd step...P E (414>
is x(a,z)+! Calculate (4, 4) and find the value!
(414). Also, P E (x, 1) is y
(Eng ps) + y (Eng, 8) is calculated, and the value is y
(xtt).

This is how I found it! (414) is X sun
and y(t, t) is Y sum.

The data transfer order in the above calculation is shown in the data path diagram of FIG. The 16 PEs marked with 0 in the same figure are
The PEs of the parallel computer shown in FIG. 3 are rearranged one-dimensionally vertically at the left end in order to make it easier to display the data paths. Also, in the same figure, the O mark is data, and the → mark is X (ieJ).
Data path 1081. The > symbol represents the data path regarding y (tel).

In the case of L=M=4, it is as above, but in the general case, in Q+m steps, Xsum tY su
+a can be obtained at the same time. That is, the sum Xsum
To find =Σ1JX(itJ), first, for each row,
Take the cascade thumb to the right and create the η phase, ie, X i ”ΣlXC1tJ), on the rightmost PE.

However, j=1, 2y...2M. Regarding the data (η phase Xt) in M PEs on the th column,
By taking the cascade sum in the upward direction, we obtain the summation Xsum=ΣjX J on the top PE (L,M).

At the same time as the above summation calculation, the summation Ys of data y(t, a)
Find um. To do this, first take a cascade sum to the left for each row, and write the rent, that is, Y, on the general left PE.
Create t = Σty(isJ). However, j=1.2.
...It is 2M. By taking the cascade sum downward for the η phase found for M PEs on the first row, the sum Ys is added to the bottom PE (z, t).
um=! Obtain ΣJY□.

The above processing procedure can be expressed uniformly as a program common to all PEs. FIG. 2 is an example written in a pseudo-Fortran language for parallel computers. First, subprograms and variables will be explained. In the figure, MYNUMB (I, J) of sentence number 0010 is a subprogram for calculating the own PE number, and ■ is given a horizontal number, and J is given a vertical number. Sentence number 0030 other 5EN
D(X, (I, J)) is data X3PE(I, J)
This is a subprogram sent to 1 statement number 0060 and other REs.
CEIV (X, (I, J)) is a subprogram that receives data from PE (I, J) and assigns it to X. Sentence number 0040 and others (71FUNCO(I, INA
X) is ■2,4. -..., is a function subprogram that calculates the number of times the remainder is O when divided by IMAX, and this value is calculated when the cascade sum is taken in ascending order.
0 statement number 0120 etc. + 7) FUNCI (I, INAX
) is 2,4.・Old...This is a function subprogram that calculates the number of times the remainder is 1 when divided by
E gives the number of steps to be performed. Also, the meaning of the variables is IMAX and JMAX, respectively, in the horizontal direction.

This is the number of PEs in the vertical direction.

Next, an overview of the processing will be explained. First, the number of the own PE is obtained using MYNUMB (statement number 0010).

Based on FIG. 3 (similar to the embodiment), the cascade thumb is performed in the horizontal direction and then in the vertical direction. Lateral cascading thumb is
When the number of sentences is an even number, processing of sentence numbers 0030 to 0090 is performed. In other words, send data y to the PE on the left (statement number 00
30) - Receive data X from the left PE and add it to X with N.

Repeat a number of times (statement numbers 0050 to OO80). After each PE finishes the last addition, it sends the value to the PE on the right (
Sentence number 0090). However, it is not sent when the PE is at the rightmost position. When all the PEs perform the above processing simultaneously, the η phase is found on the rightmost PE. On the other hand, when ■ is an odd number,
According to the processing of statement numbers 0110 to 0170, cascade sum is performed on y in descending order, and finally the η phase is found on the leftmost PE.

Similarly, regarding the vertical sum, PE of I=IMAX=L
performs the cascade sum of X in ascending order, and PE of 1=1
performs the cascade sum of y in descending order, Xsum is determined as X(L,M), and Ysum is determined as X(1, 1).

The data transfer for these two summation calculations is as follows. In the Q-th step, L/2°L/2" to the right, -
-・-・-, 1 piece, left hemo L/2, L/
22, . . ., one piece, and there is no vertical transfer.

The parallelism of data transfer in the horizontal direction is L, and since the PEs that receive and receive data are always different, data can be transferred at the same time. In steps Ω+1 to Q+m, there is no data transfer in the horizontal direction, and there is no data transfer in the upward direction M/2. M/2",...
River, 1 piece, downward M/2° M/22.・・・・・・
, and these can also be transferred at the same time. Also,
The PE used to add X and the PE used to add y are always different. From the above, two summation calculations regarding X and y can be executed simultaneously.

〔Effect of the invention〕

In this way, the PE required to calculate the sum of x demons is 7
Since it does not overlap with the PE required to calculate the sum of x,
The calculation for determining m and the calculation for determining Y sum can be performed in parallel. The method of the present invention improves the PE utilization rate by about twice as compared to the conventional method that performs a single cascade sum. For example, when N=2''0=1024 units, the PE operating rate improves to 19.9%.

The essence of the present invention is to pair two summation-type calculations and classify them so that the PEs that process each are different. Therefore, it can be applied to a combination of different types of calculations such as summation and inner product, and it is efficient even when applied to a parallel computer with a combination method different from that shown in FIG. These applications will be described below.

[Modified example]

1. Not only can you calculate the sum, but you can also calculate other operations in pairs. For example, inner product <x, x> and inner product <XI y>,
Alternatively, the maximum value Xmax and the minimum value Xmin can be determined simultaneously. This combination is summation and dot product, dot product and maximum (small) value.

It may be a different product such as a maximum (small) value and an inner product.

2. The present invention can also be easily applied to parallel computers using other coupling methods. For example, the lattice-coupled parallel computer illustrated in FIG. 4 is also capable of performing paired calculations using the same data path as the matrix-crossbar-switch-coupled parallel computer shown in FIG. However, in a lattice-coupled parallel computer, data can only be directly transferred to PEs that are vertically and horizontally adjacent, and data must be sent via intermediate PEs in order to send data to non-adjacent PEs. Therefore, at the same time as the data transfer from PE(1-J) to PE(1', J').

When data transfer from P E (t', , +) to P E (i, J) is required, one data transfer may be forced to wait. However, since the PEs required for an operation never overlap, the operations can always be executed in parallel.

3. In addition, calculations according to the method of the present invention are also possible on parallel computers using the hypercube combination method. For example, an example of an execution method on a parallel computer using a hypercube combination method consisting of eight PEs shown in FIG. 5 will be described. In the same figure, P
The three-digit number attached to E is the PE number in binary representation. The processor numbers (i, j
) is expressed one-dimensionally using the conversion rule n = i + Q umbrella (j-1), and is made to correspond to the PE number of the parallel computer using the hypercube combination method shown in FIG. By doing so, it is possible to perform paired calculations, similar to the parallel computer shown in FIG.

As in the previous example, let us consider the case of finding two sums Xsum and YSum. The data path for one operation, X sum, is as follows.

■First step: For all PEs, select the first bit (here, the bit positions are 1st, 2nd, etc. from the right).
...) is 0 to PE whose first bit is 1.
The latter PE calculates the partial sum.

■Second step... Regarding the PE whose first bit is 1, from the PE whose second bit is 0 to the PE whose second bit is 1.
The latter PE calculates the partial sum.

■Third step: Regarding the PE whose first and second bits are 1, data is sent from the PE whose third bit is 0 to the PE whose third bit is 1, and the latter PE calculates the partial sum. do.

In this example, the number of PEs is 8 = 28, so in the third step, the sum is found on the PE with the largest number 111 (number 8 in decimal representation of numbers 1 to 8), and the calculation ends. do.

On the other hand, in each step of the calculation to obtain Y sum,
Data is sent from the PE with a bit of 1 to the PE with a bit of 0 to proceed.

(1) First step: For all PEs, data is sent from the PE whose first bit is 1 to the PE whose first bit is O, and the latter PE calculates a partial sum.

■Second step... Regarding the PE whose first bit is 0, from the PE whose second bit is 1 to the PE whose second bit is O
The latter PE calculates the partial sum.

■Third step... Regarding the PE whose first and second bits are O, data is sent from the PE whose third bit is 1 to the PE whose third bit is O, and the latter PE calculates the partial sum. calculate.

In this way, the sum total is found on the PE with the smallest number 000 (number 1 in decimal representation of numbers 1 to 8).

In the first step, a pair of PEs, e.g.
PE No. E and PE No. 001 send data X and data y to each other, but since there is only one communication path, communication with one of them has to wait. After the second step, there is no interference in the data transmission/reception relationship, and both arithmetic and communication operations can be processed in parallel.

As explained above, the present invention allows the parallel computer shown in FIG. 3 to execute two types of summation type calculations completely in parallel. In the parallel computers shown in FIGS. 4 and 5, although a part of communication cannot be processed in parallel, calculations can be processed completely in parallel, and efficiency can be improved accordingly.

[Brief explanation of the drawing]

Fig. 1 is a data path diagram for calculations using the present invention, Fig. 2 is a diagram showing an example of a program for summation calculation by a parallel computer, Fig. 3 is a configuration diagram of a parallel computer using a matrix crossbar switch combination method, and Fig. 4 The figure shows the configuration of a lattice-coupled parallel computer.
FIG. 5 is a block diagram of a parallel computer using the hypercube combination method. 1...Host computer, 2...Array controller. 3... Element processor, 4... Lossbar switch to perform, 5... Column crossbar switch, 6... Cluster memo 2nd recess 0010 C^shishi ζY Lily M
R (T, J) C-----One-way phrase cascade-y sumuuuuuuuuuu--C-----one word j phrase cascade nameーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
I¥1

Claims (1)

[Claims] 1. Sequential computation that refers to data distributed and stored in storage devices attached to allotropic processors, using a parallel computer consisting of multiple element processors and capable of simultaneous communication between multiple sets of non-adjacent element processors. When processing is performed in a cascade type, a plurality of sets of sequential arithmetic processes are simultaneously executed by using an element processor that becomes vacant in the middle of an operation to perform other sequential arithmetic processes in a cascade type. Cascade parallel processing method.