JP2000020482A - Loop parallelizing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a loop parallelizing method capable of performing the parallelizing, which is conventionally difficult, of a loop in which data dependency exists over loop repetitions. SOLUTION: Concerning the multiplex loop in which the data dependency exists over the loop repetitions, a pipeline paralleligation converting part 15 generates the loop for issuing barrier synchronism before and after that multiplex loop and generates a sentence for issuing the barrier synchronism before and after the divided loop. Thus, the multiplex loop can be parallelly executed by a multiprocessor in the manner of pipeline.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a loop parallelization method using barrier synchronization, and more particularly to a loop parallelization method for a compiler suitable for use in generating a target code for a multiprocessor having a barrier synchronization mechanism. About the method of conversion.

[0002]

2. Description of the Related Art As a conventional loop parallelization technique for generating an objective code for efficiently executing parallel execution on a shared memory type multiprocessor utilizing barrier synchronization as synchronization control, for example, "Super Compiler "(co-authored by Hans Zima and Barbara Chapman, translated by Yoichi Muraoka,
Techniques described in Ohmsha, p. 276-p. 297 are known.

[0003] Loop parallelization according to the prior art is to parallelize a DOALL loop in which there is no data dependency (hereinafter referred to as loop carrying dependency) over loop iterations. Then, in the related art, when there is a dependency between the parallelized DOALL loops,
Inserting a statement that executes barrier synchronization between DOALL loops ensures that parallel execution is performed correctly.

FIG. 7 is a diagram for explaining a configuration and an execution image of a conventional multiprocessor capable of executing object codes parallelized by the loop parallelization method according to the present invention in parallel. The parallel execution of the object code will be described with reference to FIG.

The multiprocessor shown in FIG. 7 functions as a single node, and is composed of one master processor 61 and a plurality of, for example, eight slave processors 62, and each processor individually has a cache. In addition, all processors share memory. The multiprocessor shown in FIG. 7 has a configuration in which the S
A parallel execution method called an ID-SMP method is adopted. Hereinafter, this method will be described.

In FIG. 7, a program is first executed on a master processor 61. At the same time, at the beginning of the program execution, the master processor 61 uses the program executed on the master processor 61 as a parent thread and generates child threads for the number of slave processors.

The generated child thread is fixedly allocated to each slave processor 62 until the program ends. Therefore, overhead for thread generation and reduction occurs only at the beginning and end of a program. The program is divided into a serial execution part and a parallel execution part in advance by a compiler. The sequential execution part is
It is executed only by the master processor 61, and during the execution of this part, the other processors execute a spin loop waiting for execution. When the execution of the program reaches the parallel execution part, all the slave processors 62 together with the master processor 61 start executing the parallel execution part. When the execution of the parallel execution portions of all the processors is completed, the master processor 61 starts the execution of the subsequent sequential execution portion, and the slave processor 61 enters the spin loop waiting for execution again.

In the above-mentioned method, a plurality of child threads are fixedly waited on a specific processor to minimize the overhead time for starting and terminating parallel execution. is there.

In the above description, there is a case where it is desired to synchronize the processors during the parallel execution of the slave processors. At this time, generally, barrier synchronization is used as a synchronization method. Barrier synchronization is a method for achieving synchronization by waiting for all processors to reach the synchronization point before all processors have reached the synchronization point, and is realized by dedicated hardware instructions. Can be.

On the other hand, what the automatic parallelizing compiler targets for parallelization is a loop. The loop to be parallelized is not an arbitrary loop, but a loop whose loop length can be calculated before execution of the loop. For example, FORTRA
An N DO loop corresponds to this. The compiler analyzes these loops, determines whether they can be executed in parallel, and divides the iterations of the loops that are judged to be parallel executable, and executes the divided loops by each processor. Generate code that does

How to divide an iteration of a loop executed in parallel and allocate it to each processor is called a scheduling problem in a multiprocessor. Such scheduling methods include two types, a dynamic scheduling method and a static scheduling method.
One can be considered.

The dynamic scheduling method has an advantage that the processor to be allocated is dynamically determined at the time of execution, so that the load of each processor can be made close to equal. However, the overhead for allocating the processors at the time of execution is increased. Tends to be too large. On the other hand, the static scheduling method has an advantage that the overhead at the time of execution can be reduced because the compiler statically determines the processor to which the allocation is made. In particular, D
It is considered that the parallelization of the O loop can often make the load close to even by simple division, and the multiprocessor described in FIG. 7 also employs a static division method by a compiler as a scheduling method.

FIG. 8 is a diagram showing an example of a DO loop that can be parallelized and a code obtained by converting the DO loop for parallel execution by static division. This will be described below.

As a method of statically dividing the DO loop, various division methods are known, but a method of performing block division is basically used. In each of the DO loops divided into blocks, one loop is executed by each processor as a continuous iteration corresponding to the block length. The block length can be determined by dividing the loop length by the total number of processors.

The calculation of T in the loop after parallelization shown in FIG. 8 determines the block length. In this formula, T is the block length, PROC_PN is the total number of processors, and CELI is an instruction to round up when it cannot be divided. Since each processor has its own identification number, PROC_ID in the example of FIG. 8, each processor can calculate the lower limit L and the upper limit U of its own iteration from the processor number and the block length.

The code for parallel execution as shown in FIG. 8 is loaded on each processor and the same code is executed in parallel. However, variables referred to therein are divided into global variables shared by all processors and local variables assigned to an area unique to each processor. Basically, variables appearing in the original program are global, and variables generated after parallelization, such as T, L, U, are local. The loop control variable, D
Variables controlling the execution of the O loop, I in the example of FIG.
Becomes a local variable during parallelization. Otherwise, writes from different processors will occur in the same area. In the example shown in FIG. 8, the loop control variable after parallelization is set to% I to distinguish it from the original loop control variable.

[0017]

Generally, in order to improve the execution performance of a multiprocessor, a technique for efficiently extracting parallelism from multiple loops is important. But,
The above-described conventional technique using barrier synchronization is that if there is no DOALL loop in a multiplex loop, it is difficult to perform parallelization unless a DOALL loop can be created using a loop conversion technique such as loop distribution. Has problems.

An object of the present invention is to solve the above-mentioned problems of the prior art, and to perform parallelization using barrier synchronization even in a multiple loop that does not include a DOALL loop, if parallelism can be extracted. It is an object of the present invention to provide a loop parallelizing method in a compiler which is capable of generating a target code capable of obtaining efficient execution performance on a multiprocessor.

[0019]

According to the present invention, there is provided a method for parallelizing a loop in a compiler for translating a high-level language for generating an object code to be provided to a shared memory type multiprocessor having a barrier synchronization mechanism. In, for any multiple loops that have a data dependency over the loop iteration, any one of the multiple loops is divided, and the divided loops are executed in parallel in a pipeline manner. This is achieved by generating a loop for issuing barrier synchronization before and after the multiplex loop, and generating a statement for issuing barrier synchronization immediately after the divided loop.

Further, the above object is achieved by blocking the outer loop when the divided loop is the inner loop, and blocking the inner loop when the divided loop is the outer loop.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a loop parallelizing method using barrier synchronization according to the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a diagram showing the configuration of a compiler for performing loop parallelization according to the present invention, FIG. 2 is a flowchart for explaining the operation of the compiler shown in FIG. 1, and FIG. 3 is FORTRA.
FIG. 4 is a diagram showing an example of a DO loop described in N languages, FIG. 4 is a diagram for explaining a DO loop after performing parallel division of the DO loop shown in FIG. 3, and FIG. 5 is barrier synchronization with the DO loop shown in FIG. FIG. 6 is a diagram for explaining a DO loop after inserting a sentence, and FIG. 6 is a diagram for explaining a DO loop after applying blocking to the DO loop shown in FIG. In FIG. 1, 11 is a source program, 12 is a compiler, 13 is a parallelization conversion unit,
Reference numeral 14 denotes a DOALL type parallelization conversion unit, 15 denotes a pipeline type parallelization conversion unit, 16 denotes a target code generation unit, 17 denotes a target program, and 18 denotes a program storage medium.

The compiler 12 for performing loop parallelization according to the present invention receives a source program 11 described in a serial type high-level language such as FORTRAN as an input, generates a target code executable on a multiprocessor, and generates the target program 17 as a target program 17. It is output and comprises a parallelization conversion unit 13 and a target code generation unit 16. The parallelization conversion unit 13 has a function of performing parallelization determination and conversion, recognizes loops in the program, determines whether or not each of these loops can be parallelized, and performs parallelization. When the determination is made, the input source program is converted into a loop that can be executed in parallel.

The parallelization conversion unit 13 is a conventional DOALL
It is composed of a DOALL type parallelization conversion unit 14 for performing parallelization of the type and a pipeline type parallelization conversion unit 15 for performing the pipeline type parallelization according to the present invention. Then, the parallelization conversion unit 13 first attempts DOALL-type parallelization for each loop of the input source program, and attempts to apply pipeline-type parallelization to a loop to which this cannot be applied.

The compiler program for executing the processing in the compiler 12 is stored in the recording medium 18.

Next, the processing operation in the compiler 12 will be described with reference to the flow shown in FIG.

(1) Data dependency analysis is performed for each loop nest in the input source program (step 201).

(2) It is checked whether or not DOALL type parallelization can be applied based on data dependency analysis, and if DOALL type parallelization can be applied, the DOALL type parallelization conversion unit 14
(Steps 202 and 203).

(3) If it is determined in step 202 that the DOALL type parallelization is not applicable, it is checked whether the pipeline type parallelization is applicable, and the pipeline type parallelization is applicable. In this case, the parallelization conversion unit 15 is caused to execute parallelization (steps 204 and 20).
5).

(4) If it is determined in step 204 that the pipeline type parallelization is not applicable, the parallelization is not performed, and the process returns to step 201 and the processing from the analysis of the next loop nest is performed. Continue. Note that loops that are not parallelized are sequentially executed (step 20).
6).

Next, FIG. 3 shows an example of a source program.
Considering the program shown in (1), an outline of pipeline type parallelization will be described.

The example shown in FIG. 3 is a DO loop described in the FORTRAN language. This DO loop has an array A for both the outer loop 21 and the inner loop 22.
There is a loop-carrying dependency on Such a loop is first divided into inner loops and assigned to each processor on the multiprocessor. P for processor number
1, P2,..., Pn, the iteration [1: N] of the inner loop is divided and the processor P
, P2,..., Pn.

At this time, the iteration of the inner loop assigned to each processor is defined as [Lk: Uk] (where Lk is the lower limit of the iteration and Uk is the upper limit of the iteration) with respect to the processor Pk. I do. When the outer loop iteration [1: N] is taken into consideration, each processor eventually executes the iteration of [1: N, Lk: Uk] when the processor number is Pk.

Here, the iteration of the outer loop is represented by M
Since there is a dependency from the iteration [M, Lk: Uk] to the iteration [M, Lk + 1: Uk + 1], the execution of the iteration [M, Lk: Uk] must be completed. Iteration [M, Lk +
1: Uk + 1] cannot be started. Conversely, there are no other dependencies between iterations assigned to different processors. In such a case, pipeline parallelization can be applied.

The pipeline-like parallel processing using barrier synchronization is performed by starting the execution of the loop in ascending order of processor number. First, the processor P1 starts executing the loop, and the processor P1
Issues a barrier synchronization when the first iteration of the outer loop is completed, and the processor P2 starts executing the loop. Next, when the processor P1 finishes executing the second iteration of the outer loop and the processor P2 finishes executing the first iteration of the outer loop, the processor P3 issues a barrier synchronization, and the processor P3 starts executing the loop. Start.

In the same manner, the processors sequentially start executing the loop while synchronizing with each other by the barrier synchronization, and the number of processors executing the loop increases one by one. You will be Also at this time, the barrier synchronization is issued every time one iteration of the outer loop is completed. The end of the execution of the loop is inevitably completed from the one with the smallest processor number, similarly to the start of the loop. After the processor P1 finishes the execution of the loop, the number of processors that execute the loop decreases one by one. , All the processors finish executing the loop.

In the embodiment of the present invention, in order to realize pipeline-like parallelization using barrier synchronization as described above, an object code is generated according to the following policy. That is, first, a loop for issuing (processor number-1) barrier synchronizations immediately before the loop execution is generated. Next, the inner loop is divided and assigned to each processor, and a statement for issuing barrier synchronization immediately after the divided loop is generated. Finally, a loop for issuing (the total number of processors−the processor number) times of barrier synchronizations immediately after the execution of the loop is generated.

In the pipeline parallelization in the above description, one stage of the pipeline, that is, execution during issuance of barrier synchronization corresponds to one iteration of the outer loop, but the outer loop length is sufficiently long. When known or predicted, the number of barrier synchronizations issued can be reduced by performing multiple iterations of the outer loop in one stage of the pipeline. This is called blocking, and it is possible to reduce the number of times barrier synchronization is issued, and it can be expected that execution time will be reduced.

As described above, the FORTR shown in FIG.
The DOALL described in the AN program cannot be applied to the DOALL type parallelization because the outer loop and the inner loop both have loop carrying dependency. Therefore, pipeline type parallelization is applied. D obtained by dividing the DO loop shown in FIG.
The O-loop is shown in FIG. 4 and will be described below.

First, the iterations of the inner loop are equally divided by the number of processors on the multiprocessor and are sequentially allocated to each processor. B is the loop length of the inner loop allocated to each processor, and P is the total number of processors.
If N, then B = [N / PN]. here,[ ]
Indicates that if not divisible, it is rounded up to an integer. If the lower limit of the iteration of the inner loop executed by each loop is L and the upper limit is U, L = B ×
(P-1) +1, U = MIN (L + B-1, N). In this equation, P is the processor number (P = 1,
2,..., PN).

According to the above-described conversion, each processor:
It becomes possible to calculate the upper and lower limits of the loop from its own processor number and execute the loop assigned to itself.
In order to calculate the loop assigned to itself, the DO loop given to each processor includes the above-described formula 31 for calculating the loop length of the divided loop, the formula 32 for calculating the lower limit value of the divided loop, and the formula for calculating the upper limit value of the divided loop. 33 and information on the divided inner loop 34.

Further, a barrier synchronization statement for issuing barrier synchronization is inserted in order for each processor to execute the divided loop shown in FIG. 4 in parallel in a pipeline manner. As shown in FIG. 5, the barrier synchronization statement includes a pro-row group 41 by the barrier synchronization inserted immediately before the loop shown in FIG. 4, and an epi-row group 43 by the barrier synchronization inserted immediately after the loop.
And a barrier synchronization statement 42 inserted at a position immediately after the divided loop. Pro Law Group 41
Is repeated (P-1) times and the epi-row group 43
Is repeated (PN-P) times.

As a result, each processor on the multiprocessor executes the pro row group 41 until the processor with the processor number one younger than the own processor number completes one divided DO loop. Of the DO loop is started. Then, the multiprocessor starts the processing in order from the youngest processor, and all the processors can cooperate and perform pipeline-like parallel execution. In addition, the processing of the DO loop in each processor ends in order from the youngest processor, but by executing the epi-row group 43, all the processors end the processing at the same time.

In the above-described embodiment of the present invention, the inner loop is divided, and the divided DO loop is allocated to each processor constituting the multiprocessor.
The present invention may be configured such that the outer loop is divided, and the divided DO loop is assigned to each processor constituting the multiprocessor.

In the loop allocated to each processor according to the above-described embodiment, the number of times the barrier synchronization statement 42 is issued in a multiplex loop is N times, which is equal to the outer loop length. However, blocking is applied to the outer loop. By doing so, the number of executions of barrier synchronization can be reduced.

An example of a loop that can be obtained by applying blocking to the outer loop is shown in FIG. This loop is obtained by converting the loop shown in FIG. 5 into a double loop including a loop 51 and a loop 52 by blocking the outer loop. The loop 52 is a loop having a blocked loop length T, and a loop 53
Is a control loop for its execution. At this time, the barrier synchronization statement is inserted as a barrier synchronization statement 53 at a position immediately after the blocked loop. As a result, the issue of the barrier synchronization statement 53 is N times when the blocking is not performed, but is changed to N / T by applying the blocking.
Times.

Although the above description has been made on the assumption that the inner loop is divided and the outer loop is blocked, the present invention can also divide the outer loop and block the inner loop, and can obtain the same effect. .

[0048]

As described above, according to the present invention, D
If parallelism can be extracted even for multiple loops that do not include OALL loops, parallelization is performed using barrier synchronization, and object code that can provide efficient execution performance on a multiprocessor is generated. can do.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration of a compiler that performs loop parallelization according to the present invention.

FIG. 2 is a flowchart illustrating the operation of the compiler shown in FIG. 1;

FIG. 3 is a diagram illustrating an example of a DO loop described in a FORTRAN language.

FIG. 4 is a diagram illustrating a DO loop after the DO loop shown in FIG. 3 is divided in parallel.

FIG. 5 is a diagram illustrating a DO loop after inserting a barrier synchronization statement into the DO loop shown in FIG. 4;

FIG. 6 is a diagram illustrating a DO loop after applying blocking to the DO loop shown in FIG. 5;

FIG. 7 is a diagram illustrating a configuration and an execution image of a conventional multiprocessor capable of executing object codes parallelized by the loop parallelization method according to the present invention in parallel.

FIG. 8 is a diagram illustrating an example of a DO loop that can be parallelized and a code obtained by converting the DO loop for parallel execution by static division.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Source program 12 Compiler 13 Parallelization conversion part 14 DOALL type parallelization conversion part 15 Pipeline type parallelization conversion part 16 Object code generation part 17 Object program 18 Program storage medium

Claims (3)

[Claims] 1. A loop parallelizing method in a compiler for translating a high-level language for generating an object code to be provided to a shared memory type multiprocessor having a barrier synchronization mechanism, wherein a data dependency over a loop iteration exists. Generates a loop that issues barrier synchronization before and after the split multiplex loop in order to split any of the multiplex loops and execute the split loops in parallel in a pipeline manner. And generating a statement for issuing barrier synchronization immediately after the divided loop. 2. The loop according to claim 1, wherein if the divided loop is an inner loop, the outer loop is blocked, and if the divided loop is an outer loop, the inner loop is blocked. Parallelization method. 3. A recording medium storing a program for realizing the loop parallelization method according to claim 1.