JP2001306333A - Loop parallelizing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output a program or an object code to shorten parallel execution time of loops consisting of sentences to be included in a dependence cycle and sentences not to be included in the dependence cycle. SOLUTION: In this loop parallelizing method to output a parallelized program 16 or the object code by inputting a program 14 for computer including a loop processing, a processing part 123 to separate a loop main body, a processing part 125 to schedule the loops consisting of the sentences to be included in the dependence cycle to each processor so that they are successively executed in an execution order and a processing part 126 to schedule the loops consisting of the sentences not to be included in the dependence cycle so no waiting time for synchronization by the dependence cycle becomes idle by every processor are provided.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a loop parallelizing method for dividing a computer program into parallel loops of a program executable by a parallel computer, and more particularly, to a statement included in a dependent cycle and a dependent cycle. The present invention relates to a loop parallelizing method capable of reducing the execution time of a loop of a parallel computer by inputting a computer program having a loop including a statement that is not included and outputting a parallelized program or object code.

[0002]

2. Description of the Related Art As a conventional technique, a conventional parallelizing compiler converts a program into a parallel computer having a plurality of processors by dividing a loop range of a loop appearing in the program and assigning it to each processor. I do. At this time, in some cases, it is necessary to convert to a program that is executed in parallel while synchronizing between the processors. For example, consider the input program 40 shown in FIG. The program, in each formula of _{S 1, S 2, S 3} , is to exit repeatedly executes the i = 1 to 9. In the input program 40, the value assigned to D (1) when i = 1 is referred to as D (i-1) when i = 2. Cases cannot be executed independently in parallel. As described above, when there is a dependency over the loop repetition, an output program 50 as shown in FIG. 5 is obtained when a program executable on a parallel computer is converted by a conventional technique using a parallelizing compiler. The parallelizing compiler converts the loop range of the do loop of the output program 50 from 1 to n to plb to pub. plb and pub are the lower limit and upper limit of the loop range assigned to each processor by the parallelizing compiler. The do loop executes in parallel for each i, but is not executed completely independently, but is executed with appropriate synchronization.

In this example, the processor that has executed S1 issues a signal indicating this (send_signal), and i
Wait for a signal from the executing processor for -1 (wait_signal), and then execute S1. FIG. 14 is a diagram illustrating a state of the parallel execution of the output program 50. As shown in FIG. 14, it is necessary to execute while synchronizing (indicated by an arrow) for using the value defined in the immediately preceding loop iteration. In FIG. 14, unnecessary synchronization in the same processor is removed, and only synchronization between different processors is shown. Here, D (i) = D (i-1) +1 and A (i) = C
In each formula of (i) / 2 + C (i-1) * 3, i = 3 to i = 4, and i = 6 to i
It is necessary to send a signal to another processor when moving to = 7. The parallelization technology described above is based on Hans Zima, Barbara
Chapman, "Supercompilers for Parallel and Vecto
r Computers ", ACM Press, 1990.

[0004]

As described above, according to the above-mentioned prior art, when a program including a loop process is input and a parallelized program is executed on a plurality of processors, the dependency over the loop repetition is obtained. In a certain loop, when the output program 50 as shown in FIG.
I = 1 for three processors PE1, PE2, PE3
In the case of division into 33,4〜6,7〜9, as shown in FIG. 14, after PE2, the synchronization wait state until the signal indicated by the arrow is received, that is, wasted time. Since the synchronization waiting time is idle, it takes a long time to execute.

An object of the present invention is to solve the conventional problems and to provide a loop parallelizing method capable of outputting a program or an object code for reducing the loop execution time of a parallel computer. .

[0006]

In order to achieve the above object, in the loop parallelizing method of the present invention, a loop including a statement included in a dependent cycle and a loop including a statement not included in a dependent cycle is used. The main body is divided, and each divided loop is executed in a different order for each processor.
Specifically, the process is performed in a well-ordered manner by the first to seventh steps. In the first step, the data dependence on the loop is analyzed. In the second step, a parallelization loop is determined. In a third step, in the parallelization loop:
Insert a synchronization statement for a statement that depends on the loop iteration. In the fourth step, a loop body of a loop including a dependent statement that extends over loop iterations is divided from the parallelized loop. In a fifth step, the loop range of the parallelized loop is divided and assigned to each processor. In the sixth step, among the loops obtained by dividing the loop body, a loop including a statement included in a dependent cycle is scheduled to be executed by each processor in a sequential execution order. In the seventh step, for each processor, among the loops obtained by dividing the loop main body, the loop including a statement that is not included in the dependency cycle is kept in a dependency relationship so that the waiting time for synchronization is not idle. Schedule. According to the loop parallelizing method of the present invention, an output program 60 as shown in FIG. 6 is generated. As shown in FIG. 6, a loop 1L composed of a statement S1 included in a dependent cycle is scheduled by each processor so as to be executed in a sequential execution order, and a loop 2L composed of a statement S2 not included in a dependent cycle and a statement S3. Is scheduled for each processor such that the synchronization waiting time due to the dependent cycle is not idle.

FIG. 15 is a diagram showing a state of parallel execution on an output program 60 by the loop parallelizing method of the present invention. As shown in FIG. 15, the statements included in the dependent cycles are scheduled to be executed in the sequential execution order in each processor, and for each processor, an individual loop obtained by dividing the loop body is used for synchronization by the dependent cycle. Is scheduled so that it is no longer idle. According to the loop parallelization method of the present invention, the synchronization wait time due to the dependent cycle is not idle for the loop including the statement included in the dependent cycle and the statement not included in the dependent cycle. The output execution time can be reduced.

[0008]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 3 is a configuration diagram of a shared memory type parallel computer according to an embodiment of the present invention. As shown in FIG. 3, the shared memory type parallel computer includes processors 321 to
2n, shared memory 31, and bus 33 connecting them
It is composed of Each of the processors 321 to 32n includes:
At each time T = 1 to 9 shown in FIG. 15, each of them accesses the shared memory 31 and fetches the data of the expression to be operated by itself, and performs the operation. Signals are sent from the processor that computes i = 3 in the equation S1 to the processor that computes i = 4, and from the processor that computes i = 6 in S1 to the processor that computes i = 7. Similarly, the signal from the processor that calculates i = 3 of S2 to the processor that calculates i = 4 of S3 and the signal from the processor that calculates i = 6 of S2 to the processor that calculates i = 7 of S3, respectively. Is sent.

FIG. 1 is a configuration diagram of a parallelizing compiler showing one embodiment of the present invention. As shown in FIG. 1, the parallelizing compiler 10 includes a syntax analyzing unit 11, a loop parallelizing unit 12,
The parallelizing compiler 10 is configured by a code generation unit 13 and is stored in a storage medium such as a magnetic disk device, loaded into a main storage device, and executed. The syntax analysis unit 11 of the parallelizing compiler 10 reads the input program 14 from the storage medium, generates an intermediate language 15, and stores the intermediate language 15 in the storage medium. Although the storage medium storing the input program 14, the intermediate language 15, and the output program 16 is described separately, it is needless to say that the same medium may be used.
The intermediate language 15 is a representation of a program in the parallelizing compiler 10, and its format is not particularly different from that of a normal compiler, and therefore will not be described in detail here. The loop parallelizing unit 12 is a processing unit that detects a loop that can be loop-parallelized using the result of the syntax analysis unit 11, changes the loop structure, and assigns loop processing to each processor. , The parallelization loop determination unit 12
1, a synchronization generation unit 122, a loop body division unit 123, a loop range division unit 124, an inter-processor scheduling unit 125, and an intra-processor scheduling unit 126.

FIG. 2 is a flowchart showing a processing procedure of the loop parallelizing section in FIG. As shown in FIG.
This procedure includes steps 20 to 26. Each of the steps 20 to 26 corresponds to the loop dependence analysis unit 120, the parallelization loop determination unit 121, and the synchronization generation unit 12 in FIG.
2. Loop body dividing unit 123, loop range dividing unit 12
4, corresponding to the processing operations of the inter-processor scheduling unit 125 and the intra-processor scheduling unit 126. Step 20 includes a loop dependency analysis unit 120
To generate a data dependency graph for the loop. Step 21 includes a parallelization loop determination unit 121
And a DOALL loop that can be executed in parallel without synchronization, and a DOACROSS loop that can be executed in parallel with synchronization. Step 22 is executed by the synchronization generation unit 122, and inserts a synchronization statement into the DOACROSS loop into a statement that depends on the loop iteration. Step 23 is executed by the loop body dividing unit 123 to divide the loop body into DOACROSS loops.
For example, in the case of the input program of FIG. 4, the statements are S1, S2,
Since there are three of S3, each statement is divided into three loops. Step 24 is executed by the loop range dividing unit 124, and the loop range is divided and assigned to each processor. In the case of the input program of FIG. 4, i = 1
~ 3, i = 4 ~ 6, i = 7 ~ 9 and divided into processors PE1, PE2, PE3
Assign to

Step 25 is executed by the inter-processor scheduling unit 125 so that a loop including a statement included in the dependent cycle among the loops divided in the loop body in step 23 is executed by each processor in the sequential execution order. Schedule. In the case of the input program of FIG. 4, the first statement 1L is scheduled for three processors in the sequential execution order as shown in FIG. Step 26 is executed by the intra-processor scheduling unit 126, and for each processor, of the loops divided in the loop body in step 23, a loop including a statement that is not included in a dependent cycle is set to a state in which the waiting time for synchronization is idle. Schedule so that dependencies are maintained so that they disappear. In the case of the input program of FIG. 4, as shown in FIG. 13, the second statement 2L and the third statement 3L are scheduled for three processors so that the waiting time is not idle. The code generator 13 reads the intermediate language 15 from the storage medium and generates an output program 16. The generated output program 16 is stored in the storage medium again. Since the contents of these processes are not particularly different from those of a normal compiler, they will not be described in detail here.

FIG. 4 is a diagram of a source image of an input program showing one embodiment of the present invention. The source image of the input program in FIG. 4 illustrates an example of the input program 40 of the intermediate language 15 to the loop parallelizing unit 12 of the parallelizing compiler 10. FIG. 6 is a diagram of a source image of an output program showing one embodiment of the present invention. The source image of the output program in FIG. 6 illustrates an example of the output program 60 from the loop parallelizing unit 12 corresponding to the input program 40 to the intermediate language 15. Hereinafter, an example of loop parallelization by the parallelizing compiler 10 using the input program 40 and the output program 60 will be described. First, the loop dependence analysis unit 120 generates a data dependence graph (see FIG. 8) for the loop L of the input program 40.

FIG. 8 is a diagram of a data dependency graph for the loop L of the input program 40. In FIG. 8, the nodes of the graph represent the sentences S1, S2, S3 in the loop, and the edges of the graph represent the dependencies between the sentences. δ1 means that there is a dependency over the loop repetition from the start point to the end point of the edge at the loop level 1, and δ∞ means that there is a loop-independent dependency from the start point to the end point. Data dependency graph generation technology is provided by Hans Zima, Barbara Chap
man, "Supercompilers for Parallel and Vector Co
mputers ", ACM Press, 1990, which is well known in the art and will not be described in further detail here. When the path of the dependency graph is closed, it is called a cycle, as shown in FIG. The statement included in the dependency cycle is S1, and the statements not included in the dependency cycle are S2 and S3.The parallelization loop determination unit 121 determines the loop L of the input program 40 as a DOACROSS loop. This is because the loop L of the input program 40 has a dependency over the loop repetition, and thus it is necessary to synchronize between different repetitions when the loop repetition is performed in parallel. 122 is a dependent statement S1 in the loop of the input program 40 that extends over loop iterations.
Before and after S2 and immediately before S3.

FIG. 9 is a diagram showing a program obtained as a result of inserting a synchronization statement into the input program 40. FIG.
As shown in the program 90, a statement that receives a signal from the immediately preceding loop iteration is inserted immediately before the statement S1, and a statement that sends a signal immediately after the statement S1 is inserted. Also, a statement that sends a signal immediately after the statement S2 and a statement that receives a signal from the immediately preceding loop iteration are inserted immediately before the statement S3. The loop body dividing unit 123 divides the loop body with respect to the DOACROSS loop, and generates a loop 1L including S1, a loop 2L including S2, and a loop 3L including S3.

FIG. 10 shows a program 9 in which a synchronization statement is inserted.
FIG. 14 is a diagram illustrating a program resulting from dividing a loop body with respect to 0. As shown in the program 100 of FIG.
By dividing the loop body, DOACROSS loop L becomes DOACROSS
Converted to loop 1L, DOALL loop 2L, DOALL loop 3L. The loop range dividing unit 124 includes the parallelized loops 1L, 2L, 3L
Is converted from 1 to n to plb to pub. plb, pu
b is the lower limit and the upper limit of the loop range assigned to each processor by the parallelizing compiler. In addition, the synchronization statement
Move out of the loop statement assigned to each processor. FIG. 11 shows a program 100 obtained by dividing a loop body.
FIG. 14 is a diagram of a program as a result of dividing a loop range. As shown in the program 110 of FIG. 11, the loop range of the parallelized loops 1L, 2L, and 3L was converted from 1 to n to plb to pub, and the synchronization statement was divided into ranges so as to synchronize the processors. Moved out of the loop. The inter-processor scheduling unit 125 schedules the loop 1L including the statement S1 included in the dependent cycle of the program 110 so as to be executed in the sequential execution order.

FIG. 12 is a diagram showing the result of inter-processor scheduling. As shown in FIG. 12, when the horizontal axis indicates the processor number and the vertical axis indicates the execution order, the loop 1L including the statement S1 included in the dependent cycle is executed first by PE1, second by PE2,. Schedule. The in-processor scheduling unit 126 is a program 1
Loop 2L consisting of statement S2 not included in 10 dependent cycles
And a loop 3L composed of the statement S3 and scheduling is performed for each processor while maintaining the dependency so that the waiting time for synchronization is not idle. FIG. 13 is a diagram showing a result of intra-processor scheduling. As shown in FIG. 13, PE2 and PE3 schedule so as to execute a loop consisting of a statement that can precede the synchronization wait time, so that PE2 precedes a loop 1L consisting of statement S1 and a loop 2L consisting of statement S2. , The PE3 schedules to execute the loop 2L including the statement S2 and the loop 3L including the statement S3 prior to the loop 1L including the statement S1. As a result of performing the above processing, an output program 60 shown in FIG. 6 is generated.

FIG. 7 is a source image diagram of an output program showing another embodiment of the present invention. As an example of the source image of the output program, there is an output program 70 as shown in FIG. 7 in addition to FIG. The output program 70 shown in FIG. 7 has the same meaning as the output program 60 shown in FIG.
There is versatility as a (SPMD) model. That is, the processing to be executed is exactly the same as that of FIG. 6, but the description method is different, and a loop composed of the statement S1 is used as a subroutine cycle.
Only 1L is executed in the sequential order, and the loop 2L composed of the statement S2 and the statement
It is stated that the loop 3L composed of S3 is executed in the order of the processor PE1, in the order of the processor PE2, and in the order of the processor PE3. When the number of processors is small,
Although the code amount is not so different between the case of FIG. 6 and the case of FIG. 7, the code amount is considerably different as the number increases. Above,
The description of the example of the loop parallelization using the input program 40 and the output program 60 ends.

If this parallelism is not executed, the synchronization wait time due to the dependent cycle is idle, so that it takes a long time to execute. On the other hand, if the loop parallelization is performed as in the present invention, the dependent cycle Since a statement that can precede the synchronization waiting time is executed and the synchronization waiting time is not idle, the execution of the program can be sped up. It should be noted that the present invention is not limited to the above-described embodiment, and it is needless to say that various changes can be made without departing from the scope of the present invention. For example, it is possible to change synchronization by signal to barrier synchronization, and to apply to a distributed memory type parallel computer.

[0019]

As described above, according to the present invention,
For a loop consisting of a statement included in a dependent cycle and a statement not included in a dependent cycle, the time required for synchronization by the dependent cycle is not idle, so that the execution time for outputting a program or object code can be reduced. is there.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a parallelizing compiler showing one embodiment of the present invention.

FIG. 2 is a flowchart illustrating a processing procedure of a loop parallelizing unit in FIG. 1;

FIG. 3 is a diagram illustrating a configuration example of a shared memory parallel computer to which the present invention is applied;

FIG. 4 is a diagram illustrating an example of a source image of an input program.

FIG. 5 is a diagram illustrating an example of a source image of an output program according to the related art.

FIG. 6 is a diagram showing an example of a source image of an output program according to the present invention.

FIG. 7 is a diagram showing another example of a source image of an output program according to the present invention.

FIG. 8 is a diagram showing a data dependency graph for FIG. 4;

FIG. 9 is a diagram illustrating an example of a source image of a program in which a synchronization statement is inserted into FIG. 4;

FIG. 10 is a diagram illustrating an example of a source image of a program obtained by dividing a loop body from FIG. 9;

11 is a diagram illustrating an example of a source image of a program obtained by dividing a loop range with respect to FIG. 10;

FIG. 12 is a diagram showing an inter-processor schedule for FIG. 11;

FIG. 13 is a diagram showing an in-processor schedule for FIG. 12;

FIG. 14 is a diagram illustrating an example of a state of parallel execution according to the related art.

FIG. 15 is a diagram showing an example of a state of parallel execution showing one embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 10 ... Parallelization compiler, 11 ... Syntax analysis part, 12 ... Loop parallelization part, 13 ... Code generation part, 14 ... Input program, 15 ... Intermediate language, 31 ... Shared memory, 33 ... Bus,
321-32n processor, 16 output program,
40: input program, 50: output program according to the prior art, 60, 70: output program according to the present invention, 8
0: Data dependency graph, 90: Program after insertion of synchronization statement, 100: Program after division of loop body, 110:
Program after loop range division, 120: loop dependence analysis unit, 121: parallelized loop determination unit, 122: synchronization generation unit, 123: loop body division unit, 124: loop range division unit, 125: inter-processor scheduling unit, 12
6. In-processor scheduling unit.

Claims (3)

[Claims] 1. A loop parallelization method for inputting a computer program including a statement for performing a loop process and outputting a parallelized program or object code to be executed on a plurality of processors. When there is a statement included in a cycle and a statement not included in each cycle, each processor reads a plurality of statements including the above two types of statements in the same loop in a different order as compared with any of the other statements as a whole of the processor. A loop parallelization method characterized by executing. 2. A loop parallelization method for inputting a computer program including a statement for performing a loop process and outputting a parallelized program or object code to be executed on a plurality of processors. When there is a sentence included in a cycle and a sentence not included, first, the loop body is divided, then the loop range of the loop including the statement included in the dependent cycle is divided, and then the statement included in the dependent cycle Is assigned to each processor so as to be executed in a sequential execution order. Then, for each processor, another statement that can precede the waiting time for inter-processor synchronization for the loop including the statement included in the dependent cycle is assigned. Loop rearranging method for rearranging so as to execute a loop including the loop 3. A loop parallelization method for inputting a computer program including a statement for performing a loop process and outputting a parallelized program or object code executed on a plurality of processors. Analyzing, a step of determining a parallelized loop as a result of the analysis, a step of inserting a synchronization statement into a statement in the parallelized loop that depends on the loop iteration, and a step of loop iteration Dividing the loop body of the parallelized loop including the dependent statement into several parts; and dividing the allocated range for the divided parallelized loops into loops in each of the divided ranges. A step for assigning to processors and a loop including a statement included in a dependent cycle are sequentially executed for each processor. Scheduling to be executed in line order, and scheduling a loop including a statement not included in the dependency cycle for each processor while maintaining the dependency such that the synchronization wait time by the dependency cycle is not idle. And a loop parallelizing method.