JPH02132526A - Process leveling method and automatic paralleling and compiling method for parallel computer - Google Patents

Abstract

PURPOSE:To shorten the computing time for all processor elements PE by preparing the parallel sequentially executing programs and eliminating automatically the uneven loads of PEs caused at execution of programs or caused by the static and uneven allocation when the programs are executed. CONSTITUTION:The arithmetic operation of an inside loop index is carried out in division by the PEs and some correlation is secured between the computing time of each PE and an outside loop index K. Under such conditions, the information on the time elapsed that is obtained by computing the inside loop in division via PEs is preserved to the preceding K value. Then the allocation is controlled for the PEs to the index K based on the preserved information. Thus it is possible to eliminate the uneven times elapsed of each dynamic PE that is caused by the success or failure of an IF sentence against an index (K + 1). Then the time elapsed can be shortened for all PEs.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention relates to a parallel computer system, and particularly to an object program suitable for parallel execution from a source program written in a sequential high-level language. Regarding the method of generating .

[Conventional technology]

Conventionally, in parallel processing systems such as multiprocessors, it has been necessary for the user to explicitly write instructions for means of parallelization, task activation and synchronization, etc. in a sequential source program as a user interface. A.C.
M-0-8979 1174-1-12185-010
7, A DataFlow Approach to
Multitasking on CRAYX
- M P Comput8rs describes a multitasking operation in which four vector processors are operated in parallel, and a user instruction method for the multitasking operation. According to this, a library for controlling task startup and synchronization is provided in the system, and users can use FOR
Write to call these in the TRAN program. Furthermore, it is necessary to instruct the compiler about the means of parallelization for each loop in the form of comment-style control statements.

On the other hand, in a parallel processing system as shown in FIG. 5, transmission processing and reception processing are performed independently. This type of system is described in detail in Japanese Patent Application No. 61-182361. This processing system will be briefly explained below: (1) This system is composed of a host computer 121 and a parallel processing section 122, and the parallel processing section 122 is connected to a plurality of processors 123 and a network that can transfer data between arbitrary processors. It consists of 124.

(2) Each processor 123 includes a local memory 125 that holds programs and data, an instruction processing section 126 that sequentially reads instructions from the local memory and executes processing, a transmitting section 127, and a receiving section 128.

(3) Data transmission processing is realized by executing a send command. When the Send command is decoded, the transfer destination processor number, data identifier, and data are set in the register 32 of the transmitter 27 from the register specified by the operand. Register 132-3
The information is sent to network 124 as a message. A message on the network is taken in as a pair of data identifier 130 and data 131 into the receiving buffer 129 in the receiving section of the processor indicated by the destination processor number in the message.

(4) Data reception processing is realized by executing a receive command. Receive
When the instruction is decoded, a search identifier is extracted from the register that specified the operand and sent to the receiving unit 128. The receiving unit 128 searches the receiving buffer 129 for a data identifier that matches the search identifier, and if there is no matching data identifier, it waits until a matching data identifier arrives, and if there is a matching data identifier, it reports it to the instruction processing unit. do. The instruction processing unit takes in the corresponding data.

A computer consists of a host computer that controls the entire computer and a processor element (PE) that can perform calculations or send and receive data independently. problem,
Or consider the case of executing a program used for problems such as the finite element method.

In such calculations, even if programs, data, and tasks are initially assigned to each PE so that the amount of calculation and communication is equal between each PE, the processing being executed may be affected because it includes processing that can only be determined at runtime. There may be non-uniformity in each PE. As a result, there is a problem in that processing is unevenly distributed between PEs during execution, increasing the amount of time it takes for each PE to finish processing, and lengthening the elapsed time for all processing. On the other hand, conventionally, especially for static task allocation, for example, J. Ramanujan, et. al: Ta
skAllocation dy Simulated
Annealing, Third International
ional Conference on Super
computing88, ICS88. The method described above seems to be effective, but the method used there can be inserted into the compiler object that automatically parallelizes a sequential compiled language for a parallel computer, or it can be applied to PE during dynamic execution. Nothing was used during execution to resolve load unevenness. Furthermore, from the viewpoint of automatic parallelization, there has been no effective means for solving the dynamic non-uniformity of PE loads during execution.

[Problem to be solved by the invention]

Conventionally, for hardware capable of parallel processing,
There is no consideration given to implementing parallel calculations by providing a means to resolve uneven load on PEs that occurs when static allocation is insufficient or during execution, and incorporating this means into automatic parallelization. It was not possible to make effective use of resources that made use of the characteristics of the wear. Furthermore, regarding automatic parallelization, it is not possible to run the sequential programs that users have in the past in parallel as they are; they must be recoated for parallel processing and debugged, which changes the characteristics of the hardware. It was necessary to change the parallelization instructions every time, and there were problems such as the user program not being able to run on other systems, and the versatility of the user program being impaired.

It is an object of the present invention to provide means for correcting the above-mentioned static task allocation (which cannot be taken into account in the process) and the non-uniformity of the load on the PEs that occurs during execution, at the time of compilation or loading, and to further reduce the above-mentioned burden on the user. Existing sequential programs can be automatically parallelized without modification, and even when coding new ones, efficiency can be improved without considering hardware characteristics or ways to eliminate dynamic loads that occur during execution. The goal is to be able to generate good object code.

[Means to solve the problem]

The above purpose is to create a host computer that is composed of multiple numbered processors and controls the entire computer, and a memory distributed parallel computer that uses processor elements that perform operations independently and a data transfer method between processor elements, or a shared memory system. In the process of compiling and loading, which targets memory-based computers and generates object code for parallel processing execution by the parallel processor from a sequential processing source program written in a high-level language, the sequential execution source program is After converting to a parallel execution type program and allocating data and programs to each PE from the flow of processing when executing the parallel execution type program, each PE that arises from the original allocation or that occurs dynamically during execution A means is provided to measure the elapsed time when the PE executes the assigned process, and the process of reallocating data and programs to each PE is performed so that the variation in the time between PEs is evenly distributed among the PEs. This can be achieved by inserting into the object code of the source program compiler a program code that performs a task leveling method that is dynamically performed by the host computer or PE while executing the processing assigned to the PE.

[Effect]

According to the above method, a sequential execution program is parallelized,
Uneven load on PEs that occurs during execution or is statically allocated unevenly can be automatically and dynamically resolved during execution, allowing users to rewrite the program assets they have accumulated for parallel processors. There will be no need. This makes it possible to achieve the above objective.

〔Example〕

Hereinafter, with reference to the drawings, an embodiment of the method of the present invention when applied to a FORTAN compiler for a parallel processor equipped with a plurality of processor elements and a communication path for data transfer between each processor will be explained. explain.

Figure 1 shows an overview of the present invention. In Figure 1,
Taking as an example a system to which the present invention is applied, which includes a distributed memory type PE shown in FIG. 2 and a host computer, explanation will be given assuming that the role of the host computer is important. However, in the present invention, in a computer system consisting of a distributed memory type PE and a host, the calculation of the part shown in FIG.
The present invention can also be easily applied to a type of parallel computer in which a host computer and a PE share memory.

Although FIG. 1 depicts a special case in which the number of PEs is three, the present invention is applicable to any number of PEs.

In the present invention, when there is a double loop (or even multiple loops) in which the number of operations occurring during execution is uniform as shown in FIG. If the amount of computation can be considered to be a loose function of the outer loop index, the non-uniformity of the amount of computation occurring in each PE can be expressed as Find the elapsed time for the next outer loop index, and calculate the elapsed time at each PE in the inner loop for the next outer loop index.
Data is exchanged between the E or PE, the tasks handled by the PE are adjusted, and the overall processing time of the double loop is shortened.

First, the outline of the principle of the present invention will be described in detail by giving specific examples. The outer K-loop 30 requires the inner El-loop to be calculated more times. If the inner eloop index is distributed evenly to each PE, the program at each PE will be the 8th
It will look like the figure. However, I of 41.41', 41'
Because the execution time of each PE is dynamically determined by the F statement depending on the success or failure of the IF statement, variations occur in the elapsed time of operations in each PE. VAL (I
, K) is a random quantity that does not depend on K, then VAL(I,
1) and the index of K for the second time VAL(I,
Since there is no correlation between 2) and 2), the variation in the elapsed time of calculations in each PE is random. However, if VAL (I, K) is a function that depends on K and changes slowly with changes in the value of K, then the amount VAL (I, K) for the previous K index and the amount for the next K index VAL (I, K+8), (fi=1.2,3,-
) has some correlation, and there is a correlation between variations in the elapsed time of calculations in each PE.

In this case, the elapsed time τW'' (m is the PE number) when the calculation amount regarding the inner eloop is divided into PEs and calculated.
is a function of the outer loop index K. For example, it changes with respect to K as shown in FIGS. 6(a), (b), and (c). FIG. 6(a) shows the case where τ1 takes a nearly constant value with respect to the outer loop index,
Figure 6(b) shows the case where the τ leg fluctuates randomly with respect to K around a certain constant value, and Figure 6(c) shows the case where τ1 gradually increases or decreases with respect to K. shows.

In this way, when an operation related to the inner loop index is divided and performed by PEs, if there is some correlation with K in the elapsed time when executing the operation in each PE, the inner loop index is The information on the elapsed time calculated by dividing the loop into PEs is saved, and the allocation of the inner index to the PE is adjusted for the K index of the next outer loop based on the information saved previously, so that K+1 is calculated. The dynamic non-uniformity of the elapsed time in each PE caused by the success or failure of the IF statement for the index is eliminated, and the elapsed time of all PEs can be shortened. The above is the principle of the present invention.

Next, the relationship between the number of times the elapsed time of each PE is measured in order to equalize processing and the number of outer loops will be described.

As shown in Fig. 6(a), 1 is the outer loop index K.
When the value is approximately constant, it is more efficient to obtain information on the elapsed time of each PE required for the inner loop calculation once at the first time than to obtain the same information many times. However, as shown in FIG. 6(b), when τ1 randomly fluctuates around a certain value with respect to the outer loop index K, information on the elapsed time of processing of each PE with respect to the K index Processing while cumulatively averaging smoothes out fluctuations in the elapsed time of each PE, making it easier to find a target value for adjusting the allocation of inner indexes to PEs. However, even in this case, either check the first few steps of the K loop, set the above-mentioned target value, and then either do not calculate the elapsed time processed by each PE for the K index after that, or set a certain interval. It is more efficient to calculate the elapsed time only once every few times, as it saves the effort of obtaining the same information over and over again. Therefore, if the user knows these things in advance, unnecessary calculations can be avoided by giving instructions like the current option specifications of the compiler. In the case shown in Figure 6(c) where more fine-grained leveling of processing is required, information on the processing time of calculations in each PE of the inner loop is obtained for each outer loop index, and the optimization target value is set each time. It is better to update
The overall processing time can be further reduced. Therefore, although the following discussion assumes a case as shown in FIG. 6(c), it is easily applicable to general cases as well.

Furthermore, in order to repeat calculations several times for the processing required to equalize the processing in the PE, the user can also specify the ratio between the number of repetitions and the number of times elapsed time is measured.

The number of outer loops, the number of times to measure the elapsed time of each PE, and the number of iterations to determine task division for equalization can be specified by the user, but in addition, the system can specify a small value of K. It is also easy to make a prediction based on the selection.

Next, the processing roles of the host computer and PE that perform processing will be described.

Here, description will be made assuming that at K=1 in the outer DO loop, there is variation in the elapsed time of each PE as shown in FIG. In order to increase speed and use resources effectively, it is necessary to equalize the load on the PEs and to avoid concentration of data transmission and reception on specific PEs. Therefore, the host computer 1 performs a process of dynamically determining how much load distribution should be performed by each PE. To do this, each PEi measures the execution time τL when K=1 before starting the next outer loop K=2, sends the result to the host computer, and the host calculation is performed from the value of τ1 of each PE. , for example, by the dynamic dispersion method shown in Figs.
The host computer instructs each PE which data should be sent to which PE, or which data should be received from which PE. Due to this. As the loop of K progresses to K=2.3, . By doing so, for example, compared to the case where K=1 is executed without any process leveling, the elapsed processing time is increased by Δ1 in Figure 1 when K=3, and Δ2 when K=4. becomes shorter.

FIG. 2 shows the overall configuration of a compiler to which the present invention is applied. A syntax analysis process 7 in FIG. 2 inputs the FORTRAN source program 6 and converts it into an intermediate language 10. The intermediate processing 8 takes this intermediate word 6 as input and transforms the intermediate word 6 by performing optimization and parallelization as much as possible. For example, the automatic parallelization part of the intermediate processing 8 follows the parallelization compilation method of Chuken Patent No. 318705174 by Iwasawa et al.

In the present invention, after such processing, a process 11 is performed to correct parts that are insufficiently parallelized or to analyze dynamically generated non-uniformity of PE loads, and calculates the average τ of each PE. Nk is determined, and for the next value of the number of outer loops, τ plate −τ.
The data regarding the index is set to P so that l=O.
This is to insert processing such as sending and receiving with E into the intermediate code. In process 18, it is determined how much data should be sent from which PE to which PE.

Here we will explain in detail the process of leveling the calculations. The portion related to the process 11 in FIG. 2 will be explained assuming that the program in FIG. 7 is input as an example of the input source program 6 in FIG. 2. It is assumed that the parallelization code as shown in FIG. 8 is distributed to the PEs in such a way that the inner loop index I is evenly divided among the PEs by process 8 in FIG.

In process 12 in FIG. 2, initial values or other necessary values are set for the method of dynamically distributing calculations related to the inner loop to PEs. Processing 13 is executed by the host computer, for example, by performing initial processing of dividing adjacent processors into groups, as will be described later.

Processing 15 is a part performed by each PE.
The calculation amount Ct (i; PE number) is determined from, for example, the number of executed instruction steps. Processing 16 calculates the communication amount ei. between the i-th PE and the j-th PE. and the communication distance d5a defined between PE-i and PE-j (e.g. Hamming distance II
From I), in the communication load processing 17, the amount of calculations Ci in each PE is
From the expected value of the communication load Cc, the elapsed time of the calculation corresponding to the inner loop of each PE is determined for each PE.

Process 16 - Process 17 are executed in each PE, but Process 18
is performed on the host computer. I will give this as an example. However, this part can also be executed in parallel by PE.

Process 18 instructs each PE to send the numerical result of the elapsed time τ' (i = 1,...Npe) of the calculation corresponding to the inner eloop to the host computer, and based on the above numerical result. Then, the host computer calculates the assignment of the operation corresponding to the inner eloop to the PE.

That is, it is determined from the PE that the operation corresponding to which eroop index is to be reassigned to which PE. Processing 18
This will be explained in detail later.

In process 19, data is transferred between PEs according to the instructions from the host computer to distribute the PE tasks determined in process 18, and the loop index in process 45 in FIG. initial value Nl of
This is the part that adjusts the final value N2 and changes the task assignment.

The programs to be executed by the host computer and PE by the process 11 in FIG. 2 are shown in FIG. 3 or 4 and FIG. 9, respectively. According to the present invention, for the program shown in FIG. 7, the compiler outputs code having the functions shown in FIG. 3, FIG. 4, and FIG. 9 as an object code.

First, the grouping of PEs in process 13 in FIG. 2 will be described. This will be explained using Figure 10. 60-67 and 8 Ps
Suppose that there is E (the number of PEs in a parallel computer is usually an even number). 2
This process is repeated by selecting PEs one by one, creating groups, and then combining the groups to create new groups.

For example, consider 60 and 61 as one group and set it as 7o,
Let 62 and 63 be 71, 64 and 65 be 72, and 66 and 67 be 73. This group is α=1 (level 1)
group. Next, for the group of Q=1, 7
Make 0 and 71 into a new group and make it 80, then 72 and 7.
3 are grouped together as 81 to form a group of Q=2. Perform this series of operations and stop when there are two groups at a certain level. In the case of FIG. 10, the operation is stopped when α=2. The elapsed time required for the calculation of the apparent inner eloop in group units after grouping, which will be required in the subsequent process 18, is determined as follows. That is, the elapsed time in the Ω level group is determined from the calculation amount of each inner eloop of the Q-1 level group units p and q. Since the processing scene of one unit time is known in the system, it is converted into elapsed time using these values.

As described above, based on the elapsed time for computation of the inner eloop in each PE, it is possible to obtain the elapsed time for computation of the inner eloop in a group within each level for all levels. These groupings occur when the host computer performs optimization processing, which will be described later, when the system falls into a state that appears to be an optimal solution (quasi-equilibrium state in thermodynamics) even if it is not optimal. This is newly introduced in the present invention to avoid this problem.

Process 18 will be explained in detail.

As a first method, a certain PE-n and P
The difference τ in the elapsed time at PE required to calculate the inner eloop of E-m. −τ. seek. In process 181, the deviation of the elapsed time of each PE from the average processing time is calculated as Δτ. Processing when is greater than a certain value 182
Let us perform the following processing. Here, it is further determined by the user's instruction whether the processing from 182 onwards is to be performed only once for each K time or many times. Here, a is the group level number. Npe is the total number of PEs.

When T@tHkuO, from PE-n to PE-m, τ
．． −τ, we tentatively decide to send array data with the index of the process corresponding to the amount of computation that satisfies (
Processing 186). Then, the elapsed time of the PE is changed to match this (processing 187). All these operations are performed on the PE.

Processes 182 to 188 are then repeated until the difference in elapsed time for all pairs of PEs becomes almost zero.

As a result of the above, the process 189 can determine which PE should send array data corresponding to which index to which PE and redistribute tasks in order to equalize the elapsed time in each PE. In process 190, the host computer issues a data transfer transmission/reception command and a command to change the task to be added to each PE from the host computer.

The above method deterministically equalizes the elapsed time of each PE. The above can be done in the same way when grouping.

This method works as follows: 1] When the number of E is small, for each K, such as when K=1 or K=2 in the outer loop in Fig. 1,
This is particularly effective when attempting to equalize the elapsed processing time in the PE in the inner eloop, such as when K=4.

However, when the number of PEs increases, the number of combinations for sending and receiving data or tasks between PEs increases, and the amount of processing on the host computer increases.For example, when K=
When 1, when there is a large load bias on a specific PE,
There is a concern that the processing for data transfer or task change may be biased toward a specific PE, and the elapsed time required for such processing may become too long. Therefore, although it is a probabilistic method, a method for distributing the calculation load and communication load that is concentrated on a certain PE among many PEs will be described below.

? The method assumes that there is interaction between particles, and when a collection of particles is considered as a whole system, the system is brought closer to a state of thermal equilibrium at a certain temperature, and the temperature is lowered until the free energy of the system is the lowest. It uses the Metropolis Monte Carlo method, which changes the arrangement of particles and determines the arrangement.

In FIG. 4 181', the same processing as in FIG. 3 181 is performed. 4, 182' and 183' correspond to a situation in which the temperature of a system consisting of a large number of particles is slowly cooled to T m t■, and α can be compared to a parameter that controls the cooling rate. Parameter T is a controlled parameter. On the other hand, T 1n is defined as (i
: PE number) When calculated by Npe' = 1, in the same way as deciding to what extent the particle should be allowed to deviate from its stable position, it is also decided to make the elapsed time of each PE's calculation uniform. It is a parameter.

Processing 184'194' divides PEs into groups and looks at the elapsed time of PEs in detail or coarsely, so that in the multi-particle example above, the arrangement of particles appears to remain unchanged. Avoid. In other words, even if the tasks are not leveled, if you only take a closer look, this is done to avoid the system falling into a state fi (quasi-equilibrium state) where the tasks appear to be leveled even though they are not leveled. .

Processes 186', 187' and 188' are a part of randomly selecting which index array data is to be sent from which PE group to which PE group using random numbers. That is, in process 186', one PE or a group of PEs is selected using a random number between 0 and 1. Now, suppose that PE('')-m has been selected. On the other hand, in process 187', the currently selected PE or one of the inner indexes in the PE group is selected using random numbers, and the array data associated with is selected. Let the selected index be 1'. Process 188' is the process 1
Similarly to 86', a random number is used to select the destination PE selected in 187' from the previous PE or PE'.
Alternatively, the PE group is randomly selected using random numbers (now, P
Suppose that E'-n is selected. ) part. Here, a PE from the PE group can be arbitrarily selected at random. Processing 189'
-191' investigates whether Δτ(1) increases or decreases when data determined by the index selected above is sent from PE(')-m to PE(')-n.

Processing 192' sets the array related to the index of I' to P E −
Move from m to PE-n and change the task accordingly. Here, the task change will be described later. Δτ(
1) When ≧0, that is, when the cost function increases, the cost
The probability of transition from PE-m to PE-n of the array with respect to the index of is -Δτ('')/' a T t.
Yo,. below. Perform the procedure. .

e (where kB: is a certain constant), that is, random number r (0 and 1
) and ・1A @ (')/kaT, ・〈
If A c (')/kBr, the e array related to the index of engineering is moved from PE-m to PE-n and the associated task is changed. In the process 194, the process 190 in FIG.
The host computer issues an instruction to the PE to perform the same process as .

If the above operation is carried out, as shown in Figure 1, after the outer loop is repeated several times, the laws of physics state that the elapsed processing time at each PE in the inner loop will be approximately constant for all PEs. This can be guaranteed from the correspondence. The above is PE on the host computer.
This is a series of processes to change the allocation of

The object code for carrying out the process 18 used in the present invention, that is, the procedure shown in FIG. 3 or 4, is sent to the host computer so that it is executed therein. (However, processing 18
It is also possible for PEs to share the processing. ) FIG. 9 shows what kind of object code is generated by the Q-th PE when processing of the sequential program shown in FIG. 5 is shared among PEs according to the present invention. Here we will discuss changing tasks. Process 45 is shown in FIG. 8 40'
This is the part that performs the processing corresponding to ~44', and here,
SUB to clearly indicate that tasks will be shared among each PE.
It is shown in the form ROutINE A (N 1 , NZ).

Here, N 1 + N z are the initial value and final value of the DO statement.

For example, first, when PE=1 in Figure 6, SUBROUTIN
Consider that EA (L 10) is executed. In process 16, the elapsed time τ1 is calculated, and in process 51, the host computer transmits the value of Wisteria. In process 52, the host computer waits for completion of process 18. As a result, by processing 18, for example, PE-
In 1, the parts of index 1, 2, 3 and 5.6, 7
, 8, 9, and 10.11 should be executed, and it has been decided that the part related to engineering = 4 should be calculated elsewhere. Then, at a processing time of 53°, data V (
4) , VAL (4,K) and others. 17) Send P E and I=41 from r'E-Z! i data V (
11), receives VAL(11,K). At this time, the application system attaches a data identifier and issues a Ser+d or Receive command. And in process 19,
SUBROUTINE A(1, 3) and SU
Adjust the values of N1 and NZ to calculate BROUTINE A (5, 11), and in process 45, CAL
The one set as LA (1.10) is C, ALL A
The task is reallocated to CALL (1, 3) and CALL (5, 11). In the above, the CALL statement and SU
Although the task is described in the form of BROUTINE, it is easy to perform similar processing using other methods.

〔Effect of the invention〕

According to the present invention, a conventional sequential user program can be executed on a parallel processing system by dynamically leveling the load on PEs without recoding. Further, at this time, by effectively using hardware resources, it is possible to automatically generate object code with a short elapsed time and high effective efficiency.

[Brief explanation of drawings]

FIG. 1 is a schematic diagram of the present invention, FIG. 2 is an overall diagram of the compiler and a schematic diagram of the present invention, and FIGS. 3 and 4 are schematic diagrams of an embodiment of processing performed by the host or PE of the present invention. Fig. 5 is a diagram showing a parallel computer system for explaining the embodiment, Fig. 6 is a diagram showing a parallel computer system for explaining the embodiment.
The figure shows the relationship between outer loop and inner loop processing time,
FIG. 7 is a diagram showing a source program for explaining the embodiment, FIG. 8 is a diagram showing parallelization code, and FIG. 9 is a diagram showing each PE.
FIG. 10 is a diagram of the procedure for grouping PEs. 1... host computer, 11... dynamic leveling processing unit,
12... Inner loop PE distribution processing initialization unit, 13...
・PE grouping processing unit, 15, 16, 17...
A part that calculates the elapsed time of each PE from calculations and traffic in the inner loop for the outer loop, 18... Redetermines the distribution of each PE in the inner loop, 19... Readjusts the tasks in the inner loop by data communication, etc. . Kuzu Dan Kuzu 2 Figure 7 Figure DO Z K=1. 700000%30
? O I I. I. 3ρ one\-31
I1- (VAL(r,〆).LEVALO) th
a-r 3ZRV(E)=V(r)*axe 4
■33E/t/tsu IF
/111\\3/il CON ITINUE
'1~ゝ\, -35z c0n71n
ba 11-1111~-
3Δ Fig. /0 (Good)

Claims (1)

[Claims] 1. A host computer that is composed of a plurality of numbered processors and controls the entire computer, a processor element (PE) that performs operations independently of the computer, and an inter-processor data transfer method. In the process of compiling and loading that generates object code for parallel processing execution by the parallel processor from a sequential processing source program written in a high-level language for a memory-distributed parallel computer to be used, the sequential execution After converting a source program into a parallel execution type program and allocating data and programs to each PE from the flow of processing when executing the parallel execution type program, data and programs arising from the original allocation or dynamically generated during execution The process of reallocating data and programs to each PE is performed by measuring the elapsed time that occurs when each PE executes the process assigned to it, and then reallocating data and programs to each PE so that the variation in time due to PE is equalized among all PEs. The host computer or PE dynamically executes the processing assigned to each PE.
Processing leveling method for parallel computers. 2. The object code provided with the leveling method described in claim 1 is inserted into the object code of the source program as the object code for compilation into the host computer and processor at the time of compilation, and is output as the object code output by the compiler. Automatic parallelization compilation method. 3. It is written in a high-level language, targeting a so-called memory-sharing type parallel computer consisting of a host computer that controls the entire computer and a processor element (PE) that performs operations independently, consisting of multiple numbered processors. In the process of compiling and loading to generate an object code for parallel processing execution by the parallel processor from a serial source program, the sequential source program is converted into a parallel program, and the parallel program is converted into a parallel program. After allocating data and programs to each PE from the flow of processing when executing, execute the processing assigned to each PE that arises from the original allocation or that occurs dynamically during execution. The host measures the elapsed time and reallocates data and programs to each PE so that the variation in the time between PEs is equal across all PEs while executing the process assigned to each PE. A processing leveling method for parallel computers that is dynamically performed by a computer or PE. 4. Inserting the object code provided with the leveling method according to claim 3 into the object code of the source program as the object code for compilation into a host computer and processor at the time of compilation, and outputting it as the object code output by the compiler. Automatic parallelization compilation method. 5. For source programs containing two or more loops or two or more iterative operations, an inner DO loop divided into each PE, or Determine the elapsed time for each PE in the process of repeated calculations, and calculate the elapsed time for all PEs.
On the other hand, (1) the PEs are divided into two groups at a time, and the grouped PEs are further divided into two groups at a time.
Create groups of several levels, determine the magnitude of elapsed time for two PE groups in the same group for the same level of the group, and how to divide the data within the groups of the same level. (2) Create two pairs between all PEs without grouping them, and determine the size of the elapsed time. (3) Probabilistically transfer data to each other until the elapsed time is equal for all PEs.
Select E, take the difference in the elapsed time of two PEs, and stochastically determine the transfer probability between PEs for the processing amount corresponding to the difference, and calculate the progress of processing at each PE. Either by making time probabilistically uniform, each P
A processing leveling method for a parallel computer that eliminates and levels the uneven elapsed time of each PE that occurs in E. 6. For a source program containing two or more loops or two or more iterative operations, the inner DO is divided into each PE, once for every outer loop or outer iterative operation. Determine the elapsed time in each PE for processing a loop or repeated operation, and (1) divide the PEs into groups of two, and further divide the grouped PEs into groups of two. By performing the grouping operation one after another, we create groups of several levels, and for the same level of the group, we calculate the elapsed time of two PE groups from the same group. (2) Decide how to share data within groups at the same level, and then instruct which PE should send data to which PE to reprogram, or (2) decide how to share data among groups at the same level. Create two pairs between PEs, thoroughly check the magnitude of the elapsed time, and transfer data to each other until the elapsed time is equal for all PEs, or (3) Probabilistically transfer data between two PEs.
, calculate the difference in the elapsed time between two PEs, stochastically determine the transfer probability between the PEs of data corresponding to the difference, and calculate the elapsed time of processing at each PE. Each P
A processing leveling method for a parallel computer that eliminates and levels the uneven elapsed time of each PE that occurs in E. 7. For a source program containing two or more loops or two or more iterative operations, use each iteration of the outer loop or outer iterative operation to match the iterative part of the method described below, Calculate the elapsed time in each PE of the inner DO loop divided into PEs or the processing of the repetitive operation, and calculate the elapsed time (1
) Divide the PEs into groups two at a time, and then divide the PEs into two groups one after another, creating groups at several levels. 2 of the same group for the same level of
Decide on the size of the elapsed time in each PE group, decide how to share the data within the group at the same level, and decide which PE group
(2) Instruct all PEs to send data to which PE should change the program, or (2) send data to all PEs without grouping.
Create two pairs between PEs, thoroughly check the elapsed time, and transfer data to each other until the elapsed time is equal for all PEs, or (3) select two PEs probabilistically. The difference in elapsed time between two PEs is calculated, and the probability of transferring data between PEs corresponding to the difference is stochastically determined so that it follows a certain distribution, and the elapsed time of processing in each PE is stochastically determined. A processing leveling method for a parallel computer that eliminates and levels the non-uniformity of elapsed time that occurs in each PE by making it close to uniformity.