JPH03172958A - Synchronous processing method, system and method for parallel processing, and parallel program generator - Google Patents

Abstract

PURPOSE:To efficiently execute a program by waiting for the other processors to synchronize each processor with them in accordance with arrival of the processing of this processor at a prescribed waiting position and selecting an execution restart position after waiting by each processor in accordance with the processing result of this processor. CONSTITUTION:Each of processors 1 to 4 waits for the other processors and is synchronized with them based on first information indicating whether processings of respective processors 1 to 4 arrive at a prescribed waiting position or not. Meanwhile, each of processors 1 to 4 selects an execution restart position after waiting based on second information corresponding to processing results in processors 1 to 4. That is, processors 1 to 4 are synchronized with one another, and they are branched to different processings in accordance with processing results. Therefore, this system can cope with not only the for type loop but also the while type loop. Thus, information is efficiently written and referred.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a synchronous processing method, a parallel processing system, a parallel processing method, and a parallelized program generation device,
More specifically, the present invention relates to a synchronous processing method, a parallel processing system, a parallel processing method, and a parallel program generation device for proceeding with processing while synchronizing each processor when processing is performed in parallel by a plurality of processors.

[Prior Art] There is a parallel processing system in which processing is performed in parallel by a plurality of processors in order to improve the effective processing speed of a program.

In a parallel processing system, a program is divided into multiple parts,
Each divided program is executed in parallel by multiple processors. At this time, if there is an order relationship (data dependency relationship) between data definitions and references in each of the divided programs 11, it is necessary to ensure the order relationship is correct in order to obtain correct execution results. In other words, the execution results differ depending on whether the data is defined and then referenced, or referenced and then defined. , each processor must execute each divided program.

For this reason, processors in charge of programs that have data dependencies execute the programs while synchronizing so that the order of data definition and reference is correct.

As a conventional technique for the above synchronization, for example, a synchronization processing method called barrier synchronization is known.

This barrier synchronization sets a certain point as a barrier for all divided programs, and when each processor reaches this barrier, it waits for the processing of other processors to progress to the barrier without moving on to the next process. , synchronization is achieved by allowing the processing of all processors to proceed to the barrier before moving on to the next processing.

General explanations of this barrier synchronization include, for example, "Parallel Processing Machine by Shinji Tomita Ohmsha" and rlEEEESof
tware January, 1988 P, 34
-42J. For specific implementation methods, see INEEE TRANSACTIONS, for example.
ON COMPUTERS AUGUST.

1988 P, 991-1004J.

[Problem to be solved by the invention] The above rIEEE TRANSACTIONS ON C
OMPUTERS AU-GUST, 1988
The target of the concrete implementation method of barrier synchronization described in P, 991-1004J is repetitive processing, where the increment of the variable that controls the repetition is constant, and the number of repetitions is determined at the time of compilation. It is something that Since this corresponds to the for statement in the C language, this will be referred to as a for-type loop below.

Generally, the target of parallel processing is repetitive processing that consumes most of the program execution time, so f
The above-described conventional techniques applicable to or-type loops are useful.

However, if the increment of the variable that controls repetition is not constant,
Alternatively, there may be repetitive processing in which the number of repetitions is not determined at the time of compilation. This is wh in C language
In order to correspond to the le statement, this is hereinafter referred to as a while type loop. The above conventional technology is
The problem is that it cannot be applied to type loops.

Therefore, the purpose of the present invention is not only for loops (,
The object of the present invention is to provide a synchronization processing method that can be applied to a while type loop as well. Another object of the present invention is to provide a parallel processing system that executes the synchronous processing method. Another object of the present invention is to provide a parallel processing method using the synchronous processing method. Another object of the present invention is to provide a parallelization program generation device that generates a program that executes the synchronous processing method.

[Means for Solving and Accomplishing the Problems] In a first aspect, the present invention provides a synchronization processing method for queuing processors with each other in order to harmonize the progress of parallel processing by a plurality of processors. First information indicating whether or not the meeting place has been reached, and second information corresponding to the processing result of each processor are stored outside the processor for each processor, and each processor provides a synchronization processing method characterized in that each processor waits and synchronizes with other processors, and each processor selects a place to resume execution after waiting based on the second information.

In a second aspect, the present invention includes a plurality of processors, first register means having a synchronization detection bit assigned to each of the processors and set by the corresponding processor; a second register means having a restart location designating bit set by the processor that executes the process; and a synchronization system that determines synchronization from the state of a synchronization detection bit of the first register means and instructs each processor to wait or resume execution. A parallel processing system comprising: a determining means; and an execution restart location specifying means for specifying an execution restart location for each processor based on the state of a resume location designation bit of the second register means. provide. Note that the first register means and the second register means may be physically one register.

In a third aspect, the present invention provides a parallel processing method in which a plurality of processors execute processing in parallel.
outputting information to the outside of the processor, outputting second information corresponding to the processing results of each processor to the outside of the processor, and queuing processing based on the first information output from other processors. Alternatively, there is provided a parallel processing method characterized by comprising the steps of restarting execution, and selecting a place to restart execution based on the second information output from another processor.

In a fourth aspect, the present invention provides a parallelization program generation device that generates a parallel processing program from a sequential processing program, in which first information indicating that a process has reached a predetermined waiting location of the program is transmitted to the outside of the processor. means for adding a step to the parallel processing program to output second information to the outside of the processor, means for adding a step to the parallel processing program to output second information corresponding to the processing result in the program to the outside of the processor; means for adding a step to the parallel processing program for queuing processing or resuming execution based on the first information output from another processor; and resuming execution based on the second information output from another processor. A parallel program generation device is provided, comprising: means for adding a step for selecting a location into a parallel processing program. In addition,
This parallelization program generation device may be in the form of a compiler.

[Operation] In the synchronous processing method of the present invention according to the first aspect, each processor waits and synchronizes with other processors based on the first information indicating whether the processing of each processor has reached a predetermined meeting place. . On the other hand, each processor selects a place to resume execution after waiting based on second information corresponding to the processing result of each processor. In other words, each processor can be synchronized and branched into different processes depending on the processing results. Therefore, it becomes possible to handle not only for type loops but also (and while type loops. Also, since the first information and second information are held outside the processor, writing and referencing of information can be performed efficiently. Become so.

In the synchronous processing method of the present invention according to the second aspect, each processor sets the allocated synchronous detection bit of the first register means depending on whether the processing has reached a predetermined meeting place. The synchronization determining means determines synchronization from the state of the synchronization detection bit of the first register means and instructs each processor to wait or resume execution. On the other hand, each processor sets the assigned restart location designation bit of the second register means according to the processing result.

The execution restart location designation means designates the execution restart location for each processor based on the state of the restart location designation bit of the second register means. This allows each processor to be synchronized and to branch to different processes depending on the processing results. Therefore, not only for-type loops (and while-type small loops can also be supported).

In the parallel processing method of the present invention according to the third aspect, the processor outputs first information indicating that the processing has reached a predetermined meeting place and second information corresponding to the processing result to the outside of the processor. . Then, at a timing based on the first information output from another processor,
Processing is restarted from the execution restart location based on the second information output from the other processor. Therefore, since it becomes possible to synchronize with other processors and to branch to different processes depending on the processing result, it becomes possible to support not only for-type loops but also while-type loops.

In the parallelization program generation device of the present invention according to the fourth aspect, when generating a parallel processing program from a sequential processing program, the first information indicating that the processing has arrived at a predetermined waiting place is transmitted to the outside of the processor. a step of outputting second information corresponding to the processing result to the outside of the processor; and a step of waiting or resuming execution of the process based on the first information output from another processor; A step of selecting an execution restart location based on the second information output from another processor is added to the parallel processing program. Therefore, the generated parallel processing program can handle not only for-type loops, but also while-type loops.

[Example] Hereinafter, the present invention will be explained in more detail with reference to Examples shown in the drawings. Note that the present invention is not limited thereby.

FIG. 1 is a block diagram of a parallel processing system 100 implementing the synchronous processing method of the present invention.

This parallel processing system 100 includes four processors 1,
2, 3.4, the shared main storage device 10, and bits Al, A2, A3, which indicate whether each of the processors 1 to 4 is executing a queuing process or a process other than the queuing process. A register A101 that holds A4, and a bit B that specifies the location where program execution will resume after the end of the waiting process for each processor 1 to 4.
l, 132°B3. A register B102 that holds B4, a register A determination circuit 103 that determines each bit A1 to A4 of the register A101 and issues an execution restart instruction 109 to each processor 1 to 4, and each bit 81 of the register B102.
- Processor control signals 105a, 105b, 10 that determine B4 and specify the execution restart location for each processor 1-4
6a, 106b, 107a, 107b, 108a, 10
8b.

The register A determination circuit 103 takes the AND of all bits A1 to A4 of the register Al01, and uses the result to issue an execution restart instruction 1.
09 to each processor 1 to 4.

Register B determination circuit 104 encodes bits B1 to B4 of register B 102 and outputs a processor control signal 105.
It is output to each processor 1-4 as a-108b. FIG. 2 shows the relationship between the values of bits B1 to B4 of register B102 and processor control signals 105a to 108b. Note that * in FIG. 2 indicates that either O' or '1' may be used.

FIG. 3 shows wh processed by the parallel processing system 100.
An example of a program for an ile type loop is shown below.

This program 400 assigns 1° to i, and if array element a[i] is not 0°, performs some processing, then assigns 0° to array element a[i], and increments i. This is a program that terminates the process when element a [i] of the array is found to be 0°.

The program 400 includes programs 401, 402, 403, 404 for parallel execution on each of the processors 1 to 4.
divided into Although illustration of the contents of the programs 403 and 404 is omitted, they are similar to the contents of the programs 401 and 402.

The following statements are added to the divided programs 401 to 404 in order to control the synchronization of processing.

clear register A [xl... register Al0
I's beep) Clear Ax.

clear register' register B [xl... register B10
2nd pick) Clear Bx.

set register A [xl... register Al0I pin) Set Ax.

set register B [xl...Sets bit Bx of register B102.

Waiting process: Waiting for synchronization. This will be explained later by showing some specific examples.

Programs 401, 402, 403, and 404 collect necessary information automatically or through interaction with the programmer by a compiler with a parallelization conversion function.
Generated from 00.

The compiler has a register A101 for synchronization processing,
Generate a statement to clear the bit of register B102,
Start converting the program 400.

Since the variable i that controls the loop is temporarily used by each processor 1 to 4, it is allocated to the register regj and initialized. The initial value is the same value as the program 400 for the processor 1, increases by the increment of the variable i of the program 400 for the subsequent processors 2 to 4, and increases by 2. 3 for processor 3. For processor 4, it is 4.

In converting the main body of the loop, starting from the beginning, there is a statement that sets the bit of register A101, a statement for waiting processing, and a label "addr Oo" indicating the effective restart location when processor control codes 105a to 108b are "00°". A statement that clears the bit of register A101 is generated.Then, the instruction string of the loop body of the program 400 is arranged.

However, an instruction to update variable i in program 400 is converted to an instruction to update register regj, and the increment in register reg-j is converted to (number of processors x increment in variable i in program 400), that is, 4.

After this, register B102. A statement for setting the bit of register Al0I and a statement for queuing processing are generated.

Then, processor control signals 105a-108b become 01
Label 'addr 1' indicating where to resume execution when °
Then, an instruction obtained by removing the update instruction for variable i from the instruction sequence of the loop body of the program 400, and a branch instruction to the end label "exit" of all processing are generated.

Further, the processor control signals 105a to 108b are 11
Label 'addr indicating where to resume execution when ° 3°
Then, an instruction to bring register regj into variable i of the program 400 is generated.

Finally, the processor control signals 105a-108b are 10
Label indicating where to resume execution when ' addr 2
° and the end label °exit' for all processes are generated, and the conversion is ended.

Next, the operation of the parallel processing system 100 will be explained. It is assumed that the array a[i] shown in FIG. 4 is given as data.

Processor 1 uses register A by program 401.
The first bit A1 of register B101 and the first bit B1 of register B102 are cleared. Similarly, in processor 2.3.4, the second bit A2 of both registers 101 and 102 is
．． B2 and 3rd pit) A3. B3 and the fourth bit A4.
Clear B4.

Next, register regj of each processor 1 to 4 is initialized. The initial values are 1" for processor 1, 1" for processor 2, and 1" for processor 2.
3.4 are 2°, 3°, and 4°, respectively.

Next, in the while condition judgment, processor 1 selects array element a[1], processor 2 selects array element a[1], and processor 2 selects array element a[1].
2], the processor 3 determines array element a[3], and the processor 4 determines array element a[4].

As shown in Figure 4, each element a[1] to a[4] of the array
Since the content of is not 0°, the main body of the loop is executed.

In the processing of the loop body, processor 1 stores register Al0.
The first bit A1 of the processor 2.3.4 is the second bit A2. Set the third bit A3 and the fourth bit A4.

Then, each of the processors 1 to 4 enters a waiting process.

The queuing process uses the interrupt function of the processor, and its flowchart is shown in FIG.

First, wait until an interrupt occurs in an infinite loop (501
).

Execution restart instruction 1 from register A determination circuit 103 during standby
When an interrupt occurs in each of the processors 1 to 4 due to the output of 09, the interrupt processing routine is started and the process moves to step 502.

In step 502, processor control signals 105a-1
08b, if the value is 'oo' go to addr Oo, if the value is '01' go to addr 1°, if the value is '10' go to addr 2°, if the value is '11' go to 'addr 3'
°Hedge jump and resume execution.

Now, in the register A determination circuit 103, the register Al0
Since the first bit A1 to the fourth bit A4 of I are set, an execution restart instruction 109 is issued to each processor 1 to 4.

In this way, after 1 to A4 are set in all the pits of the register A101, each processor 1 to 4 is instructed to resume execution 1.
Since 09 is issued, processors 1 to 4 can be synchronized (barrier synchronization).

The register B determination circuit 104 encodes the first bit B1 to the fourth bit B4 of the register B 102 according to the table shown in FIG.
Outputs 108b. At this point, all bits 81 to B4
is 0°, processor control signals 105a to 108b are sent to all processors 1 to 4 to indicate where to resume execution.
outputs °00° as

Processor control signals 105a-10 indicating where to resume execution
8b and an execution restart instruction 109 are input to each processor 1 to 4, the above-mentioned waiting process is skipped and the respective processor control signals 105a, 105b, and 106a are input.
and 106b, 107a and 107b, 108a and 108b
Resumes execution from the point indicated by . At this point,
Execution will resume from label 'addr Oo.

When the processor 1 resumes execution from 'addr Oo, it clears the first bit A1 of the register A101 and then executes the processing of the loop body in the program 400. Then, array element a[reg-jE has °0°
Substitute. Furthermore, the register reg-j has (number of processors x increment of variable i of program 400) °4°
After adding , the process returns to the while condition determination.

Processor 2.3. The same is true for restarting execution from °addr Oo in step 4.

In the second loop processing, each of the processors 1 to 4 determines the elements a[5] to a[8] of the array, but as shown in FIG. Since the content of [8] is not 0°, the operation is similar to the first loop process.

Thereafter, each of the processors 1 to 4 performs loop processing according to the contents of the array a[i] shown in FIG.

At the 2500th loop, processor 1 executes w
Since the content of element a[9997] of the array is not 0° in the hile condition determination, execution of the main body of the loop is started, the first bit Al of register Al0I is set, and the waiting process is performed.

However, in processor 2.3, array element a [9
998], a Since the content of [9999] is °0°, the process does not enter the main body of the loop, and the second . The third bit is set and the waiting process begins.

Since the contents of array element a [100001 are not 0, processor 4 enters execution of the main body of the loop, sets the fourth bit A4 of register Al0I, and performs a waiting process.

The register A determination circuit 103 determines whether each processor 1
-4, an execution restart instruction 109 is issued.

The register B determination circuit 104 codes the values of each bit 81 to 84 of the register BIO2 according to the table shown in FIG.
4) = (0, 1, 1, O), so the processor ail ti+ signal with the contents of 01" is output to the processor 1, 11° to the processor 2, and 10" to the processor 3.4.

Processor 1 receives processor control signal 105a.

Since °01' will be input as 105b,
Resuming execution from 'addr 1', processor 400
After executing the instructions in the main body of the loop, the program branches to the end label 'exit' of all processes, and ends the process.

The processor 2 receives the processor control signal 106a.

Since °11° will be input as 106b,
'addr Resume execution from 3° and register regj
The main memory 10 corresponding to the variable j of the program 400
The data is stored within the area and processing ends.

Processors 3 and 4 receive processor control signals 107a and 1
Since 10' will be input as 07b, 108a and 108b, execution will be restarted from 'addr 2' and the process will end.

The execution result is that array element a [l col a [9999] is °0, array element a [10000 is 1000
It becomes 0'. The variable i becomes '9998'.

Next, another example of the waiting process will be described with reference to FIGS. 6 to 10.

The first other example is to create a special command.

The instruction format is shown in FIG. 6, and the instruction processing flowchart is shown in FIG. 7.

As shown in FIG. 6, instructions 600 include operation 6
01, addr O, addr 1° addr2. addr3(602,603,604,6
05). The processor that has executed this instruction 600 waits until an execution restart instruction 109 is input.

As shown in FIG. 7, when the execution restart instruction 109 is input (701), the processor control signals 105a to 108b
702), the address of the execution restart location is transferred to the PC (program counter), and execution is resumed from this address.

The second example is to create a special instruction similar to the above example, but the method of specifying the execution restart location is different. The instruction format is shown in FIG. 8, and the processing flowchart is shown in FIG. 9.

As shown in FIG. 8, instructions 800 include operation 8
01 and the offset group (802, 803°804, 8
05). Offset 0. offset l
, offset 2. Offset 3 is addr O, addr l at the execution restart location.

addr 2. This is the offset up to addr 3. The processor on which this instruction was executed is instructed to resume execution
Wait until 09 is input.

As shown in FIG. 9, when the execution restart instruction 109 is input (901), the processor control signals 105a to 108b
902), the offset to the execution restart location is added to the PC (program counter), and execution is resumed from the obtained address.

The third example is a software method;
A flowchart is shown in Figure 0.

First, the system waits until the execution restart instruction 109 is manually issued.

When the execution restart instruction 109 is input (1001), the processor control signals 105a to 108b are judged and branched (1002), and the process jumps to the address of the execution restart location.
Resume execution.

Next, FIG. 11 shows a program example of a forffi loop processed by the parallel processing system 10O.

vinegar.

This program 1100 is a [4 pieces = b [2 pieces, b [4] = a [1 piece.

a [5] = b [3 pieces, b [5 pieces =
a [2].

a [6 pieces = b [4], b [6] =
a [3].

a [1000] = b [998], b
[1000] = a [997 pieces a [1001 pieces =
b [999] b [1001 pieces = a [
998].

Since there is a data dependency relationship between b [k coni a [k-3] when j = and a [k + 2] - b [k] when j = +2, a [k + 2] = b The execution of [k] must occur after the execution of b[k a [k-3]. Therefore, the loop processing for j= and j
=+2 loop processing cannot be executed in parallel. Therefore, two instructions for loop processing at j= and two instructions for loop processing at +1 for j= are executed in parallel.

As described above, by executing the loop processing for j= and the loop processing for +1 for j= in parallel, each processor 1
~ In a program divided into 4, program 1100
The inner loop in is no longer needed.

Programs 1101 to 1104 for each processor 1 to 4 are generated by a compiler in the same way as in the previously described embodiment.

Programs 1101 to 11 in each processor 1 to 4
The execution of 04 is as follows.

First, processors 1, 2, 3.4 input the first bits A1 and B1, the second bits A2 and B2, the third bits A3 and B3 . Clear the fourth bit A4 and B4.

Next, processors 1, 2, 3.4 enter a loop and write the first bit Δ1, second bit A2 . . . of register A101, respectively. Third bit A3. The fourth bit A4 is set and the waiting process begins.

As mentioned in the embodiment of the while-type loop, the register A determination circuit 103 checks all bits A of the register A101.
When 1 to A4 are set, an execution restart instruction 109 is issued to each processor 1 to 4.

The register B determination circuit 104 determines each bit B1 to B4 of the register BIO2 and outputs the processor control signal 105a.
~108b is output. That is, in the first loop processing, each bit B1 to B4 of register B102 are all set to °.
O゛, so addr to all processors 1 to 4
Instructs to resume execution from O'.

Processors 1, 2, 3.4 resume execution from 'addr O' and read the first bit AI, second bit A2 . 3rd bit A3゜4th
Clear bit A4.

Then, processor 1 has a [4] = b [2], processor 2 has a [4] = b [11, processor 3
is a[5]=b[3], processor 4 is a[5]=b
Execute [2] and enter the second loop process.

In the second loop, similarly to the first loop processing, processor 1 calculates a [6] = b [4], processor 2
is b [6] - a [3], processor 3 is a [7]
= b [5], processor 4 has b [7] = a [
4].

Thereafter, the processing proceeds in the same manner, and the program 1101
．． The control variable j of the loop of 1102 is 1002, and the control variable j of the loop of programs 1103 and 1104 is 1003.
, processor 1゜2.3.4 ends the loop and writes register B102 and register A101, respectively.
The first bit B1 and Al, the second bit B2 and A
2. Third bit B3 and A3. The fourth bit B4 and A4 are set and the waiting process begins.

The register A determination circuit 103 issues an execution restart instruction 109 to each of the processors 1 to 4 when all bits A1 to A4 of the register Al01 are set.

The register B determination circuit 104 codes the values of each bit B1 to B4 of the register BIO2 according to the table in FIG. 2, and (Bl, B2.B3, B4) = (1,
1,1,1), so processor 1 has a degree of 11 degrees, processor 2... 3.4, processor control signals 105a to 108b of 10° are output.

Processor 1 resumes execution from addr 3' because °11° is specified as the execution restart location,
Finish the process. In addition, processors 2 and 3゜4 ad
Execution is restarted from dr 2° and the process is ended.

As described above, the parallel processing system 100 of the present invention enables parallel execution of both while-type loops and for-type loops.

[Effect of the invention] According to the invention, not only for-type loops.

Even for a while type loop, it becomes possible to perform parallel processing while synchronizing a plurality of processors. Therefore, there is an effect that the processing speed due to parallel execution of programs can be improved more than before.

[Brief explanation of the drawing]

Fig. 1 is a block diagram of a parallel processing system according to an embodiment of the present invention, Fig. 2 is a diagram showing the relationship between the input and output of the register B judgment circuit shown in Fig. 1, and Fig. 3 includes a while type loop. An explanatory diagram of a program and a parallelized program, FIG. 4 is an example of array data, FIG. 5 is a flow diagram of an example of a queuing process, FIG. 6 is a format diagram of an example of a queuing command, and FIG. A flow diagram of another example of the waiting process,
FIG. 8 is a format diagram of another example of a waiting instruction, and FIG.
FIG. 10 is a flowchart of yet another example of the meeting process, FIG. 10 is a flowchart of still another example of the meeting process, and FIG.
The figure is an explanatory diagram of a program including a f o ru small loop and a program that parallelizes the program. (Explanation of symbols) 100...Parallel processing system 101...Register A 102-...Register B 103...Register A judgment circuit 104...Register B judgment circuit 105a, 105b...To processor 1 Processor control signals 106a, 106b...Processor control signals 107a, 107b...Processor control signals 108a, 108b...Processor control signals 109 to processor 4...Instruction to resume execution.

Claims (1)

[Claims] 1. In a synchronization processing method in which processors are made to wait with each other in order to harmonize the progress of parallel processing by a plurality of processors, the first step is to determine whether the processing of each processor has reached a predetermined waiting place. information and second information corresponding to the processing results of each processor are held outside the processor for each processor, and each processor waits and synchronizes with other processors based on the first information, and A synchronous processing method characterized in that each processor selects a place to resume execution after queuing based on the second information. 2. A plurality of processors, first register means having a synchronization detection bit assigned to each of the processors and set by the corresponding processor, and a restart location assigned to each of the processors and set by the corresponding processor. a second register means having a designation bit; a synchronization determination means for determining synchronization from the state of the synchronization detection bit of the first register means and instructing each processor to wait or resume execution; 1. A parallel processing system comprising: execution restart location designating means for designating a restart location for each processor based on the state of a restart location designation bit of the register means. 3. In a parallel processing method in which processing is executed in parallel by a plurality of processors, the step of outputting first information indicating that the processing of each processor has reached a predetermined meeting place to the outside of the processor; outputting second information corresponding to the result to the outside of the processor; waiting or resuming processing based on the first information output from another processor; A parallel processing method, comprising: selecting a location to resume execution based on the second information. 4. In a parallelization program generation device that generates a program for parallel processing from a program for sequential processing, the step of outputting first information indicating that the processing has reached a predetermined waiting place of the program to the outside of the processor is performed for parallel processing. means for adding to the program a step for outputting second information corresponding to a processing result in the program to the outside of the processor; A means for adding a step for queuing processing or resuming execution based on information into a parallel processing program, and a step for selecting a place for resuming execution based on the second information output from another processor for parallel processing. 1. A parallelization program generation device characterized by comprising: a means for adding information into a program for use in the program;