JP2000200254A - System and method of parallel processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decrease the frequency of data communication and to improve the process speed of a parallet processing system on the whole by sending data other than the data whose instantaneous reception is requested by a computer to a target computer through another computer. SOLUTION: A computer 1 saves the arithmetic results of computers 3 to 5 in a storage device by repeating an (n)th process. Those arithmetic results, however, are the arithmetic results of the computers 3 to 5 obtained by repeating an (n-4)th process, so the computers 3 to 5 need not send their arithmetic results to the computer 1 every time the process is performed. Then, data which should be sent on each repetitive process so as to avoid trouble to the processes of other computers are sent immediately to other computers after the arithmetic process ends, but other data causing no trouble to other computing processes are sent at a time together with other data. When a computer sends data, the data other than data whose instantaneous reception is requested by a computer are sent to the target computer through another computer.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel processing system and a parallel processing method in which a plurality of computers mutually connected via a communication path repeatedly execute parallel processing.

[0002]

2. Description of the Related Art FIG. 21 is a block diagram showing a conventional parallel processing system. In FIG. 21, reference numeral 11 denotes a transmission function for transmitting data to another computer, a reception function for receiving data from another computer, and a predetermined operation. A computer having an arithmetic function of executing data and a function of storing data, 12 to 15 have a transmission function of transmitting data to another computer, a reception function of receiving data from another computer, and an arithmetic function of executing a predetermined operation. Computers A to J have communication lines connecting between the two computers.

Next, the operation of the conventional parallel processing system will be described. Each of the computers 11 to 15 repeatedly executes parallel processing by executing a predetermined operation by using an operation result of another computer. Therefore, the computers 11 to 15 repeat predetermined operation processing while receiving data necessary for the operation from another computer. Hereinafter, a parallel processing system in which the operation results of each computer are collected and stored in the computer 11 will be described as an example.

[0004] 1. A. n-th repetitive processing in the computer 11 (n = 1, 2, 3,...) Receiving process-Receive the calculation result of the computer 12 executed in the (n-1) th repetition process from the computer 12. -Receive the calculation result of the computer 13 executed in the (n-1) th repetition process from the computer 13. Receive the calculation result of the computer 14 executed in the (n-1) th repetition process from the computer 14. -Receive the calculation result of the computer 15 executed in the (n-1) th repetition process from the computer 15.

B. Saving process The calculation result of the (n-4) th iterative process in the computer 11, the computer 12, the computer 13, the computer 14, and the computer 15 is stored in the storage device.

C. Arithmetic Processing A predetermined arithmetic operation is performed using the arithmetic result of the (n-1) th iterative processing in the computer 11 and the arithmetic result of the (n-1) th iterative processing in the computer 12. D. Transmission processing: The calculation result of the n-th repetition processing in the computer 11 is transmitted to the computer 12.

[0007] 2. A. n-th repetition processing in the computer 12 (n = 1, 2, 3,...) Receiving process-Receives the calculation result of the computer 11 executed in the (n-1) th iterative process from the computer 11. -Receive the calculation result of the computer 13 executed in the (n-1) th repetition process from the computer 13. -Receive the calculation result of the computer 15 executed in the (n-1) th repetition process from the computer 15.

B. Arithmetic processing: The arithmetic result of the (n-1) th iterative processing in the computer 11, the arithmetic result of the (n-1) th iterative processing in the computer 12, the arithmetic result of the (n-1) th iterative processing in the computer 13, and the computer 15 A predetermined calculation is executed using the calculation result of the (n-1) th repetition process in. C. Transmission processing • The calculation result of the n-th repetition processing in the computer 12 is transmitted to the computer 11. The operation result of the n-th iterative process in the computer 12 is transmitted to the computer 13. The calculation result of the n-th repeated geography in the computer 12 is transmitted to the computer 15.

3. A. n-th repetition processing in the computer 13 (n = 1, 2, 3,...) Receiving process-Receive the calculation result of the computer 12 executed in the (n-1) th repetition process from the computer 12. Receive the calculation result of the computer 14 executed in the (n-1) th repetition process from the computer 14.

B. Arithmetic processing: Using the arithmetic result of the (n-1) th iterative process in the computer 12, the arithmetic result of the (n-1) th iterative process in the computer 13, and the arithmetic barrier of the (n-1) th iterative process in the computer 14 , Execute a predetermined calculation. C. Transmission processing: The calculation result of the n-th repetition processing in the computer 13 is transmitted to the computer 11. The operation result of the n-th iterative process in the computer 13 is transmitted to the computer 12. The operation result of the n-th iterative process in the computer 13 is transmitted to the computer 14.

4. A. n-th repetition processing in the computer 14 (n = 1, 2, 3,...) Receiving process Receives the calculation result of the computer 13 executed in the (n-1) th repetition process from the computer 13.

B. Arithmetic Processing A predetermined arithmetic operation is performed using the arithmetic result of the (n-1) th iterative processing in the computer 13 and the arithmetic result of the (n-1) th iterative processing in the computer 14. C. Transmission processing: The calculation result of the n-th repetition processing in the computer 14 is transmitted to the computer 11. The operation result of the n-th iterative process in the computer 14 is transmitted to the computer 13.

5. A. n-th repetition processing in the computer 15 (n = 1, 2, 3,...) Receiving process-Receive the calculation result of the computer 12 executed in the (n-1) th repetition process from the computer 12.

B. Arithmetic Processing: A predetermined arithmetic operation is performed using the arithmetic result of the (n−1) th iterative processing in the computer 12 and the arithmetic result of the (n−1) th iterative processing in the computer 15. C. Transmission processing: The calculation result of the n-th repetition processing in the computer 15 is transmitted to the computer 11. The operation result of the n-th iterative process in the computer 15 is transmitted to the computer 12.

Here, for convenience of explanation, the computers 11 to 15
TC1, TC2, TC
3, TC4, and TC5, and assume that the transmission time of data transmitted from the computers 11 to 15 to other computers is uniformly TS regardless of the data amount. It is also assumed that the reception time required for the computers 11 to 15 to receive data from other computers is TR, and the storage time required for the computer 11 to store data in the storage processing is TM. However, it is assumed that the delay time from when the computers 11 to 15 transmit data to when other computers receive data is negligibly small. Here, for convenience of explanation, the relationship between TR and TS is TR = 3TS.

Assuming the time of each processing as described above,
The time required for one repetition process in the computers 11 to 15 is as follows. Computer 1: 4TR + TM + TC1 + TS Computer 2: 3TR + TC2 + 3TS Computer 3: 2TR + TC3 + 3TS Computer 4: TR + TC4 + 2TS Computer 5: TR + TC5 + 2TS

Therefore, if TC = TC1 = TC2
Assuming that = TC3 = TC4 = TC5, the processing load of the computer 1 becomes the largest, and the execution speed of the parallel processing system is determined by the processing time of the computer 1.

[0018]

Since the conventional parallel processing system is configured as described above, the computers 11 to 1
5 can execute predetermined arithmetic processing in parallel if data can be received from another computer.
No. 15 needs to receive all necessary data from another computer in one repetition process. Therefore, when the required data increases, the number of times of data communication increases,
There were problems such as a reduction in the processing speed of the entire system.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a parallel processing system and a parallel processing method capable of reducing the number of data communications and increasing the processing speed of the entire system. Aim.

[0020]

A parallel processing system according to a first aspect of the present invention comprises: a transmitting unit for transmitting data to another computer; a receiving unit for receiving data from another computer; And a plurality of calculators that repeatedly execute predetermined processing.
When each computer transmits data to a target computer, if the computer is data other than the data for which immediate reception is requested, the data is transmitted to the target computer via another computer.

In the system having the above configuration, each computer stores in advance information such as the position of data in a communication packet transmitted to and received from another computer and the necessity of immediate transfer of data. .

Further, in the system having the above-mentioned configuration, a communication device having a transmitting function and a receiving function of the computer is connected to each computer.

[0023] A parallel processing method according to a second aspect of the present invention comprises a transmitting means for transmitting data to another computer, a receiving means for receiving data from another computer, and an arithmetic means for executing a predetermined operation. It has to be executed by using a plurality of computers that repeatedly execute a predetermined process, and when transmitting data to each computer to a target computer, the computer requires data other than data that requires immediate reception. When the data is data, the data is transmitted to the target computer via another computer. .

In the above-mentioned processing method, each computer stores in advance information such as the position of data in a communication packet transmitted / received to / from another computer and the necessity of immediate transfer of data.

Further, in the above-mentioned processing method, a communication device having a transmitting function and a receiving function of the computer is connected to each computer.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Some embodiments of the present invention will be described below. Embodiment 1 FIG. FIG. 1 is a configuration diagram showing a parallel processing system according to Embodiment 1 of the present invention. In the drawing, reference numeral 1 denotes a transmission function for transmitting data to another computer, a reception function for receiving data from another computer, and a predetermined function. A computer having an operation function of executing an operation and a function of storing data,
Reference numeral 5 denotes a computer having a transmission function of transmitting data to another computer, a reception function of receiving data from another computer, and an arithmetic function of executing a predetermined operation, and A to J denote communication lines connecting between the two computers. It is. FIG. 2 is a flowchart showing the parallel processing method according to the first embodiment of the present invention, and schematically shows the repetitive processing in each computer.

Next, the operation will be described. Computers 1-5
Is to execute parallel processing repeatedly by executing a predetermined operation by using an operation result of another computer. Therefore, the computers 1 to 5 repeat predetermined operation processing while receiving data necessary for the operation from another computer.

Hereinafter, a parallel processing system in which the operation results of each computer are collected and stored in the computer 1 will be described with reference to FIGS. 1 and 2 (the final processing result of the parallel processing system is described in the column of the conventional example). Is the same as the parallel processing system
The processing contents of each computer in the repeated processing are different.)

1. A. n-th repetition process in computer 1 (n = 1, 2, 3,...) Reception processing From the computer 2, the calculation result of the computer 2 executed in the (n-1) th iteration processing, the calculation result of the computer 3 executed in the (n-2) th iteration processing, and the (n-3) th iteration processing And the calculation result of the computer 5 executed in the (n−2) -th repetition processing is received.

B. Save process-The calculation results of the (n-4) th iterative process in the computer 1, the computer 2, the computer 3, the computer 4, and the computer 5 are stored in the storage device.

C. Arithmetic Processing A predetermined arithmetic operation is performed using the arithmetic result of the (n−1) th iterative processing in the computer 1 and the arithmetic result of the (n−1) th iterative processing in the computer 2. D. Transmission processing • The calculation result of the n-th repetition processing in the computer 1 is transmitted to the computer 2.

2. The n-th repetitive processing in the computer 2 (n = 1, 2, 3,...) Receiving process Receives the calculation result of the computer 1 executed in the (n-1) th repetition process from the computer 1. Receive the calculation result of the computer 3 executed in the (n-1) th repetition process and the calculation result of the computer 4 executed in the (n-2) th repetition process from the computer 3. Receive the calculation result of the computer 5 executed in the (n-1) th repetition process from the computer 5.

B. Arithmetic Processing • The arithmetic result of the (n-1) th iterative processing in the computer 1, the arithmetic result of the (n-1) th iterative processing in the computer 2, the arithmetic result of the (n-1) th iterative processing in the computer 3, and the computer 5 A predetermined calculation is executed using the calculation result of the (n-1) th repetition process in. C. Transmission processing: The calculation result of the n-th repetition processing in the computer 2, the calculation result of the (n-1) -th repetition processing in the computer 3, the calculation result of the (n-2) -th repetition processing in the computer 4, and the n in the computer 5 The calculation result of the first iteration is transmitted to the computer 1. -The calculation result of the n-th iterative process in the computer 2 is transmitted to the computer 3. The operation result of the n-th iterative process in the computer 2 is transmitted to the computer 5.

3. A. n-th repetition process in the computer 3 (n = 1, 2, 3,...) Receiving process Receives the calculation result of the computer 2 executed in the (n-1) th repetition process from the computer 2. -Receive the calculation result of the computer 4 executed in the (n-1) th repetition process from the computer 4.

B. Arithmetic processing: Using the arithmetic result of the (n-1) th iterative processing in Computer 2, the arithmetic result of the (n-1) th iterative processing in Computer 3, and the arithmetic result of the (n-1) th iterative processing in Computer 4 , Execute a predetermined calculation. C. Transmission processing The calculation result of the n-th repetition processing in the computer 3 and the calculation result of the (n-1) -th repetition processing in the computer 4 are transmitted to the computer 2. The operation result of the n-th iterative process in the computer 3 is transmitted to the computer 4.

4. A. n-th repetitive processing in the computer 4 (n = 1, 2, 3,...) Receiving process Receives the calculation result of the computer 3 executed in the (n-1) th iterative process from the computer 3.

B. Arithmetic Processing A predetermined arithmetic operation is performed using the arithmetic result of the (n-1) th iterative processing in the computer 3 and the arithmetic result of the (n-1) th iterative processing in the computer 4. C. Transmission processing • The calculation result of the n-th repetition processing in the computer 4 is transmitted to the computer 3.

5. A. n-th repetition process in the computer 5 (n = 1, 2, 3,...) Receiving process Receives the calculation result of the computer 2 executed in the (n-1) th repetition process from the computer 2.

B. Arithmetic Processing A predetermined arithmetic operation is performed using the arithmetic result of the (n-1) th iterative processing in the computer 2 and the arithmetic result of the (n-1) th iterative processing in the computer 5. C. Transmission processing: The calculation result of the n-th repetition processing in the computer 5 is transmitted to the computer 2.

As described above, the final processing result of the parallel processing system is the same as that of the parallel processing system described in the column of the conventional example (see the storage processing in the computer 1). Are different in the processing contents of each computer.

The reason why the processing content of each computer is different is to reduce the number of times of communication between the computers, and is specifically as follows. That is, in the conventional parallel processing system,
For example, Calculator 1 needs the calculation result of Computer 2 executed in the (n-1) th iteration processing in the n-th iteration processing, and therefore, Computer 2 transmits the calculation result to Computer 1 for each iteration processing. There is a need to. On the other hand, computer 1
Executes the storage process of storing the calculation results of the computers 3, 4, and 5 in the storage device in the n-th iterative process. Because of the calculation results of 4, 5
4 and 5 do not need to transmit the calculation results to the computer 1 every time the processing is repeated. Therefore, if the data (operation result) that hinders the processing of another computer is not transmitted for each repetition processing, the data (operation result) is transmitted to the other computer immediately after the completion of the operation processing. Also, data (operation result) that does not hinder the processing of other computers is transmitted together with other data (operation result) to reduce the number of times of communication.

The calculation results that need to be transmitted for each repetition process are as follows. Indispensable communication set ・ Computer 1 → Computer 2 (Computation result of Computer 1 in the nth iteration) ・ Computer 2 → Computer 1 (Computation result of Computer 2 in the nth iteration) ・ Computer 2 → Computer 3 (Nth (Calculation result of Computer 2 in the repetition processing of (2))-Computer 2 → Computer 5 (Calculation result of Computer 2 in the n-th repetition processing)-Computer 3 → Computer 2 (Calculation result of Computer 3 in the n-th repetition processing)-Computer 3 → computer 4 (calculation result of computer 3 in n-th iteration) ・ computer 4 → computer 3 (computation result of computer 4 in n-th iteration) ・ computer 5 → computer 2 (computer in n-th iteration) 5, the arrow indicates that communication is being performed from the left computer to the right computer.

The calculation results that do not need to be transmitted for each repetition process are as follows. Calculator 3 → Calculator 2 → Calculator 1 (calculation result of Calculator 3 in the (n−4) th iteration processing) Calculator 4 → Calculator 3 → Calculator 2 → Calculator 1 (calculation result of Computer 4 in the (n−4) th iteration processing Calculator 5 → Computer 2 → Computer 1 (Calculation result of Computer 5 in the (n−4) th iterative process) For example, the calculation result of Computer 3 is calculated from Computer 3 to Computer 2
Is transmitted to the computer 1 via the.

With this, each computer can transmit the operation result that does not need to be transmitted for each repetition process to the target computer via another computer from a margin of time. When it is necessary to always send the calculation result to the computer (when sending the calculation result that needs to be sent for each iteration), it can be included in the calculation result and sent at the same time, and as a result, the system The total number of communications can be reduced.

Here, the time required for one repetition processing of the computers 1 to 5 in the first embodiment is as follows. Computer 1: TR + TM + TC1 + TS Computer 2: 3TR + TC2 + 3TS Computer 3: 2TR + TC3 + 2TS Computer 4: TR + TC4 + TS Computer 5: TR + TC5 + TS

Therefore, if TC = TC1 = TC2
= TC3 = TC4 = TC5 (where T
M is a time sufficiently shorter than TR or the like), the processing load of the computer 2 becomes the largest, and the execution speed of the parallel processing system is determined by the processing time of the computer 2.
Therefore, the time required for one repetition process is 3
TR + TC + 3TS, which is shorter than 4TR + TM + TC + TS, which is the time required for one repetitive processing in the conventional parallel processing system, and the processing speed of the entire system is increased.

Embodiment 2 In the first embodiment, the calculation result that does not need to be transmitted for each repetition process is transmitted to another computer via another computer. However, the computer that receives the calculation result is destined for itself. It is necessary to be able to identify whether it is data (calculation result) or data (computation result) to be transferred to another computer.

Although the first embodiment does not particularly refer to a data identification method, generally, header information (data attribute information such as data destination information and data amount) is included in transmission data (calculation result). Is added, and transmission data and header information are transmitted in pairs.

FIGS. 3 to 10 show data structures transmitted and received between the computers. It should be noted that the number in each figure indicates the number of the byte in the data where the data is stored. Each data shown in these figures will be described below. Data 1-2-c Data required for calculation in the (n + 1) th iterative process in Computer 2 (Computer 1 → Computer 2)

Data-2--1-c Data necessary for calculation in the (n + 1) th iterative process in the computer 1 (computer 2 → computer 1) data-2--1-r Saved in the (n + 4) th iterative process in the computer 1 Necessary for (computer 2 → computer 1) data-3-1-r Data necessary for storage in the (n + 4) th iterative process in computer 1 (computer 3 → computer 1) data-4-1-r computer 1 Necessary for storage in the (n + 4) th repetition process in (computer 4 → computer 1) data-5-1-r Data required for storage in the (n + 4) th repetition process in computer 1 (computer 5 → computer 1)

Data-2--3-c Data necessary for the calculation in the (n + 1) th iterative process in the computer 3 (computer 2 → computer 3) data-3--2-c Compute in the (n + 1) th iterative process in the computer 2 Data required for computer (Computer 3 → Computer 2)

Data-3--4-c Data necessary for the operation in the (n + 1) th iterative process in the computer 4 (computer 3 → computer 4). Data-4--3-c Perform the operation in the (n + 1) th iterative process in the computer 3. Data required for computer (computer 4 → computer 3)

Data-2-5-c Data necessary for the operation in the (n + 1) -th iterative process in the computer 5 (computer 2 → computer 5) data-5-2-c Perform the operation in the (n + 1) -th iterative process in the computer 2 Data required for computer (computer 5 → computer 2)

However, in this method, each computer needs to refer to the header information to identify the destination of the data and the attribute of the data (such as the amount of data) for each communication, so that the load on each computer is large. However, the processing speed of the entire system was reduced. However, the communication required for the parallel calculation using the repetition processing often involves the same computer repeating the same amount of data at the same timing.

Therefore, in the second embodiment, in the parallel calculation using such an iterative process, before starting the iterative process, which computer sends in advance which amount of data to which computer and which timing The position / transfer information indicating whether or not to transmit is shared between the computers (see FIG. 11). Thus, when a communication packet (data group) is transmitted from a certain computer during the execution of the repeated calculation, the transfer destination of each data constituting the communication packet can be immediately recognized without referring to the header information. Can be executed promptly. In addition, since it is not necessary to include information about a destination computer or the like in the transmission data, the amount of data to be transmitted is reduced, and the processing speed of the entire system can be improved.

FIGS. 12 to 19 show data received by each computer. As is clear from the figures, FIGS.
The header information is not added to the data, and only the so-called data (calculation result) is transmitted.

Embodiment 3 In the first and second embodiments, each computer performs transmission processing and reception processing in addition to arithmetic processing. However, as illustrated in FIG. 20, each computer performs transmission processing and reception processing of computers 1 to 5. Communication devices 6 to 10
May be connected to the computers 1 to 5. Thereby, the computers 1 to 5 do not execute the transmission processing and the reception processing (the communication apparatuses 6 to 10 are replaced with the computers 1 to 5,
The transmission processing and the reception processing are executed), and the arithmetic processing can be executed, so that the loads on the computers 1 to 5 are reduced, and the processing speed of the entire system is improved.

Embodiment 4 In the first to third embodiments, each computer is connected to another computer via the communication lines A to J. However, the present invention is not limited to this. For example, each computer is connected via a bus-type network. It goes without saying that they may be connected.

[0059]

As described above, according to the present invention, when each computer transmits data to another computer, data other than the data that the other computer requests to receive immediately passes through another computer. Since it is configured to transmit data to another computer, the number of times of data communication can be reduced, and as a result, a parallel processing system capable of high-speed processing can be obtained.

According to the present invention, each computer stores in advance the position and transfer information of the data in the communication packet transmitted / received to / from another computer, so that the destination of the data can be stored without adding header information. Can be recognized, and as a result, a parallel processing system capable of high-speed processing can be obtained.

According to the present invention, since a communication device having a transmission function and a reception function of a computer is connected to each computer, the load on the computer is reduced and the processing speed of the parallel processing system is improved. is there.

According to the present invention, when each computer transmits data to another computer, the other computer transmits data other than the data for which immediate reception is requested to another computer via another computer. With such a configuration, there is an effect that the number of data communications can be reduced and the processing speed of the entire system can be increased.

According to the present invention, each computer stores in advance the position and transfer information of data in a communication packet transmitted / received to / from another computer, so that the destination of the data can be stored without adding header information. Can be recognized, and as a result, the processing speed of the entire system can be further increased.

According to the present invention, since a communication device having a transmission function and a reception function of a computer is connected to each computer, the load on the computer is reduced and the processing speed of the entire system can be increased. There is.

[Brief description of the drawings]

FIG. 1 is a configuration diagram illustrating a parallel processing system according to a first embodiment of the present invention;

FIG. 2 is a flowchart illustrating a parallel processing method performed by the parallel processing system according to the first embodiment of the present invention;

FIG. 3 is a data structure diagram showing a communication data structure transmitted from the computer 1 to the computer 2.

FIG. 4 is a data structure diagram showing a communication data structure transmitted from the computer 2 to the computer 1.

FIG. 5 is a data structure diagram showing a communication data structure transmitted from the computer 2 to the computer 3.

FIG. 6 is a data structure diagram showing a communication data structure transmitted from the computer 3 to the computer 2.

FIG. 7 is a data structure diagram showing a communication data structure transmitted from the computer 3 to the computer 4.

FIG. 8 is a data structure diagram showing a communication data structure transmitted from the computer 4 to the computer 3.

FIG. 9 is a data structure diagram showing a communication data structure transmitted from the computer 2 to the computer 5.

FIG. 10 is a data structure diagram showing a communication data structure transmitted from the computer 5 to the computer 2.

FIG. 11 is a table showing data position / transfer information.

FIG. 12 is a data structure diagram showing a communication data structure transmitted from the computer 1 to the computer 2.

FIG. 13 is a data structure diagram showing a communication data structure transmitted from the computer 2 to the computer 1.

FIG. 14 is a data structure diagram showing a communication data structure transmitted from the computer 2 to the computer 3.

FIG. 15 is a data structure diagram showing a communication data structure transmitted from the computer 3 to the computer 2.

FIG. 16 is a data structure diagram showing a communication data structure transmitted from the computer 3 to the computer 4.

FIG. 17 is a data structure diagram showing a communication data structure transmitted from the computer 4 to the computer 3.

FIG. 18 is a data structure diagram showing a communication data structure transmitted from the computer 2 to the computer 5.

FIG. 19 is a data structure diagram showing a communication data structure transmitted from the computer 5 to the computer 2.

FIG. 20 is a configuration diagram showing a parallel processing system according to a third embodiment of the present invention.

FIG. 21 is a configuration diagram showing a conventional parallel processing system.

[Explanation of symbols]

 1-5, 11-15 Computer, 6-10 Communication device.

Claims (8)

[Claims] An apparatus comprising: a transmitting unit that transmits data to another computer; a receiving unit that receives data from another computer; and an arithmetic unit that performs a predetermined operation, and includes a plurality of units that repeatedly execute a predetermined process. When a computer transmits data to a target computer, if the computer is data other than the data for which immediate reception is requested, each computer transmits the data to the target computer via another computer. A parallel processing system for transmitting to a computer. 2. The parallel processing system according to claim 1, wherein a communication device having a transmission function and a reception function of the computer is connected to each computer. 3. The parallel computer according to claim 1, wherein each computer stores in advance information such as a position of data in a communication packet transmitted / received to / from another computer and an immediate necessity of data transfer. Processing system. 4. The parallel processing system according to claim 3, wherein a communication device having a transmission function and a reception function of the computer is connected to each computer. 5. A plurality of transmitting means for transmitting data to another computer, receiving means for receiving data from another computer, and operating means for executing a predetermined operation, wherein the processing means repeatedly executes a predetermined process. In a parallel processing system executed by using a computer, when transmitting each computer data to the target computer, if the computer is data other than the data that requires immediate reception, via another computer A parallel processing system for transmitting to the target computer. 6. The parallel processing method according to claim 5, wherein a communication device having a transmission function and a reception function of the computer is connected to each computer. 7. The parallel computer according to claim 5, wherein each computer stores in advance information such as a position of data in a communication packet transmitted / received to / from another computer and an immediate necessity of data transfer. Processing method. 8. The parallel processing method according to claim 7, wherein a communication device having a transmission function and a reception function of the computer is connected to each computer.