JPH11110362A - Method for communicating data between computers - Google Patents

Abstract

PROBLEM TO BE SOLVED: To execute the reduction in the number of times of communication or communication processing time while suppressing the required resources to a minimum by performing communication in certain order so that a computer to be communication waiting can not appear in each phase. SOLUTION: Each computer defines a computer having a value, for which the rank number of the computer itself is subtracted from N-1 (5), as a rank number as a communicating party in a phase 0. After a phase 1, each computer defines a computer adding '1' to the communicating party in the preceding phase as the communicating party. In this case, when a value adding '1' becomes N(6), it is returned to 0. This is repeated to a phase N-1. Therefore, the communication for totally N phases from the phase 0 to the phase N-1 is performed. When a computer (a) defines a computer (b) as the communicating party in each phase, the relation that the computer (b) also defines the computer (a) as the communicating party is established. Therefore, the communication processing of each computer can be simultaneously executed and mutual data transmission can be executed while being overlapped.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data communication method between a plurality of computers connected by a network including a parallel computer, and more particularly to a message passing interface (hereinafter referred to as a standard interface for data communication between computers). , MPI) for speeding up overall communication.

[0002]

[Prior art]

(Communication method of parallel computer) A distributed memory type parallel computer is a computer of a method in which computers called nodes are connected by a high-speed network and calculations are performed in parallel while exchanging each other's data using the high-speed network communication. There are various methods for high-speed networks, but one method allocates a specific memory area for high-speed communication, and the sending node writes data directly to the high-speed communication memory of the receiving node to reduce the OS overhead of the receiving node. Reduce and realize high-speed communication. In this case, by allocating the data that needs to be transmitted / received to the high-speed communication area and allocating the other data to the normal memory area, it is possible to perform processing that effectively uses the high-speed network.

When the high-speed network supports two-way communication, the transmission and reception of data are overlapped in the process of exchanging data with each other, thereby further improving the communication performance. Becomes possible.

(Communication method using message passing library) There are various models for programming for performing parallel processing in such a distributed memory type parallel computer, and the most common model is a message model. It is a passing model. In the message passing model, each computer performs calculations independently of each other, and if necessary, exchanges data explicitly by using a communication function in the form of message transmission and reception. Recently, a specification of a message passing function called MPI (Message Passing Interface) has been established by the MPI Forum to unify functions and interfaces of communication functions for the purpose of portability of programs and data communication between different types of devices. In MPI, in order to clearly indicate a communication partner, an identification number called a rank number is assigned to each computer, and each communication function specifies a communication partner using this rank number. Hereinafter, a computer to which a rank number n is assigned is referred to as a computer n. In the communication process defined by MPI, in addition to one-to-one communication for performing communication between a pair of computers, general communication for performing data transfer in a predetermined pattern between all computers is defined. Broadcast communication that transfers data from one computer to all other computers, Scatter communication that divides data from one computer and distributes them to other computers, and distributed to all computers Gather communication that collects data in one or all computers, Alltoall communication that divides data in each computer and distributes it to all computers, aggregates data in all computers and transfers it to one or all computers There is Reduce communication.

[0005] In the following, of these communication processes related to the present invention, Gather / Scatter communication, alltoall communication, and re
al which distributes the result among all computers among duce communication
The l-reduce communication will be described.

(Gather / Scatter Communication Pattern) First, gather / scatter communication will be described. Gather / Scatt
As shown in FIG. 8, the er communication collects data distributed to a plurality of computers into a computer called a root to form a group of data (Gather), or divides a group of data in one computer. This is a process of distributing to a plurality of computers.

A general processing method in Gather / Scatter communication will be described using Gather communication as an example. Here, in order to simplify the explanation, the number of computers is four, and the root computer is computer 0.

FIG. 9 shows a communication pattern of gather communication. First, the root computer 801 copies its own transmission data 810 to the reception buffer 820. At this time, the other computers 802 to 804 issue a data transmission function to the root computer 801. Next, the root computer 80
1 issues a reception function to the computer 1 (802) and receives the data 811 to the reception buffer 820. The root computer 801 performs the same processing for all computers, and collects data in the reception buffer 820.

[0009] By such processing, the root computer terminates the Gather processing by performing the communication of the number of computers minus one time and the processing of one memory copy of itself. This method is called a sequential processing method.

(Higher-speed Gather / Scatter communication method) On the other hand, as a means for reducing the number of times of transmission / reception, there is a method using a binary tree. FIG. 10 shows a method of realizing Gather communication using a binary tree. FIG. 10 shows an example in which computer 0 is used as a root computer for parallel programs of four computers and Gather communication is performed. In the binary tree method, in addition to the buffers 810 to 813 for storing transmission data and the buffer 820 for storing reception data, a work buffer 103 for each computer is provided.
0 to 1033 are prepared. The required work buffer capacity differs for each computer, but calculating the required capacity in advance for each computer takes a long time, so all computers need the maximum amount of data, that is, the data that the root computer finally receives. Allocate a work buffer corresponding to the amount. And
As the first phase, the computers 801 to 804 copy the transmission data in the transmission data storage buffers 810 to 813 to the working buffers 1030 to 1033. The subsequent data communication is performed on the data in the working buffer.
Next, in the second phase, computer 1 (802)
01), the computer 3 (804) transmits data to the computer 2 (803). Next, in the third phase, the computer 0
(801) and computer 2 (803) are communicated, and computer 2 (803) and computer 3 possessed by computer 2 (803).
The data of (804) is transmitted to the computer 0 (801). Thus, the working buffer 1 of the computer 0 (801), which is the root,
030 means that all data has been collected. Finally, the root computer 801 copies the data in the working buffer 1030 to the receiving buffer 820, and the Gather communication process is completed. In this method, when the Gather processing is performed by n computers, the processing can be performed in log (n) phases, and the processing can be performed in a shorter time than the first method.
However, in this method, since it is necessary to prepare a working buffer area in each computer, an extra memory resource is used when the data amount is large. Further, since data is copied to the working buffer at the beginning and end of the processing, overhead is required.

(Communication pattern of Alltoall) Next, Alltoall
Communication will be described. Alltoall communication is based on the Ga
This is a process of performing ther or Scatter communication for all computers, and is a communication process in which all computers divide their own data and distribute the data to other computers.

In Alltoall, each computer transmits data to all computers including itself. An example in which four computers perform Alltoall communication will be described with reference to FIG. 440
443 indicates a computer, and 400 to 433 indicate data transmission between the computers. Computers 440, 441, 442, and 443 are respectively assigned numbers 0, 1, 2, and 3 as identifiers. In MPI, this is called a rank number (hereinafter, "a computer assigned a rank number n"
"Calculator n"). The transmission 4ab indicates data transmission from the computer a to the computer b. FIG. 4A shows a state in which the computer 440 transmits data to all computers including itself. In Alltoall, since all the computers perform the same data transmission, the result is as shown in FIG. Here, if bidirectional communication can be used between the computers, transmission 4ab
And transmission 4ba can be executed in an overlapping manner.

(Implementation of Alltoall by Non-blocking Communication) Generally, the operation of Alltoall is realized by non-blocking communication. In the non-blocking communication, a transmission (reception) request is unilaterally issued to a communication partner, and an actual communication process is performed when the communication partner can receive (transmit). Allt by non-blocking communication
In the oall realization method, each computer first issues a transmission request and a reception request to all computers at once, and when each computer is ready to process the request, the actual communication process is sequentially performed. Do. Because of such an operation, the order in which the transmission processes are executed is undefined.

(Simultaneous Communication System Between All Computers) Japanese Patent Laid-Open Publication No. HEI 9 (1999) discloses a method in which each computer in a parallel computer transmits data to all computers other than itself when each computer can communicate with each other one-to-one. 8-263349. If a process of transmitting data to itself is added to this method, MP
It has the same function as I Alltoall. In this method, unlike the implementation of Alltoall using non-blocking communication in MPI, each computer determines the order of communication partners and performs communication simultaneously for each phase. Thereby, the communication processing of each computer proceeds synchronously and smoothly.

The method will be described. In this method, N computers communicate in 2 ⁿ phases (however, 2 ^n-1 <N ≦
2 ⁿ ). Each computer has a binary number representing a rank number 0 to N-1 assigned to itself, and a phase number 0 to N-1.
The computer to which the exclusive OR of the value expressing 2 ⁿ -1 in a binary number is assigned as the rank number is set as the communication partner for each phase.

[0016] With reference to FIG. 3 (A), 8 units (= 2 ³ units) of the computer will be described an example in which the communication. For example, in phase 2, each computer (0,1,2,3,4,5,6,7) communicates with each computer (2,3,0,1,6,7,4,5) ( The communication partner of each computer is the exclusive OR of its own rank number and the phase number "2". Here, if the computer a has the computer b as the communication partner, the relationship holds that the computer b also has the computer a as the communication partner. Therefore, the communication processing of each computer can be executed at the same time, and data transmission to each other can be executed in an overlapping manner. The same applies to other phases. Since each computer transmits data to all the computers including itself during the eight phases of phase 0 to phase 7, the function is equivalent to that of Alltoall.

Next, referring to FIG. 3B, six units (2 ² <
An example in which the computer of 6 ≦ 2 ³ ) performs communication will be described. For example, in phase 2, each computer (0,1,2,3,4,5) communicates with each computer (2,3,0,1,6,7). Here, the computers (6, 7) which are the communication partners of the computers (4, 5) do not actually exist. Therefore, the computers (4, 5) wait until the communication processing of the other computers (0, 1, 2, 3) ends. As a result, communication of 8 (= 2 ³ ) phases is performed.

In the original patent, since the communication device does not include itself in the communication partner, the communication process in the phase 0 is not performed, and the communication is a 2 ⁿ -1 phase communication.

(Outline of all-reduce communication) Next, reduce communication will be described. Reduce communication uses data owned by all nodes to perform some operation (sum, maximum value, logical sum, etc.)
This is the process of collecting the information by a specific root computer or distributing it to all computers. In particular, all-computers must implement a reduce communication process in which all the
Called reduce communication.

The present invention is based on a computer having two types of memories, a high-speed memory for communication and a normal memory.

The buffer specified by the transmission / reception function is allocated to the communication high-speed memory, but the buffer dynamically allocated at the time of execution is allocated to the normal memory. all-
In reduce, a data buffer in which original data is stored and an operation buffer for storing an operation result are specified. These two buffers are allocated for fast memory.

When all-reduce by n processes is implemented without securing an internal buffer, a sequential method is adopted. Specifically, first, the n-th process transmits data to the (n-1) -th process, and the (n-1) -th process receives the data in the operation buffer. The calculation result of the received data and the original data of the own process is stored in the calculation buffer, and the data is transmitted to the (n-2) th process using the calculation buffer. Thereafter, each process sequentially receives data from other processes, performs an operation with the data of the own process, and transmits the result to the next process, and repeats the phase, and the operation result is stored in the operation buffer of the first process. Is done. Apply the result to all processes
Communication is completed by oadcast. In this method, n phases are required for computation and log (n) phase is required for broadcast.

On the other hand, when the use of the internal buffer is permitted, the communication process can be completed in the log (n) phase by using the Hyper Cube method. The Hyper Cube method is an exclusive logic of the process number n rounded to the power of 2 and the process of the power of 2 assigned to the rank number assigned to itself for each phase and the power of 2 (the number of communication times -1). This is a method of communicating with a computer having a rank number equal to the sum.
The case of five processes will be specifically described with reference to FIG.
Each process is assigned a rank number from 0 to 4 for identification. Each process uses a buffer (601,
602, 603, 604, and 605). Communication usually uses two buffers, one for transmission and one for reception.
Here, one buffer that can transmit and receive is assumed. First, in order to round the number of processes to a power of two, a process of rank 4 transmits data to rank 0 (6
06). As a result, the process 0 is assumed to be the operation result of 0 and 4 (SUM representing the sum is illustrated in the figure). After that, rank 0
The communication is performed between. In the first communication, an exclusive OR of the 2 ^0, communicates rank 0 and 1, 2 and 3 (607, 608), the process 0,1 SUM (0,1,
4), and processes 2 and 3 have SUM (2,3). The second take the exclusive OR of the 2 ^1, the rank 0 and 2,1 and 3 perform communication (609, 610). Finally, the result is transmitted from rank 0 to rank 4, and communication is completed in all processes (611). When Hyper Cube is used, each process requires a reception buffer for receiving data from another process in addition to a data buffer and an operation buffer.
Each process first receives data in the reception buffer, performs an operation between the received data and the original data (the operation result data from the second operation to the previous operation), and stores the result in the operation buffer. In the next phase, the data in the operation buffer is transmitted to the partner process. This receive buffer is not specified by a function argument, but is allocated dynamically at the time of execution, and is therefore usually allocated to memory. Therefore, Hype
The r Cube method can perform communication in a small number of phases, but the communication time per operation is longer than that of the sequential method because a normal memory is used.

[0024]

The following problems are involved in the whole communication of MPI.

(1) All computers are Gather as root computers
For all-to-all communication with / Scatter communication,
When Gather / Scatter communication is performed several times, the processing time increases in proportion to the square of the number N of computers. Further, even when the method shown in the conventional method is used, a communication phase proportional to the number obtained by rounding N up to a power of 2 is required, so that extra communication waiting time occurs.

(2) In Gather / Scatter communication, if the root computer uses a sequential processing method of transmitting and receiving data to and from all other computers, the processing time increases in proportion to the number of computers. When a high-speed binary tree method is used, each computer must secure a work buffer, and the load increases as the data amount increases. In addition, since the overhead of memory copying to the working buffer occurs, if the data becomes large or the number of computers is small, the overhead does not improve the performance.

(3) In the all-reduce communication, if the internal buffer is not used, n + log (n) times of communication are required, and if the internal buffer is used, the high-speed memory for communication cannot be used. .

An object of the present invention is to provide an inter-computer data communication method for reducing the number of times of communication and the communication processing time of the entire MPI communication processing while minimizing necessary resources such as buffers. Is to do.

[0029]

[Means for Solving the Problems]

(1) In alltoall communication, communication is performed in such an order that computers waiting for communication in each phase do not appear.

(2) For the Gather / Scatter communication processing, both the sequential processing method and the binary tree method are prepared, and the processing using the advantages of each is performed by the hybrid method in which a more efficient method is switched according to the conditions. provide.

(3) In the all-reduce communication, a method for reducing the number of communication times by using an internal buffer and using a high-speed memory for communication to reduce the communication time per time. I will provide a.

[0032]

BEST MODE FOR CARRYING OUT THE INVENTION

(Alltoall communication method) The Alltoall communication method in the present embodiment will be described with reference to FIG. FIG. 1A shows an example in which even number (six) computers perform Alltoall communication.
(B) shows an example in which an odd number (five) of computers perform Alltoall communication.

First, a communication method according to this embodiment will be described (the communication method is common to FIGS. 1A and 1B). In phase 0, each computer communicates with a computer having a rank number obtained by subtracting the rank number of its own computer from N-1 (5 in the example of FIG. 1A). After the phase 1, each computer sets the computer obtained by adding 1 to the communication partner in the previous phase as the communication partner. However, the value obtained by adding 1 is N (FIG. 1).
In the example of (A), when it becomes 6), it is reset to 0. This is repeated up to phase N-1. Therefore, the Alltoal
l is communication of a total of N phases of phase 0 to phase N-1.

Next, with reference to FIG. 1A, when an even number of computers perform Alltoall communication, how this embodiment determines a communication pair in each phase will be described.

In phase 0, each computer (0,1,2,3,
4 and 5) are computers (5, 4, 3, 2, 1, and 0) respectively. Where (5,4,3,2,1,
Since the sequence of (0) is the reverse of the sequence of (0,1,2,3,4,5), if computer a communicates with computer b, computer b also communicates with computer a. The relationship of being with the other party holds.

In the phase 1 and thereafter, the number of communication partners is 2
Divided into two parts. For example, in phase 2, each computer (0,1,2,3,4,5) has its own computer (1,0,5,
4, 3, 2) will be the communication partner. here,
When the arrangement of (1,0,5,4,3,2) is divided into two parts 101a and 101b surrounded by a dotted line, the arrangement of 101a is (1,0), and the arrangement of 101b is (5,4, 3,2).
These sequences are the inverse of the (0,1) sequence,
Since the arrangement of (2, 3, 4, 5) is reversed, if the computer a has the computer b as the communication partner, the relationship holds that the computer b also has the computer a as the communication partner.

In the next phase 3, each computer (0, 1,
2,3,4,5) are computers (2,1,0,5,4,3)
Is the communication partner. Here, the computer 1 and the computer 4 each have their own communication partner.
At this time, 102a (2, 1, 0) or 102b (5, 4,
In the case where the arrangement is odd as in 3), the communication partner is itself in the middle. (In the figure, circles indicate where the communication partner is yourself.)
Next, referring to FIG. 1B, an odd number of computers are assigned to Alltoa
In the case of performing II communication, how this embodiment determines a communication pair in each phase will be described.

In phase 0, each computer (0,1,2,3,
4) means that the computer (4, 3, 2, 1, 0) is the communication partner. Here, the sequence of (4,3,2,1,0) is the reverse of the sequence of (0,1,2,3,4), so if computer a is using computer b as the communication partner, ,calculator
The relationship is established that b also has computer a as the communication partner. Also, since the arrangement of (0, 1, 2, 3, 4) is an odd number, the communication partner of the computer 2 in the middle is itself.

In the phase 1 and thereafter, the number of communication partners is 2
Divided into two parts. For example, in phase 2, each computer (0,1,2,3,4) has its own computer (1,0,4,3,4)
2) will be the communication partner. Where (1,0,
4, 3, 2), two parts 103a enclosed by dotted lines
And 103b, the arrangement of 103a is (1, 0), 1
The arrangement of 03b is (4, 3, 2). Since these arrangements are the reverse of the arrangement of (0,1) and the arrangement of (2,3,4), if computer a is the communication partner of computer b, computer b will also The relationship holds that the computer a is the communication partner. Also, since the arrangement of (2, 3, 4) is an odd number, the communication partner of the middle computer 3 is itself.

The above is the same as the case where an even number of computers perform Alltoall communication.

Here, when the number of computers is odd, one computer is always used in each phase, and only one computer uses itself as a communication partner.

As described above, according to the communication method of the present embodiment, in each phase, if the computer a has the computer b as the communication partner, the computer b also has the computer a as the communication partner. Holds. Therefore, the communication processing of each computer can be executed at the same time, and data transmission to each other can be executed in an overlapping manner.
Also, no extra communication waiting time occurs.

(All-to-All Communication System with Other Computers When the Number of Computers is Even) Referring to the example of FIG. 2, when the number of computers N is even (6 in the example of FIG. 2), A method of communicating with the other party in the N-1 (5 in the example of FIG. 2) phase is shown.

First, the communication order in N-1 (= 5) Alltoall communication systems is determined. Since N is even, N-1
Becomes odd. Therefore, in each phase, one and one
Only one computer will communicate with itself.

The five computers (computer 0 to computer 4) are Al
An example of performing ltoall communication is as shown in FIG. Communication is performed in a total of five phases, that is, phases 0 to 4. In each phase (0,1,2,3,4), the computer (2,
0, 3, 1, 4) have themselves as communication partners.

Each of the computers (0, 1, 2, 3, 4) uses the sixth computer 5 instead of its own communication partner.
Is the communication partner, communication with the other five devices is performed in five phases. Correspondingly, computer 5
Is the computer (2,0,2) for each phase (0,1,2,3,4)
Communication is performed in the order of (3, 1, 4).

As described above, the six computers 0 to 5 can communicate with the other party in five phases.

(Gather / Scatter Communication Method) Next, an embodiment of the gather / scatter communication processing according to the present invention will be described with reference to the flowchart of FIG.

When performing communication processing using the MPI function,
Since each computer performs the initialization processing in the first part of the program, each computer knows the total number of computers after the initialization processing is completed (processing 1101).

Next, when the Gather / Scatter communication function is called, the amount of good data is obtained from its argument (process 1102), and the total number of computers obtained at the time of initialization is checked. If the value is equal to or less than the value (process 1103) and the total number of computers is equal to or more than a certain value (process 1104), data is communicated using the binary tree method thereafter (process 1104).
105). If the condition is not met, processing is performed using the sequential processing method (processing 1106). Finally, an end process is performed (process 1107), and the program ends.

Each switching point differs depending on the characteristics of the computer to be mounted, that is, the amount of installed memory, the performance of one-to-one communication, the memory copy performance, and the like.

(Embodiment of all-reduce) Next, an implementation method of all-reduce in the present invention will be described in detail with reference to the drawings. FIG. 5 shows an overall configuration diagram of the present invention. 501, 502
Represents a memory area inside the processor, and each memory has a high-speed communication memory (503, 504) and a normal memory (50
5, 506). Two buffers specified as function arguments of all-reduce, a data buffer (50
7, 508) and the operation buffers (509, 510) are allocated to the communication high-speed memory. Furthermore, receive buffers (511, 512) dynamically allocated at the time of execution are usually allocated to memory.

Conventionally, in each phase of the Hyper Cube method, each process transfers data from the data buffer to the receiving buffer of the partner process in the first communication, and receives the partner process from the arithmetic buffer in the second and subsequent communications. Transfer data to the buffer. In the present invention, high-speed all-reduce is realized by using a high-speed memory for communication by always using a data buffer and an operation buffer during communication.

A method for realizing the present invention will be described with reference to FIG. First, each process copies the data in the data buffer to the reception buffer before starting communication (713, 71).
4). The reception buffer is thereafter used only for holding the original data. At the time of the first communication, each process transmits data from the data buffer to the operation buffer of the other process (715, 716), calculates the data of the other process and the data of the own process received by the operation buffer, and stores the data in the operation buffer. . In the second and subsequent communications, data is transmitted from the operation buffer to the data buffer of the partner process (717, 718), and an operation is performed using the operation result up to the previous time in the operation buffer and the received data in the data buffer. To be stored. This is sequentially repeated, and the final operation result is stored in the operation buffer of each process. After the end of the communication, the original data held in the reception buffer is copied to the data buffer at last (719, 72).
0) Terminate the communication.

[0055]

According to the present invention, the following effects can be obtained.

(1) All-to-all communication can be realized in a time proportional to the number of computers without requiring extra communication waiting.

(2) Even if the total communication data amount or the number of computers changes, processing can always be performed in an optimal manner.

(3) In the all-reduce function, the communication can be performed at a higher speed by adopting the Hyper Cube method with a small number of communication times and using a high-speed memory for communication.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a communication method of Alltoall.

FIG. 2 is an explanatory diagram of an all-to-all communication system with a computer other than itself when the number of computers is an even number.

FIG. 3 is an explanatory diagram of a conventional simultaneous communication method between all computers.

FIG. 4 is an explanatory diagram of a communication pattern of Alltoall.

FIG. 5 is a configuration diagram of a computer in All-reduce.

FIG. 6 is an explanatory diagram of a data transfer order of the Hyper Cube method.

FIG. 7 is an explanatory diagram of a communication order of All-reduce.

FIG. 8 is an explanatory diagram of a communication pattern of gather communication.

FIG. 9 is an explanatory diagram of a sequential processing method in gather communication.

FIG. 10 is an explanatory diagram of a binary tree method in Gather communication.

FIG. 11 is a flowchart showing switching processing between a sequential processing method and a binary tree method in gather communication.

[Explanation of symbols]

101a, 101b, 102a, 102b, 103a,
103b. . . Line of communication partner, 440-44
3. . . Computer, 401-433. . . Data transmission, 501, 502. . . Memory area, 50
3,504. . . High-speed memory for communication, 507, 50
8. . . Data buffer, 601-605. . . Buffer, 606-611. . . Communication, 801-
804. . . Computer, 810-813. . . 820. Buffer storing transmission data; . . Buffer for storing received data, 1030 to 1033. . . Working buffer, 1101-1107. . . 9 is a flowchart illustrating switching processing between a sequential processing method and a binary tree method in Gather communication.

 ──────────────────────────────────────────────────の Continued on the front page (72) Inventor Shunji Takubo 1-280 Higashi Koigakubo, Kokubunji-shi, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd. Hitachi Software Development Division

Claims (5)

[Claims] 1. A data communication method between element computers in a distributed memory type parallel computer connected by a network, comprising:
In each communication step, two element computers perform mutual communication by a set of two element computers, and each communication step forms a different set from the communication set in the previous communication step, and performs n communication steps Wherein the data in each element computer is mutually transmitted and received between all the element computers. 2. A method for arbitrarily assigning serial numbers from 0 to n-1 to said n element computers, and in the first communication, each element computer
-1 is subtracted from the serial number given to itself, and is paired with a computer of a value.In the subsequent communication steps, each element computer is an element computer in which the communication partner number is increased by one, but n- After the 1st element calculator, the 0th element calculator,
2. The data communication method between computers according to claim 1, wherein the communication is performed in pairs. 3. A high-speed network, wherein a specific memory area is allocated for high-speed communication, and a transmitting node writes data directly to a high-speed communication memory of a receiving node, thereby reducing the OS overhead of the receiving node and achieving high-speed communication. A data communication method between element computers in a distributed memory type parallel computer that realizes communication, and a binary tree in which a work memory for all data is secured in each element computer and sequentially collected in half of the element computers for each communication step Select the method of communicating with the model and the method of one element computer collecting data without securing working memory and communicating with all the element computers sequentially using the number of element computers and data amount as parameters. A data communication method for inter-computer, characterized in that communication is performed so as to optimally collect data in all of the element computers into one element computer. 4. A high-speed network, wherein a specific memory area is allocated for high-speed communication, and a transmitting node directly writes data to a high-speed communication memory of a receiving node, thereby reducing the OS overhead of the receiving node and achieving high-speed communication. A data communication method between element computers in a distributed memory type parallel computer that realizes communication, and a binary tree in which a work memory for all data is secured in each element computer and sequentially distributed to double element computers at each communication step. Select the method of communicating with the model and the method of one element computer that distributes data sequentially with all element computers without securing working memory, using the number of element computers and data amount as parameters A data communication method for inter-computer communication characterized by performing communication that optimally distributes data in one element computer to all element computers. 5. A high-speed network, a specific memory area is allocated for high-speed communication, and a transmitting node directly writes data to a high-speed communication memory of a receiving node, thereby reducing an OS overhead of a receiving node and achieving high-speed communication. A data communication method between element computers in a distributed memory type parallel computer that realizes communication, and secures a working memory for storing initial data, thereby enabling transmission and reception allocated to a high-speed communication memory. An inter-computer data communication method characterized in that an operation result for data in all element computers is communicated to all element computers using only a memory area.