JPH07219916A - Communication method for parallel computer - Google Patents

Abstract

PURPOSE:To solve flow control and overwriting problems in the case of the speed up and transmission of transmission process and reception process are overwritings to a reception area. CONSTITUTION:A function for inhibiting data reception to a receiving area, a function for holding data reception and a function for allowing data reception to the receiving area by a receiving process are applied to prevent the generation of overwriting on preceding received data due to the execution of transmission in main storage residentiary communication. When data reception due to communication is generated in a data reception inhibiting state, operation for forced overwriting, data abolition and data holding can be executed. A network interface monitoring mechanism and a timer are provided with a function for informing the close of the network to a processor when the network is closed more than a fixed time due to the holding of received data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data communication method between parallel computers connected by a mutual communication network.

[0002]

2. Description of the Related Art In a distributed memory parallel computer or a distributed processing system, speeding up data transfer between nodes is indispensable for improving the processing capacity of the entire system. Usually, at the application program level, data transfer is data transfer between processes on different nodes. On the hardware level, on the other hand, data transfer only transfers data from one node to a designated node. That is, there is a difference between the transfer between processes and the transfer between nodes in the function level of the application and the hardware.

In order to absorb this difference, an operating system such as UNIX adds / deletes a packet header for instructing a destination process, disassembles / assembles a packet, or runs even when the receiving process is not running. In order to enable data transfer after the start, processing such as buffering for temporarily holding received data is performed.
The processing of such an operating system, while providing a general-purpose communication function, increases software overhead and hinders speeding up of communication.

Main memory resident communication is a technique for reducing such overhead and realizing high-speed communication. For example, US Ultra Network Technologies (Ul
UltraNetwork of traNetwork Technologies) provides the function to transfer data directly between the network adapter (interface hardware) and the main memory resident area in the user application space. That is, in the main memory resident communication, the communication area of the sending process and the receiving process is fixedly resident in the physical memory,
Data is directly transferred from the physical memory on the sending node side to the physical memory on the receiving node side. Since the virtual space is always mapped on the physical address, page-in processing when the transmitted data is paged out, received data is buffered once, and stored until the receiving process is in a running state, etc. It eliminates the need for software overhead and realizes faster communication.

[0005]

The above-mentioned main memory resident communication system reduces the software overhead and provides simple and high-speed communication, but has the following problems.

(1) Difficult to avoid overwriting in the receiving area In the main memory resident communication, the contents of the communication area of the sending process are directly transferred to the communication area of the receiving process without buffering or the like. That is, the transmission of the main memory resident communication is to forcibly overwrite the reception area with the contents of the transmission area. For this reason, it is impossible for the receiving process to avoid the overwriting by the transmitting process to the received data that is being processed.

(2) Flow control is indispensable To avoid the above-mentioned overwriting, that is, to avoid the state in which the data sent last time is overwritten while it is in the process of being processed, the flow control on the transmission side and the reception side is performed. Is essential. Therefore, MIMD (Muliple Instruction Mu
(ltiple Data) processing, the data receiving side controls the data flow, and in order to realize pipelined data transfer by main memory resident communication, whether to send from the receiving side to the sending side is required. The flow control message to notify is indispensable and this is an overhead.

The present invention aims to solve these problems.

[0009]

According to the first aspect of the present invention, a communication in which a transmission / reception area for data is made resident in a physical memory, data is transferred between the physical memories, and the contents of the transmission area are overwritten in the reception area. In the method, the operating system sets the reception area to a reception non-permission state upon completion of data transfer to the reception area by reception, while the reception process completes processing of data in the communication area Allows you to set to receive permission. Then, when data transfer by another communication occurs to the communication area in the reception non-permission state, the received data can be discarded.

In a second aspect of the invention, an operating system is used in a communication system in which a data transmission / reception area is made resident in a physical memory, data is transferred between the physical memories, and the contents of the transmission area are overwritten in the reception area. When the data transfer to the reception area is completed by the reception, the reception area is set to the reception non-permission state, while the reception process sets the communication area to the reception permission state when the processing of the data in the communication area is completed. It can be so. Then, when data transfer by another communication occurs to the communication area in the reception non-permission state, the received data can be held.

In the third invention, in order to monitor the state of the received data, when the received data is held for a certain period of time, the processor is notified by means such as an interrupt.

[0012]

According to the first method of the present invention, it is possible to prevent the received data from being overwritten by the execution of the transmission by the transmitting side process even though the previously transmitted data is being processed.

According to the second method of the present invention, even if the flow control is not explicitly performed from the receiving side to the transmitting side by the message, the communication initiated by the receiving side can be performed without overwriting or loss of data. .

According to the third method of the present invention, the blockage or deadlock of the interconnection network caused by the long-term data hold can be efficiently eliminated.

[0015]

1 shows an example of the configuration of a parallel computer system to which the present invention is applied. The entire system has a configuration in which nodes 101 are connected by a network 102. Each node 101
Is a processor 104 connected by a system bus,
It has a main memory 105 and a network interface adapter (NIA) 103. The NIA also includes an arbitration circuit 106, a transmission circuit 113, a transmission buffer 112, a reception circuit 107, a reception buffer 110, a DMA circuit 111,
It has a control circuit 108 and can transfer data to any node. A control register 109 readable and writable by the processor is provided in the control circuit 108, and is used for receiving a command from the processor to the NIA and outputting the state of the NIA. The function of the control circuit can be realized by a general-purpose microprocessor. The timer 114 can start and end counting triggered by a state transition in the NIA and output an interrupt accordingly, and is used for measuring the state in the NIA and notifying the processor of the state.

In the main memory resident communication, the communication area of the transmitting process and the communication area of the receiving process are fixedly resident in the physical memory between the transmitting node and the receiving node, and the physical memory of the transmitting node side to the receiving node side. It is a communication method that directly transfers data to a physical memory through a network. Here, the communication area made resident in the main memory is called Combuf.

FIG. 2 and then the main memory resident communication procedure seen from the user. “Combuf_xxx” in the sentence is the name of a routine in the operating system (hereinafter referred to as OS) called by the user for realizing main memory resident communication.

1) Combuf_open is issued on both the transmitting side and the receiving side, and Combuf is allocated on the page fixed area.

2) Both the transmitting side and the receiving side issue Combuf_connect to establish a connection between the transmitting and receiving nodes.

3) The receiving side issues Combuf_attri_set,
Set the attributes of the receiving area.

4) The transmitting side issues Combuf_send and transfers the data to the receiving side.

5) The receiving side issues Combuf_process_end and permits writing to the receiving area.

6) The receiving side issues Combuf_recv and waits for the arrival of data.

7) The transmitting side issues Combuf_disconnect to release the connection with the receiving node.

8) Both the transmitting side and the receiving side issue Combuf_close to release Combuf on the page fixed area.

Next, the contents of each processing in the OS will be described.

Combuf_open is a Combuf of the specified size.
Secured in the page fixed area, and its address and identifier (Com
buf_ID), and returns the access right code. Combuf_ID is a unique value for the entire parallel computer system. See also Combuf_open
Is a Comb called Local Combuf Table (LCT)
The management table of uf is also updated. The LCT is shown in FIG. L
The CT has a Combuf_ID 701, an access right code 702, a start physical address 703, a size 704, and a processing completion flag 7 for all Combufs used by the node.
05, protection mode 706, hold timeout time 707
It is a table that summarizes information such as.

Combuf_open is Combuf_ID 70 of LCT.
1, access right code 702, start physical address 70
3, size 704 is set.

Combuf_connect registers information of a destination Combuf in a table called a remote Combuf table (RCT) that manages a destination Combuf_ID, and establishes a connection between the transmitting and receiving nodes. The RCT is shown in FIG. RCT
Is the node number 801, Combuf_ID 802, access right code 80 of the Combuf with which the connection is established.
3 is a table storing information such as size 804.
By registering this RCT, the Combuf of the transmission destination is set to Combuf_ID.
Can be referred to. Prior to calling this routine, the user exchanges Combuf information between the nodes establishing the connection by normal communication.

Combuf_attri_set refers to the LCT and sets the protection mode 706 and the hold timeout time 707 of the specified Combuf. In the procedure of transmission / reception using Combuf in the present invention, the user receives Combuf_process after receiving the data.
Unless _end is issued and the processing completion flag of the corresponding LCT is changed from “not completed” to “completed”, data is not continuously received in the same Combuf. LCT protection mode 706
Defines the operation when data is received in Combuf when the processing completion flag is “not completed”. When data is transmitted with the processing completion flag in the "incomplete" state, the OS reception routine determines the processing for the received message according to this information. There are three protection modes: forced overwrite, data destruction, and data suspension. When selecting data hold, also specify the hold time.

Combuf_send is a Combu specified to another node.
Send the contents to f. Data transmission is performed as follows. First, LCT and RCT are searched using the transmitted / received Combuf_ID as a key. Using the information in these tables,
In the Combuf_send routine, the start physical address where the transmission data exists, the data size, the node number of the partner, etc. are set to N.
The data is written in the control register of the IA to instruct the data transfer. In response, NIA transfers the contents of the specified area to the target node.

Combuf_process_end changes the processing completion flag of the corresponding LCT from "not completed" to "completed", and the Co
Allow receiving data to mbuf. Combuf in the present invention
In the procedure of transmission / reception using the
uf_process_end is issued and LCT reception permission flag 70
Unless 5 is changed from "not completed" to "completed", the same Co
Do not continue to receive data to mbuf. Where LCT
Setting the reception permission flag of "Complete" to permit reception
It is Combuf_process_end. If the Combuf has data pending for reception, Combuf_process_end restarts the reception and gives an instruction to start writing data.

Combuf_watch, for the specified Combuf,
Reports whether data has been received from other nodes. Finally Co
Data reception from the time of issuing mbuf_process_end to the current time is the target. The data reception process itself is activated by an interrupt when the data arrives, and Combuf_watch
Etc. are executed asynchronously with other communication processes.

Combuf_disconnect cancels the connection between the transmitting and receiving nodes by deleting the information of the specified Combuf from the RCT.

Combuf_disconnect releases the page fixed area used by the specified Combu, and deletes the information of the Combuf from the LCT.

FIG. 5 shows an operation during transmission / reception of main memory resident communication. The transmission is started by the transmission process issuing Combuf_send (). In response to this, the operating system searches the LCT / RCT, writes the starting physical address where the transmission data exists, the data size, the node number of the other party, etc. in the control register of the NIA, and issues a data transfer instruction. In response to this, the NIA transfers the contents of the designated area to the target node.

On the other hand, data reception is started by the arrival of data and is executed asynchronously with the processing of the receiving process.

When the data arrives at the NIA, the NIA writes the header portion of the data in a specific area of main memory. The header is fixed length data at the beginning of the transfer data. The header stores the processor number and process number of each transmission / reception, access right code, reception Combuf_ID, offset of write address, and the like. When the writing of the header is completed, the NIA issues an interrupt to the processor to notify the arrival of data and the end of writing the header. In response to this, the receiving routine of the operating system is activated.

The processing of the reception routine is shown in FIG. First, the local Combuf table with the Combuf_ID on the header as the key
Search (1001) for (LCT). As a result of the search (1002), the received Combuf_ID and access right code of the received data are LCT
If not present, the NIA is instructed to discard the data that arrived as an illegal access (1006). Receive Combuf
If _ID and access right code exist in LCT,
The following processing is performed based on the information written in the LCT. First, the processing completion flag for the Combuf_ID in the LCT is checked (1003). The processing completion flag is
This is a flag that indicates whether the user has completed the processing for the received data, and the user has set Combuf_process_end ().
Can be set to "Complete" at any time. Therefore, if this flag is "Complete", the user has permitted data reception, so the NIA should write the received data to the Combuf indicated by the target Combuf_ID.
(1004), the OS sets the processing completion flag to “not completed” (1005). On the other hand, if the processing completion flag is “not completed”, the LCT is referred to and the processing according to the protection mode set by the user is performed (1007).

This processing is shown in FIG. 7, and these processings will be described next. When the protection mode is "forced overwrite", the same processing as the conventional main memory resident communication is performed. That is, N
The IA is instructed to forcibly overwrite the received area with the received data regardless of the processing state of the user (1104).

When the protection mode is "data discard", the NIA is instructed to discard the received data to discard the received data (1105), and the fact of the discard is notified to the receiving process of the user (1106) and is logged. Etc. (1
107). This mode is also a specific embodiment of the first method of the present invention. The data discard mode is mainly used for a parallel program in which the receiving side and the transmitting side communicate while performing explicit flow control, and data is not transmitted at a timing that the user's receiving process does not intend. In such a program, data itself is transmitted at a timing that the user's reception process does not intend,
This means that the processing desired by the user has not been performed due to a program bug or the like. Therefore, in order to protect the data transmitted last and stored in the memory, the transmitted data is discarded and the fact is notified to the receiving process. The reception process can perform exception processing such as interruption of the current processing, continuation of processing up to a possible portion, and rollback of the processing so far by notifying the data discard.

When the protection mode is "data hold", the data hold time is set in the timer (1108) and the receiving process is held. If the reception completion flag is set to "Complete" by Combuf_process_end within the hold time, the received data is read. This mode is also a specific embodiment of the second method of the present invention. Data hold mode is MI
It is mainly used for communication between parallel programs performing MD-like operations.

By using the data hold mode, when the data processing speeds of the receiving side and the transmitting side are almost the same, overwriting of data in the receiving area or loss of received data is performed without explicit flow control. Communication that depends on the data request from the receiving side is realized without the need for the above. Further, in the data hold mode, when the hold time expires, all the received data is discarded, the fact of the discard is notified to the receiving process of the user, and it is recorded in a log or the like. This is to prevent the network from being blocked or deadlocked due to a hold of the received data when the received data is not taken in for a certain period of time and is regarded as a program bug.

[0044]

According to the first aspect of the present invention, in a parallel program in which the receiving side and the transmitting side communicate while performing explicit flow control, and data is not transmitted at a timing unintended by the user's receiving process, This is to prevent the received data from being overwritten by unintended reception on the side. Also, in order to protect the data sent last in the memory, the sent data is discarded, and the fact is notified to the receiving process. Therefore, the receiving process interrupts the current process by notifying the data discarding, Exception processing can be performed such as continuing processing up to the possible part and rolling back the processing so far.

According to the second aspect of the present invention, for communication between parallel programs operating as MIMD, a message is sent from the receiving side to the receiving side even if flow control is not explicitly performed by a message from the receiving side to the transmitting side. When the data processing speeds are almost the same on the side, the communication depending on the data request on the receiving side can be realized without overwriting the data in the receiving area or losing the received data.

According to the third aspect of the present invention, it is possible to effectively eliminate the blockage of the interconnection network and the deadlock caused by the data hold for a long time.

[Brief description of drawings]

FIG. 1 is a block diagram of a system according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a procedure of main memory resident communication.

FIG. 3 is an explanatory diagram of an example of a local Combuf table.

FIG. 4 is an explanatory diagram of an example of a remote Combuf table.

FIG. 5 is an explanatory diagram of an operation during transmission / reception of main memory resident communication.

FIG. 6 is a flowchart of processing of a reception routine.

FIG. 7 is a flowchart of processing when reception is not permitted.

[Explanation of symbols]

101 ... Node, 102 ... Network, 103 ... Network interface adapter, 104 ... Processor, 105 ... Main memory, 106 ... Arbitration circuit, 107 ... Reception circuit, 108 ... Control circuit, 109 ... Control register, 11
0 ... reception buffer, 111 ... DMA circuit, 112 ... transmission buffer, 113 ... transmission circuit, 1149 ... timer.

Claims (3)

[Claims] 1. A communication area in a virtual address space of a transmission and reception process of a computer system in which a plurality of nodes each having a processor and a memory are connected by an intercommunication network is made resident in a physical memory of each node, In a communication method in which data is directly transferred between the areas of the physical memory and the contents of the transmission area are overwritten in the reception area, the operating system completes the transfer of the data to the reception area by reception and When the reception process is set to the non-reception state, and the reception process completes the processing of the data in the communication area, the communication area can be set to the reception-permitted state, and data transfer by another communication occurs to the communication area in the non-reception state. In this case, the communication method of the parallel computer is characterized in that the received data can be discarded. 2. A communication area on a virtual address space of a transmission and reception process of a computer system in which a plurality of nodes each having a processor and a memory are connected by an mutual communication network is made resident in a physical memory of each node, In a communication method in which data is directly transferred between the areas of the physical memory and the contents of the transmission area are overwritten in the reception area, the operating system completes the transfer of the data to the reception area by reception and When the reception process is set to the non-reception state, and the reception process completes the processing of the data in the communication area, the communication area can be set to the reception-permitted state, and data transfer by another communication occurs to the communication area in the non-reception state. In this case, the parallel computer communication method is characterized in that the received data can be held. 3. A communication method of a parallel computer according to claim 2, wherein interface hardware with a network is monitored, and when received data is not taken in for a certain time or longer, an interrupt is issued to a processor to notify a blocked state of the network. .