JPH09212474A - Method for data transfer between processors and computer system suitable to same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a reception area which is not fixed to be used as a main storage and reduce synchronous processing as well. SOLUTION: A transfer source node 10 transmits data, the virtual address of a data transfer destination area which is not fixed as the main storage, the address of a transfer control flag in use, a comparison value, and a method for comparison. At a reception-side node 210, a network adapter 30 decides that conditions are met according to a semaphore in a transfer control flag 330 specified with the received transfer control flag address and the received comparison value and comparing method. The received virtual address of the transfer destination area and an address conversion table 140 are used to inspect whether or not the transfer destination area 340 that the virtual address indicates is in the main storage 50. When the conditions are not met or when the transfer destination area 340 is not in the main storage 50, the received data are stored in a transfer buffer 370 in an area for an OS. When a reception- side program issues a specific system call or when the program issues a read instruction for data in the transfer destination area and a page fault is caused, the OS transfers the received data from the transfer buffer 370 to the transfer destination area 340.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data transfer method in a computer system in which a plurality of nodes each having a processor are connected by a data transfer network, and a computer system suitable therefor.

[0002]

2. Description of the Related Art In a parallel computer, speeding up data transfer between processors leads to speeding up of the entire system. There are two factors that determine the performance of data transfer between processors: (1) data transfer rate and (2) transfer activation delay. The transfer start delay is an overhead of hardware / software processing required to start data transfer.

When transferring a large amount of data at once,
Data can be transferred at high speed by increasing the data transfer rate. However, when repeatedly transferring short data,
If the transfer activation delay is not reduced, the performance will not be obtained even if the data transfer rate is high. While the data transfer rate is determined by a physical factor such as the transfer data width, the size of the transfer start delay mainly depends on the transfer method, and therefore the transfer method itself becomes important in transferring short data.

In current parallel computers, SEND /
A data transfer called RECEIIVE type is adopted. S
In the END / RECEIVE type, when the processor on the data transmission side executes the transmission request command (SEND), the transfer data is transmitted to the reception side. The processor on the receiving side receives the transfer data by executing the reception request instruction (RECEIVE). The area from which transfer data is read (transfer source area) is specified in the SEND instruction, and the area to which transfer data is written (transfer destination area) is specified in the RECEIVE instruction. As a method for designating an area, it is general to use the start logical address of the area and the transfer data size.

For SEND / RECEIVE type transfer,
(1) The sending side can execute the SEND instruction regardless of the timing of the receiving side's RECEIVE instruction execution (asynchronous transfer), making it easy to write a program. (2) The receiving side uses R
Since the data is not written until the write area is specified by the ECEIVE command, there is a low risk that the data on the receiving side will be destroyed by a program bug on the transmitting side, which is widely used from workstation clusters to large-scale parallel computers. ing.

However, in the SEND / RECEIVE type transfer, the data sent from the data sending side is transmitted to the RE on the receiving side.
There is a problem that the transfer destination area cannot be written until the execution of the CEIVE instruction. Therefore, in the SEND / RECEIVE type transfer, a method of temporarily buffering the transfer data in the receiving side node is generally adopted.

As a simple management method for this buffering, there is a method in which the receiving processor always buffers the transfer data, and the data in which the RECEIVE instruction is executed is copied from the buffer to the transfer destination area. However, in this method, the memory copy is always generated along with the data transfer, and the performance does not appear.

Japanese Unexamined Patent Publication No. 6-324998 discloses RECEIV.
A means for directly writing the data arriving at the receiving side after the execution of the E instruction to the transfer destination area and buffering only the data arriving before the execution of the RECEIVE instruction is disclosed. This reduces memory copy,
High performance can be demonstrated.

However, in the parallel computer of Japanese Patent Laid-Open No. 6-324998, when the receiving node receives the transfer data, it checks whether or not the RECEIVE command corresponding to the data has already been issued, and further issues it. If so, it is necessary to retrieve the information specified by the RECEIVE command and write the transfer data in the transfer destination area specified by the information. In this way, SE
In the ND / RECEIVE type, there are many hardware / software processes required to realize asynchronous transfer,
The transfer activation delay does not decrease.

As a solution to this problem, there is PUT type data transfer. In PUT-type data transfer, not only the transfer source area but also the transfer destination area is specified in the transmission request command (PUT) of the destination processor. As a method of designating an area, it is general to use a start logical address of the area and a transfer data size. After the transfer is started, the data is unconditionally written in the transfer destination area, so that the receiving side does not need to issue the reception request command and the buffer for the received data is not necessary.

In PUT type data transfer, (1) it is necessary for the user to protect the execution of the PUT instruction from the transmitting side after the receiving side is ready to accept the transferred data (synchronous transfer), (2) transmitting side There is a problem that if there is a bug in the program, the data on the receiving side will be easily destroyed. However, since the processing on the receiving side only writes the data in the transfer destination area according to the header information of the transfer data, the transfer start delay can be made small, and the performance of the short data transfer can be higher than that of the SEND / RECEIVE type. .

"Efficient Execution of Parallel Programs by Hardware Support of PUT / GET Interface"
(Parallel Processing Symposium JSPP'94 Proceedings, pp.
233-240, May 1994) discloses a method of implementing PUT type data transfer. In this PUT instruction,
The transfer source area and transfer destination area are specified by virtual addresses.

Now, consider a case where a computer for managing virtual addresses transfers data between processors. If a virtual address space wider than the actual size of the main memory is prepared, or if the memory capacity required by multiple processes running simultaneously exceeds the size of the main memory, the transfer source / destination area is not the main memory but a hard disk, etc. It may exist on the external storage device of.

In response to this problem, if the transfer source area is partially present in the external storage device, the transmission side does not start the transmission operation but loads the entire area to the main storage and then starts the transfer. This procedure can be easily realized by software / hardware. This allows the data to be sent out without interruption during transmission. However, the receiving side cannot perform the receiving operation if part or all of the receiving area exists in the external storage device at the time when the data arrives. As a result, when data is reserved on the interprocessor network, network congestion occurs, which causes a performance problem.

As disclosed in Japanese Patent Laid-Open No. 6-110845, there is a means for ensuring that the transfer destination area always exists in the main memory by the OS (main memory fixed). This allows
Since data can be received without interrupting the transfer, network congestion can be prevented.

However, in the PUT type data transfer, the receiving side cannot determine where the transfer destination area is until the transfer data arrives. For this reason, the data area that may receive a large amount of data must be fixed in the main memory. For example, if it is impossible to determine which column the received data belongs to, even though only one column of the matrix data is actually received, the entire data reception area that may receive the matrix data I cannot help fixing my main memory. This significantly impedes the flexibility of virtual memory management.

A technique in which the main memory is not fixed to all of the data areas that may receive data is disclosed in Japanese Patent Laid-Open No. 4-291.
660. Here, when the data area for storing the received data is swapped out and is not in the main memory and the data to be stored in the area is received, the data is temporarily buffered in the reception buffer of the OS management. , Hardware generates an interrupt after buffering is completed. With this as an opportunity, the OS interrupts the execution of the program being executed and then executes the interrupt processing in the OS. In this process, the reception area of this data is paged in, and as soon as the page-in is completed, the reception data in the reception buffer is transferred to the data reception area.

Aside from the above-mentioned receiving area swap-out technique, basic programming using PUT type communication will be described below. In a PUT-type data transfer system, the area where data is to be written from the processor on the sending side must be synchronized between the program on the sending side and the program on the receiving side in order for the data to be read by the receiving side processor. To guarantee the end of writing. If this is not done, inconsistency will occur, such as writing from the sending processor while the receiving processor is reading the data receiving area. For example, when exchanging the contents of the data M in the node A and the contents of the data N in the node B, the processors of the nodes A and B execute a program as shown in FIG.
The nodes A and B execute the instructions 3000A and 3000B for copying the data M and the data N, respectively. This is a PUT instruction 3020A, 3020B for exchange operation.
This is because it is premised that the data is directly written in the data M and N. Thus, it is necessary to first secure the write destination area and then execute the instructions 3010A and 3010B between the node A and the node B for barrier synchronization. If this is not done, there is a possibility that another node may execute the PUT instruction 3020B (or 3020A) and write data while the area to be written by the PUT instruction 3020A (or 3020B) is not secured. In particular, when transferring a large amount of data, it is often impossible to secure the area to be written in advance, and in some cases, every time PUT transfer is performed, the area must be secured immediately before the PUT transfer to achieve synchronization. Next, the PUT instructions 3020A and 3020B are executed in each node. Each node needs to execute the instructions 3030A and 3030B for obtaining the barrier synchronization again before reading the data written from another node. This is because there is no means for knowing whether or not the data writing by the PUT command from the other party has been completed. After synchronization, the completion of the PUT instruction at both nodes can be guaranteed, so that both nodes can read the written data (exchanged data).

[0019]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
In the method disclosed by 1660, each time the CPU receives data to be written to the swapped-out data area, the CPU needs to be interrupted and execute interrupt processing. For this reason, there is a problem that the execution of the program being executed at that time is interrupted. In particular, swapping in the swapped-out page requires accessing the external storage device, which causes a long interruption of the program being executed. At the time of this interruption, the program being executed may not be in a state of using the received data. Therefore, interruption of the program being executed in such a state is not desirable from the viewpoint of program execution efficiency. Further, if the executing program does not access the page holding the received data for a while even after the page is held, the page may be swapped out again. When this happens, the swap-in immediately after reception is wasted.

In the conventional PUT type data transfer, it is necessary to frequently perform synchronization between processors having a large overhead such as barrier synchronization. Frequent synchronization between processors for PUT-type data transfer causes PUT
The advantage of low overhead of type data transfer cannot be utilized.

Therefore, an object of the present invention is to provide a PUT type data transfer method capable of transferring data to a data receiving area which is not fixed in the main memory and having no great influence on a program being executed at the time of data reception. It is to provide a computer system suitable for it.

Still another object of the present invention is to provide a data transfer method suitable for PUT type data transfer and a computer system suitable for the same, which has less synchronization overhead between processors.

[0023]

In order to achieve the above object, the data transfer method of the first operation mode of the present invention executes the following steps.

The data and the virtual address of the transfer destination area in which the data should be stored are transferred from the transfer source node to the transfer destination node, and the virtual address received at the time of receiving the data at the transfer destination node is When the above-mentioned allocated area exists in the main memory in the transfer destination node,
A transfer buffer for storing the received data in the area and temporarily holding the received data provided in the transfer destination node when the transfer destination area does not exist in the main memory of the transfer destination node When the received data is stored in the area, the transfer destination area does not exist in the main memory of the transfer destination node, and when a predetermined instruction issued subsequently from the program executed in the transfer destination node is executed , The received data stored in the transfer buffer area is transferred to the transfer destination area. Here, this transfer is executed after the transfer destination area is assigned to the main memory.

The data transfer method of the second operation mode of the present invention executes the following steps.

The data, the virtual address of the transfer destination area in which the data should be stored, and the comparison value are transferred from the transfer source node to the transfer destination node, and the semaphore value specified by the program being executed in the transfer destination node. Is stored by the transfer destination node, and the transfer destination node determines whether or not the comparison value and the stored semaphore value satisfy a predetermined condition for writing the data in the transfer destination basin. If, as a result of the determination, it is determined that the condition is satisfied, the received data is stored in the transfer destination area to which the virtual address in the transfer destination node is allocated, and when the condition is not satisfied, Store the received data in the buffer area for temporarily holding the data provided in the transfer destination node, and respond to the transfer request issued later from the above program. And forwards the data to the destination area from the buffer area.

[0027]

BEST MODE FOR CARRYING OUT THE INVENTION The data transfer method and computer system according to the present invention will be described below in more detail with reference to the embodiments of the invention shown in the drawings.

<1. Overview of Device and Operation><1.1 Overview of Device> FIG. 1 shows a schematic configuration of a parallel computer system that employs a data transfer method according to the present invention. In this system, a plurality of nodes 10 each having a CPU 20, a main memory 50, and a network adapter 30 are connected by an interprocessor network 1000. Each node has its own OS. The network adapter 30 is a unit in charge of data transfer between nodes, and the CPU 20
Is started by another node, for example, node 2
Transfer the data to 10. Further, it has a function of writing data sent from another node, for example, the node 210, into the main memory 50. Note that the nodes such as the node 10 and the node 210 may be completely the same hardware or different hardware, but in the present embodiment, each node has the same structure. However, in FIG. 1, in the network adapter 30 and the main memory 50 of the node 10, the operation on the transmission side when the data stored in the transfer source area 120 of the node 10 is transferred to the transfer destination area 340 of the node 210. I have filled in the necessary elements to explain. Similarly, in the network adapter 230 and the main memory 50 of the node 210 on the receiving side, the elements necessary for explaining the operation of the receiving side node are entered. Each node, for example 10, has a CPU 20, a network adapter 30, a main memory control unit 40, and a main memory 50.
Furthermore, an external storage device 60 is prepared for each node. Inside the main memory 50, there are a user control area 90 (main memory fixed) and a user data area 100 (virtual memory management). The data existing in the user data area 100 does not always exist, but is swapped out to the external storage device 60 as needed. Therefore, the transfer source area 120 and the transfer destination area 340 may exist in the main memory 50 or may be swapped out to the external storage device 60.
As the area for the OS, the OS area 130 is provided separately from the user area and managed so that the main memory is fixed.

<1.2> Details of Node The following seven elements are prepared in the main memory 50.

(1) Transfer source area 120 This is an area for storing transfer data of the transmitting side node 10.
It is prepared in the user data area 100 and managed as a virtual memory.

(2) Transfer destination area 340 This is an area in which the transfer data of the receiving side node 210 should be written. It is prepared in the user data area 100 and managed as a virtual memory.

(3) Transfer buffer 370 This is an area for temporarily recording the transfer data of the receiving side node 210. It is prepared in the OS area 130 and fixed in the main memory.

(4) Address conversion table 140 This is an area for recording address conversion information at both the transmitting and receiving nodes. It is prepared in the OS area 130 and fixed in the main memory.

(5) Transfer source flag 110 This is a flag for recording the data transfer operation status (transmission / end, etc.) of the transmission-side node 10. The recorded values are shown in FIG. Details will be described in <2.2>. User control area 9
It is prepared in 0 and fixed in the main memory. The user program of the transfer source node 10 can grasp the current status of the transfer operation by reading this flag.

(6) Transfer destination flag 320 This is a flag for recording the data transfer operation status of the receiving-side node 210 (such as receiving / buffering completed). The recorded values are shown in FIG. The details of this flag will be described in <2.2>. This flag is prepared in the user control area 90 and fixed in the main memory. The user program of the receiving-side node 210 can grasp the current state of the transfer operation by reading this flag.

(7) Transfer control flag 330 This is a flag including a semaphore, a buffering number, etc. for synchronizing the reception of data at the receiving side node with the progress of the program in that node, and the configuration of this flag is shown. 9 shows. The details will be described in <2.2>. This flag is prepared in the user control area 90 and fixed in the main memory. Specifically, the semaphore 2010 is in a state where it cannot transfer the data to the transfer destination area 340 because the receiving side node 210 has not finished using the transfer destination area 340 at the time of receiving the data. It is used to judge whether or not. In this embodiment, the transfer destination area 34
A method for forcibly storing transfer data in the transfer buffer 370 when the transfer destination area 340 is in a state in which data transfer cannot be accepted, such as when the receiving side node 210 has not finished using 0. Take.

The structure of the network adapter 30 is shown in FIG. The main elements are listed below.

(1) Transmission control register 70 The transmission request from the CPU 20 is recorded. The internal structure is shown in FIG.

(2) Transmission control unit 410 Performs a transmission operation based on the information in the transmission control register 70.

(3) Network transmitter 430, network receiver 450 In charge of connection with the interprocessor network 1000.

(4) Reception control register 170 When transfer data from another node arrives, transfer control information is recorded. The internal structure is shown in FIG.

(5) Reception control section 440 The reception operation is performed based on the information in the reception control register 170.

(6) Main memory access unit 400 The main memory is accessed in response to a request from the transmission control unit 410 or the reception control unit 440 to access the main memory 50.

(7) Table management register 80 It has real address information of the address conversion table 140, specifically, the head address 1510 of the table. The internal structure is shown in FIG.

(8) Address translation unit 480 It is in charge of virtual-real address translation based on the information in the address translation table 140 in response to the access request from the main memory access unit 400.

(9) Buffer management register 190 It holds the address information of the transfer buffer 370. The internal structure is shown in FIG.

(10) Buffer control section 470 Based on the information in the buffer management register 190, it operates to store the transfer data in the transfer buffer 370.

The operation of the above components is <2.2.
>.

<1.3> Data Transfer Command FIG. 20 shows a data transfer command (PU) used in this embodiment.
(T instruction) format is shown. The command has the following nine parameters.

(1) Transfer data length 2710 Designates the number of bytes of data to be transferred. The maximum transfer length on the hardware is 1 MB (details will be described later).

(2) Transfer source area start address 2720 The virtual address in the transmission side node 10 of the transfer source area 120 is designated.

(3) Transfer source flag address 2730 The virtual address of the transfer source flag 110 in the transmitting side node 10 is designated.

(4) Transfer destination processor number 2740 Specify the node number of the data transfer destination (node 210 number).

(5) Transfer control flag address 2750 The virtual address of the transfer control flag 330 in the receiving side node 210 is designated.

(6) Comparison value 2760 Information newly used in the present embodiment specifies a value to be compared with the semaphore value (2010 in FIG. 9) in the transfer control flag 330. It is 32-bit data without a code.

(7) Comparison method 2770 Information newly used in the present embodiment is semaphore 2
A method of comparing 010 and the comparison value 2770 will be described. The relationship between the value of this comparison method 2770 and the comparison method executed is shown in FIG.
Shown in By using the value 2770 and the comparison value 2760 of this comparison method, various conditions can be specified and the synchronization between the data reception and the progress of the reception side program can be easily performed. Specifically, these pieces of information are in a state in which the receiving side node 210 has not finished using the transfer destination area 340 at the time of receiving the data, and therefore, whether or not the data cannot be transferred to the transfer destination area 340. Used to determine if. Therefore, in the present embodiment, as will be described later, the receiving side process is configured to rewrite the semaphore 2010 according to the progress of the program. If the condition specified by the comparison method 2770 is not satisfied at the time of receiving the data, the transfer data is forcibly stored in the transfer buffer 370 regardless of the value of the comparison method 2770. As a result, the transfer data is not transferred to the transfer destination area 340 but is transferred to the transfer buffer 370, and this transfer data is transferred from the transfer buffer 370 to the transfer destination area 340 later.
Transfer to This reduces the overhead of synchronization processing between processes required for data transfer.

(8) Transfer destination area start address 2780 The virtual address in the receiving side node 210 of the transfer destination area 340 is designated.

(9) Transfer destination flag address 2790 Specifies the virtual address of the transfer destination flag 320 in the receiving side node 210.

In the present embodiment, page boundaries (1M
Do not allow data transfer across B boundary). Therefore, when at least one of the transfer source area and the transfer destination area extends over a page, the data transfer needs to be realized by a plurality of data transfers. Similarly, long data exceeding 1 MB can be dealt with by multiple times of data transfer.

The PUT instruction shown in FIG. 20 represents a function call to the communication library provided in the user program of the data transfer source. The various parameters shown in the figure represent the arguments of this function call. However, in the following, the term instruction or parameter will be used for simplicity. This communication library, when called from the user program, stores these parameters in appropriate registers and then activates the network adapter 30 for this PUT operation. When this adapter 30 is activated for PUT operation, it will use the various parameters described above to
Perform the action.

<1.4> Specific Usage Example of Transfer Control Flag Next, data transfer between processors using the transfer control flag 330, the comparison value 2760, and the comparison method 2770 will be described with reference to FIG.
Description will be made using a specific program example shown in FIG. 21 is similar to the case of the conventional technique shown in FIG. 21 in that it is a program for swapping the contents of the data M in the node A and the contents of the data N in the node B. Unlike the above, the PUT instruction shown in FIG. 20 is used. However, in FIG. 22, unlike FIG. 20, the notation of the PUT instruction is simplified. That is, PUT instruction 3120A
Alternatively, in 3120B, dataM, flag (s
a) and nodeB are the transfer source area start address 2720 and the transfer source flag address 273 in the PUT instruction of FIG.
0 represents the transfer destination processor number 2740. when
flag (b)> 0 is the transfer control flag address 275.
The transfer control flag b is specified by 0, and the comparison value 2760 is 0.
Is designated, and ">" is designated by the comparison method 2770. Also, dataN and flag (rb) are the transfer destination area start address 2780 and the transfer destination flag address 279.
Indicates 0. The transfer data length 2710 of FIG. 20 is omitted in FIG.

The first instruction sequence 3100A of the transfer source user program and the first instruction sequence 3100B of the transfer destination user program have a transfer control flag (a, b) and a transfer source flag (sa, sb) used by these programs. ),
The transfer destination flag (ra, rb) is initialized. In order to ensure that both sides can be initialized, the instructions 3110A and 3110B for realizing barrier synchronization between the processors are executed. The flag group has a small size and can be easily initialized at the beginning of the program, and accordingly, the barrier synchronization by the barrier synchronization instructions 3110A and 3110B can also be reserved at the beginning of these programs.
Nodes A and B then execute PUT instructions 3102A, 310 at appropriate points after executing various program instructions.
Perform 2B.

PUT instruction 3120A executed by node A
"Uses the transfer data of the data area M as the transfer source flag and the PU as the transfer source area N of the node B.
T. However, if the semaphore value in the transfer control flag b in the node B is not larger than 0, the condition that the transfer data is temporarily stored in the transfer buffer is added. The end of writing is recorded in the transfer destination flag rb in the node B. Has the meaning. Similarly, the PUT instruction 3120B executed by the node B “PUTs the transfer data of the data area N to the data area M of the node A by using flag (sb) as the transfer source flag. If the semaphore value in the transfer control flag a is not greater than 0, the condition is set that the transfer data is temporarily stored in the transfer buffer. The end of writing is recorded in the transfer destination flag ra in the node A. " To have.

As will be described in detail later, P of node A
When the UT instruction 3120A is executed, the value of the transmission source flag sa is updated according to the transfer state of the transfer data in the data area M according to FIG. Further, the value of the transfer destination flag rb in the node B is updated according to the transfer state of this transfer data according to FIG. Similarly, when the PUT instruction 3120B of the node B is executed,
In the process of execution, the value of the transfer source flag sb in the node B changes according to the transfer state of the transfer data in the data area M.
Updated according to 2. Further, the value of the transfer destination flag ra in the node A is set according to the transfer state of the transfer data shown in FIG.
Updated according to 3.

When the data in the data area M in the node A is transferred to the node B by the execution of the PUT instruction 3120A in the node A, the transfer control in the node B will be described in detail later in the description of the operation of the receiving node. The semaphore value in the flag a is compared with the value 0 specified by this instruction according to the comparison method (>) specified by this PUT instruction, and when this semaphore value is not greater than 0 or when this semaphore value is greater than 0. However, when the data area N designated by this PUT instruction does not exist in the main memory, this transfer data is stored in the transfer buffer of the node B. When this semaphore value is larger than 0 and the data area N designated by this PUT instruction exists in the main memory, this transfer data is
It is immediately stored in the data area N. PUT on Node B
The same applies when the data in the data area N in the node B is transferred to the node A by the execution of the instruction 3120B.

The semaphore in the transfer control flag a at the node A and the semaphore in the transfer control flag b at the node B are updated by the instruction sequences 3130A and 3130B as follows. After executing the appropriate instruction after the PUT instruction 3120A of the node A, the instruction sequence 3130A is executed. The WAIT instruction in the instruction sequence 3130A is an instruction to wait until the transfer of the transfer data designated by the PUT instruction 3120A executed in the node A is completed. Specifically, it waits until the data transfer state becomes the transmission completion buffering state or the transmission completion incoming call completion state shown in FIG. That is, the value of the transfer source flag sa is equal to or larger than the smaller value 0010 of the flag value 0010 (binary number) indicating the transmission completion buffered state and the flag value 1010 indicating the transmission completion incoming completion state shown in FIG. Wait to become. SFIN in the WAIT instruction indicates this specific flag value 0010. The SET command is WAI
It is an instruction to set the semaphore value in the transfer control flag a to 1 when the waiting of the T instruction is successful.

Similarly, the PUT instruction 3120B of the node B
After executing the appropriate instruction, the instruction sequence 3130B is executed. The WAIT instruction in this instruction sequence is an instruction to wait until the transfer of the transfer data designated by the PUT instruction 3120B executed by the node B is completed. Specifically, it waits until the value of the transfer source flag sb becomes equal to or more than the specific flag value 0010. The SET instruction is an instruction to set the semaphore value in the transfer control flag b to 1 when the wait of the WAIT instruction succeeds.

Therefore, from the node B, the PUT instruction 3120
By the time the data in the data area N of the node B transferred by B is received by the node A, if the instruction sequence 3130A has already been executed in the node A, the value of the semaphore in the transfer control flag a is Because it is 1,
This transfer data is the PUT instruction 3 executed by the node B.
The condition specified by 120B is satisfied. Therefore,
This transfer data is written in either the data area M of the node A or the transfer buffer 370 depending on whether or not the data area M in the node A exists in the main memory. In this way, the network adapter 30 of the node A prevents the data transferred from another node from being written to the data area M before the end of the use of the data area M by the program on the node A side. . This also applies to the node B.

In the case of data transfer in which the data M in the node A and the data N in the node B are swapped, the barrier synchronization instructions 3010 and 3030 are executed in the conventional PUT type data transfer program of FIG. However, the program of FIG. 22 does not require such an instruction. As described above, according to the present embodiment, regarding data transfer between processors, it is possible to reduce instructions for synchronization between processors. As shown in FIG. 22, when the program of the receiving side node finishes using the data area by the program, a semaphore indicating the end of use is set in the transfer control flag, and the network of the receiving side node is set. Based on the value of the semaphore, the adapter determines whether the program of the receiving node has finished using the data area to write the transfer data, and the program of the receiving node uses the data area to write the transfer data. This is to prevent the transferred data from being written in the data area before the end. In the present embodiment, in the PUT instruction executed by the program of the node on the transmission side, the transfer control flag used to determine whether or not to write the transfer data in the transfer destination area is specified, and the determination condition is further specified. The comparison value used for the judgment is also specified to realize more flexible control.

Further, in the program of FIG. 21, the instruction 3
Although it is necessary to explicitly execute an instruction to copy data before executing a data transfer instruction, such as 000A and 3000B, this embodiment does not require such an instruction. This is also because, as described above, the network adapter 30 on the receiving side uses the transfer control flag to control whether to write the transferred data in the data area.

Now, the node A confirms whether or not the data is put into the data area M by the value of the transfer destination flag of the node A itself. That is, the subsequent WAIT instruction 3140A of the node A is an instruction to wait for the completion of the data transfer to the node A by the PUT instruction 3120B of the node B. Specifically, this instruction waits until the transfer state of the data specified by this PUT instruction 3120B becomes the reception completion buffered state or the reception completion state shown in FIG. 13, that is, the PUT instruction of the node B is The value of the flag ra designated as the transfer destination flag of the data in the data area M is the smaller value of the flag value 0110 indicating the reception completion buffered state and the flag value 0100 indicating the reception completion write completion state shown in FIG. 01
Wait for 00 or more. Note that R in the WAIT command
FIN indicates this specific flag value 0100. Next I
In the F instruction 3150A, the value of the transfer destination flag ra is RFIN.
, That is, when this value is 0110, that is, when the data to be transferred from the node B to the data area M has been written in the transfer buffer of the node A, but has not yet been written to the data area M, It includes a system call that requests the OS to extract this transfer data from the transfer buffer 370 to the data area M. OS is
In response to this system call, this data retrieval is executed. At this time, when the data area M is swapped out from the main memory, the main memory is allocated to the data area M and then this data is extracted.
When the value of the transfer destination flag ra is 0100, this IF instruction is not executed.

At the node A, after the IF instruction 3150A, a READ instruction 3160A requesting the reading of the data in the data area M is issued. RE at node A
When the AD instruction 3160A is executed, the transfer destination data area M
Already exists in the main memory and the transfer data has already been written therein, the READ instruction is immediately executed.

Similarly, the node B confirms whether or not the data is put into the data area N by the value of the transfer destination flag of the node B itself. That is, the subsequent WAIT instruction 3140B of the node B is an instruction to wait for the end of data transfer to the node B by the PUT instruction 3120A of the node A. Specifically, this instruction is such that the transfer state of the data to be transferred becomes the reception completion buffering state or the reception completion state, that is, the PUT instruction 3110A of the node A is used as the transfer destination flag of the data of the data area N. It waits until the value of the designated flag rb becomes equal to or more than the specific flag value 0100. Next IF instruction 3
In 150B, when the value of the transfer destination flag rb is not equal to RFIN, that is, the data to be transferred from the node A to the data area N is written in the transfer buffer of the node B, but not yet written in the data area M. At times, this is a system call that requests the OS to extract this transfer data from the transfer buffer to the data area. The node B also issues the READ instruction 3160B requesting the reading of the data in the data area N after the IF instruction 3150B.

These instructions 3140A, 3150A, 3
As can be seen from the execution modes of 160A, 3140B, 3150B, and 3160B, even when each node reads transfer data from another node, the arrival of the data can be confirmed by the transfer destination flag. There is no need to synchronize with.

<2. Device Details> In this section, a memory space management method, device configuration, and user interface of the computer system will be briefly described. <2.1> Memory management method FIG. 14 shows real addresses in each node in the present embodiment. Indicates a space. OS area 211 in the range where main memory exists
It has 0 and a user area 2120. As the I / O area, the OS-dedicated I / O area 21 that can be accessed only by the OS
40 and user I / O that can be accessed in user mode
And an O region 2150. OS dedicated I / O area 2140
Access from the user mode is protected by generating an interrupt in address translation. The details will be described later.

FIG. 15 shows a mapping method of the virtual address space to the real address space in this embodiment.
The mapping from the virtual address space to the real address space is performed for each node. FIG. 15 shows a mapping method when process A and process B are operating in one node. For each process A (or B), the control area 2290 (or 23
10) and the data area 2300 (or 2320) are prepared. Of these, the control areas 2290 and 2310 are OSs.
Fixed in the real address space. That is, the OS guarantees that it always exists in the main memory. Control area 229
0 and 2310 include the transfer control area 2 of the process A, respectively.
210 and 2240 are allocated. On the other hand, the data areas 2300 and 2320 manage virtual storage. In other words, it does not always exist in the real address space and is allowed to be swapped out to the external storage device. A transfer destination area 2220 and the like are assigned to these data areas 2300 and 2320. In this embodiment, the OS area 2280 used by the OS is fixed in the real address space. That is, the fact that it is always in the main memory is realized by OS management.

In this embodiment, the swap-in / swap-out unit size (which is equal to the page size) in virtual memory management is 1 MB. That is, the data areas 2300 and 2320 of each process are swapped in / out between the external storage device and the main storage in page size (1 MB) units. Also, each virtual address space has a 32-bit address, that is, 4 GB.
Have a size of. That is, each virtual address space is composed of 1 MB × 4K pages.

<2.2> Details of Operation In this section, the operation of the OS, the operation of the transmitting side node, and the operation of the receiving side node will be described in this order.

<2.2.1> Operation of OS One of the important operations performed by the OS is the creation of the address conversion table 140. Before explaining the address conversion, the process number which is a management number given to each process by the OS will be described. Prepare 0 to 15 as the process number. Since 0 is always reserved for the OS itself, the processes of numbers 1 to 15 can simultaneously operate as other processes.

First, the address conversion method in this embodiment will be described with reference to FIG. The address conversion is performed by the address conversion unit 480 of the network adapter 30.
(FIG. 2). The operation of the address conversion unit 480 itself will be described in <2.2.2> and <2.2.3>.

In the present embodiment, the 32-bit virtual space is set to 1
Since it is managed by the MB page, the virtual address to be translated is 12 bits of the upper 2430. The 12 bits, the address conversion table address 1510 (FIG. 5) held in the table management register 80 (FIG. 2),
Process number 242 of the process that requested address translation
The address conversion table 140 (FIG. 1) is subtracted from 0, and the real address 2470 is obtained using the real page number 2460 obtained there.

Here, the virtual address upper 2430, the table management register information 2410, the process number information 24
20, a real address calculation method for drawing an entry in the address conversion table 140 will be described with reference to FIG. The least significant bit is automatically set to 0. This is because each entry of the address conversion table 140 has 16 bits. The virtual address upper 2430, the process number 2420, and the table management register information 2410 are piled up in that order, and the address conversion table 140 is accessed using this as a real address. Address conversion table 14
0 has a format in which 16 8 KB tables (= 16 bits × 4K entries) exist for each process.

The contents of each entry of the address conversion table 140 are shown in FIG. Each entry has a V bit 261 in addition to the real page number 2460 for address translation.
0, R bit 2620, and M bit 2630. The meaning of each bit is shown in FIG.

Another important operation in this embodiment is an operation of setting the process number and the node number in the transmission control register 70 in the network adapter 30 in advance before data transfer. Transmission control register 7
0 is shown in FIG. It consists of multiple registers. These are mapped to the user I / O area 2150, the register group 70B, and the OS-dedicated I / O area 215.
Register group 70A mapped to 0. Each register is mapped in the real address space by the so-called I / O mapping method. Whether the access from the user process is permitted or not is whether the address to be accessed is mapped to the OS dedicated I / O area 2140,
It is determined depending on whether it is mapped to the user I / O area 2150. In other words, from the perspective of the user process,
The address conversion table 1 is an entry for accessing a register having an address belonging to the OS-dedicated I / O area.
40 does not exist, or even if the entry exists, the V bit is 0, so access is denied at the time of address translation.

O in the transmission control register 70 shown in FIG.
In the register group 70B mapped to the address belonging to the S-dedicated I / O area 2140, a field 1210 for recording the process number of the currently executing process and a field 1220 for recording the number of the node on which the network adapter 30 is mounted are provided. Exists. In the example of FIG. 1, the node number given to the node 10 is held in the field 1220 of the transmission control register 70. The node number is set at system startup, and the process number is set by the OS at the time of process switch.

Further, the table management register 80 of FIG.
The OS also sets a value in the buffer management register 190. As shown in FIG. 5, the table management register 80 stores the first real address of the address conversion table 140. The OS transfers the transfer buffer 3 in the OS area 130.
An area for 70 is reserved for a page boundary for a multiple of 1 MB. When securing this area, the current buffer address 1610 in the buffer management register 190
The start address of the area is set in (FIG. 6), and the value obtained by subtracting 1MB + 28B from the end address of the area is set in the buffer end address 1620 (FIG. 6). O
Each time S stores transfer data in the transfer buffer 370, the current buffer address 1610 is increased by the length of the stored data. The operation is shown in <2.2.3>. At this time, if the value of the current buffer address 1610 exceeds the buffer end address 1620, the CPU
Issue an interrupt request to 20. As described above, the maximum transfer data length is 1 MB, and the maximum length that can be stored in the buffer at one time is 1 MB + 28 even with the packet header described later.
B. Therefore, by the above processing, it can be guaranteed that the last one transfer can be completely stored in the transfer buffer 370.

When the above-mentioned interrupt in the transfer buffer management is received, the OS newly prepares a buffer area in the main memory, and uses the address conversion table 140 so that this area becomes an area connected from the end address of the transfer buffer 370. Set and end buffer end address 16
Update 20. The operation at this time is also shown in <2.2.3>.

The table management register 80 and the buffer management register 190 are also mapped in the OS-dedicated I / O area 2140 described in FIG.
Only accessible.

The CPU 20 actually causes the transmission control register 7
0, the flow of signals generated when accessing the table management register 80, the buffer management register 190, and the CP
A procedure for the U20 to access the register group in the network adapter 30 will be described with reference to FIGS.

The CPU 20 displays the I / O dedicated to the OS on the line a10.
A request for an address is issued to one of the registers existing in the O area. This address value is
4 is an address to one of the transmission control register 70, the table management register 80, and the buffer management register 190 prepared in the OS-dedicated I / O area 2140 shown in FIG. To the network adapter 30 through the line a20. The request is made by the selector 42 in the network adapter 30.
0 is transmitted. Based on the address value, the selector 420 determines whether the request is for the transmission control register 70, the table management register 80, or the buffer management register 1
90 to determine the lines c30 and c5 accordingly
The corresponding one of 5 and c75 is activated. With the above series of operations, the line d10, the main memory control unit 40,
Through the line d20, the CPU 20 and the transmission control register 7
0, the table management register 80, and the buffer management register 190 can exchange data.

<2.2.2> Operation of Transmission-Side Node The operation of the transmission-side node will be described step by step, focusing on the node 10 in FIG. 1 and FIG. The numbers enclosed by "" in the following description are values in binary notation. [1] Transmission start Data transfer instruction PUT (see FIG. 2) from the user program.
0) is issued, the CPU 20 starts the transmission start processing. The CPU 20 issues to the line a10 a request for an address corresponding to the register group 70A existing in the user I / O area of the transmission control register 70. Since this address value is an address to information to be stored in the transmission control register 70 prepared in the user I / O area 2150 shown in FIG. 14, the main memory control unit 40 sends the received request to the network. The line a20 is put out to the adapter 30. The request is transmitted to the selector 420 in the network adapter 30. Based on the address value, the selector 420 determines that it is an access request from the CPU 20 to the transmission control register 70, and activates the line c30. With the series of operations described above, it becomes possible to exchange data between the CPU 20 and the transmission control register 70 through the line d10, the main memory control unit 40, and the line d20.

The access of the CPU 20 to the transmission control register 70 is as follows. First, the CPU 20 reads the control register status 1010. The control register status field 1010 is set to 1 if the use of the transmission control register 70 has not been completed due to the previously activated data transfer.
Has already been set. When the use of the transmission control register 70 is completed, 0 is set. if,
When the read information indicates that the use of this register is not completed, the CPU 20 determines that the transmission control register 7
Poll and wait for control register state 1010 until 0 is available. When this register becomes available, the CPU sets in the transmission control register 70 nine parameters specified by the data transfer instruction PUT (FIG. 20) in the user program.
Finally, the CPU 20 writes “00” indicating that the transfer is in the transfer start state in the transfer start field 1020 in the transmission control register 70. As a result, the transmission control unit 410 is activated, and the network adapter 30 starts the data transmission operation as follows.

[2] Update of transfer source flag (1) When the transmission control unit 410 is activated, the transmission control unit 410 receives the transfer activation process, and transfers the transmission source flag address 1 of the transmission control register 70.
In the transfer source flag 110 on the main memory 50 designated by 040 (FIG. 3), a value 0001 (FIG.
A write request requesting to write
50 to the main memory access unit 400. The write request designates a set of the transfer destination flag address in the field 1040 and the process number 1210 in the transmission control register 70, which has already been set by the OS, as a main storage address. Main memory access unit 400
Outputs the virtual address and process number of the transfer source flag 110 to the address conversion unit 480 via the line a70. The address translation unit 480 calculates the address of the entry in the address translation table 140 by the method described above, and the real address obtained as a result is passed through the line a75,
The data is output to the main memory access unit 400. Main memory access unit 40
0 indicates a read request for data having the real address through the line a55, the mixer 490, the line a30, the main memory control unit 40, and the line a40 to obtain the address conversion table 14
Put out for 0. At the same time, the main memory access unit 400
The switch 500 is connected to the address conversion unit 4 through the line c45.
Switch to 80. The address translation information (see FIG. 18) read from the address translation table 140 is the line d5.
0, the main memory control unit 40, the line d30, the switch 500, and the line d80 to enter the address conversion unit 480. If the V bit 2610 is 0 among the obtained address translation information, the address translation unit 480 sends the CPU 20 through the line c10.
Interrupt. This is because the V bit 2610 being 0 indicates that the transfer request designated by the user was invalid. Therefore, the process is passed to the OS,
Take appropriate action such as killing the process. Also, when both the R bit 2620 and the M bit 2630 are 0, the CPU 20 is interrupted through c10. Since the transfer source flag 110 needs to exist in the user control area 90, that is, the area fixed to the main memory, when the OS receives this interrupt, the OS considers that the transfer request specified by the user is invalid, and kills the process. Take appropriate treatment of.

When the address translation table information can be accessed legitimately, the real page number 2460 (FIG. 18) in it can be accessed.
The address conversion unit 480 determines that the transfer source flag 110
The main memory access unit 400 is requested through line a75 to calculate the real address of and write the value "0001" (see FIG. 12) indicating the start of transmission to that address. The main memory access unit 400 has a line a55 and a mixer 49.
0, the line a30, the main memory control unit 40, and the line a40, a write request is issued to the transfer source flag 110 in the main memory, and the value "0001" indicating the start of transmission is set to the line d8
5, mixer 460, line d40, main memory control unit 40, line d
It is sent to the transfer source flag 110 through 50.

[3] Start of Data Transfer Next, the transmission control section 410 sends the information stored in the transmission control register 70, which is necessary for creating a data transfer packet, to the network transmission section 430 through the line c40. This required information is specifically the control register status 1010, transfer start bit 1 among the information shown in FIG.
This is information other than 020. .

At the same time, the transmission control unit 410 sends a read request for the transfer source area 120 to the main memory access unit 400.
Put out Main memory access unit 400 and address conversion unit 48
0 is the same as the above, and the address conversion table 14
Address translation information is read from 0. Here, the address conversion unit 480 uses the V bit 2610 or the M bit 2
If 630 is 0, the CPU 20 is interrupted through the line c10. When the V bit 2610 is 0, the OS takes appropriate processing such as killing the process. If the M bit is 0, the transfer source area 120 is transferred from the external storage device 60.
Swap in the page containing and restart the transfer operation.

When the address conversion table information can be accessed legitimately, the address conversion unit 480 is based on the information.
Allows the real address of the transfer source area 120 to be calculated. As described above, data transfer across pages is prohibited in terms of hardware, so the address translation table 140 may be accessed once for each transfer.

In this state, next, the main memory access unit 400
To read the transfer data from the transfer source area 120,
Line a70, address conversion unit 480, line a65, mixer 4
90, the line a30, the main memory control unit 40, and the line a40, a read request for the transfer data is issued to the transfer source area 120, and the output of the switch 500 is switched to the network transmission unit 430 through the line c45. The transfer data read from the transfer source area 120 is transmitted to the network transmission unit 430 through the line d50, the main memory control unit 40, the line d30, the switch 500, and the line d70. The network transmission unit 430 generates the packet shown in FIG. 7 from the read transfer data and the data in the transmission control register 70 that was previously transmitted from the transmission control unit 410, and the packet shown in FIG. Send to network 1000. In the packet format of FIG. 7, the valid bit 1710 is set to 1 (valid). Further, "00" indicating data transmission is entered in the packet type 1720. Packet type 1720
FIG. 11 shows the values and meanings of the above. The reply packet will be described later in [5].

The interprocessor network 1000 uses the information at the beginning of the packet to transfer the node 210 that is the transfer destination.
To the network adapter 230 of the above through the line n20. The operation on the node 210 side from now on is described in <2.2.3>.

[4] Update of transfer source flag (2) By the same procedure as in [2], the transmission control unit 410 updates the transfer source flag 110 to "0010" indicating completion of transmission.

[5] Reception of write completion notification and update of transfer source flag (3) In the receiving side node 210, transfer data is stored in the main memory 50.
Upon completion of writing (see <2.2.3>), a reply packet (see FIG. 8) is returned from the reception control unit 440. The reply packet is (1) returned when the transfer data is written in the transfer destination area 340 by the receiving side node 210,
A packet whose packet type 1910 is "01" (see FIG. 11)
2) and (2) a packet whose packet type 1910 is “10” which is returned when the transfer data is written in the transfer buffer 370 by the receiving side node 210. The operation by the receiving node 210 to send these packets is <2.2.3.
>.

The network receiving unit 45 in the network adapter 30 of the node 10 on the transmission side transmits these packets.
0 is the line n20 from the interprocessor network 1000.
When the packet is received through, the packet information is temporarily stored in the reception control register 170. The structure of the reception control register is shown in FIG. It should be noted that "0" is entered for information not included in the packet, for example, for the field of the transfer data length 1350. When the packet information is stored in the reception control register 170, the reception control unit 440 requests the main memory access unit 400 to update the transfer source flag 110 through the line a60 based on the information. After that, the transfer source flag 110 is updated as in [3]. Packet type 1910
For the packet with “01”, the transfer source flag 110
Is updated to "1010". Packet type 1910
For the packet with “10”, the transfer source flag 110
Is updated to "0110".

<2.2.3> Operation of Receiving Side Node [1] Activation of Receiving In the network adapter 230 of the receiving side node, when transfer data arrives at the network receiving section 450 from the interprocessor network 1000, the receiving operation is activated. It The top information (see FIG. 7) of the received transfer data packet is recorded in the reception control register 170 through the line c65.

[2] Flag check According to the information in the reception control register 170, the reception control unit 44
0 is activated. The reception control unit 440 uses <2.2.2>
An access request is issued to the transfer control flag 330 similarly to [5]. However, this access operation is <2.2.2
The operation of [5] is a write operation, while the operation is a write operation. The read data passes through the switch 500 in the network adapter 230 and is stored in the reception control unit 440. The reception control unit 440 uses the transfer control flag 33.
The semaphore value 2010 in 0 and the comparison value 1080 in the arrival packet are compared by the method shown by the comparison method 1090 in the arrival packet (see FIG. 10).

[3] Confirmation that the transfer destination area 340 exists in the main memory 50 The reception control unit 440, as a result of the above comparison, the above comparison method 1
When the condition indicated by 090 is satisfied, the transfer destination area 34
It is confirmed whether 0 exists in the main memory 50. Specifically, when starting data transfer in <2.2.2> [3], the address conversion unit 480 is set in the same manner as in the case of confirming whether or not the transfer source area 120 exists in the main memory 50. Used for this confirmation. The difference is that when the transfer source area 120 does not exist in the main memory 50, the address conversion unit 480 interrupts the CPU 20 through the line c10, but the transfer destination area 340 does not exist in the main memory 50. Has
The address translation unit 480 sends the reception control unit 4 through the line c50.
Tell 40 to that effect.

Note that the reception control unit 440 does not confirm the existence of the transfer destination area 340 in the main memory 50 if the condition indicated by the comparison method 1090 is not satisfied as a result of the comparison.

[4] Update of transfer destination flag (1) As described below, the reception control section 440 causes the reception control section 440 to perform the above [2].
Then, it is determined that the condition indicated by the comparison method 1090 is satisfied, and the transfer destination area 340 is stored in the main memory 5 in [3].
If it is confirmed that it exists in 0, the operation of writing the transfer data to the transfer destination area 340 in the main memory 50 is activated. Otherwise, transfer data is transferred to the transfer buffer 3
The operation stored in 70 is started. The reception control unit 440 updates the transfer destination flag 320 according to which operation is performed before these operations. This is <2.2.2>
This is realized by the same operation as the updating of the transfer source flag in.
The value of the updated flag is 0001 or 0011, which indicates the state of writing during reception or the state of buffering during reception, as shown in FIG.

Furthermore, if the transfer buffer 370 stores data as a result of comparison with the transfer control flag 330, the buffering number field (2020 in FIG. 9) of the transfer control flag 330 is counted up. The value of the buffering number is used as a standard for determining whether or not data is accumulated in the transfer buffer 370 when the semaphore value 2010 is updated by the program. This count-up operation also requires the same procedure as other flag updates.

[5] Writing Transfer Data to Main Memory 50 The reception control section 440 uses the comparison method 10 described in [2] above.
It was determined that the condition indicated by 90 was satisfied, but the above [3]
When it is confirmed that the transfer destination area 340 does not exist in the main memory 50, or when it is determined in [2] that the condition indicated by the comparison method is not satisfied, the transfer data is stored in the main memory 50. The write operation to the transfer buffer 370 is started.

That is, the transfer destination area 340 is the main memory 50.
If not, the transfer data is transferred to the transfer buffer 370.
By writing the data in, the transfer reception data can be effectively used even if the transfer destination area 340 is swapped out from the main memory.

Further, in the above [2], the comparison method 109
If the condition indicated by 0 is not satisfied, the transfer destination area 340 cannot be used for writing because the transfer destination process is in use by the transfer destination process. In the present embodiment, as described above, the process on the receiving side receives the semaphore 201 according to the progress of the program.
It is configured to rewrite 0. If the condition specified by this comparison method is not satisfied when the data is received,
Regardless of the value of the comparison method 2770, the transfer data is forcibly stored in the transfer buffer 370, and the transfer data is not transferred to the transfer destination area 340. Later, this transfer data is transferred from the transfer buffer 370 to the transfer destination area 340. This reduces the overhead of synchronization processing between processes required for data transfer.

The reception controller 440 activates the line c70 to activate the writing of the transfer data to the transfer buffer 370.
A buffer write request is issued to the buffer control unit 470 through. In response to this request, the buffer control unit 470 checks the buffer management register 190 and grasps the current write start address (virtual address) from the current buffer address 1610. This address is line a80
Through the main memory access unit 400. Although the virtual address is converted to the real address by the main memory access unit 400, since the transfer buffer exists in the OS area 130, the conversion is performed using "0000" as the process number. The conversion procedure (such as means for fetching the conversion information up to the address conversion unit 480) requires the same procedure as described above. The converted address is output to the transfer buffer 370 on the main memory through the line a70, the address conversion unit 480, the line a65, the mixer 490, and the like, and at the same time, the reception control unit 440 transmits the transferred packet itself to the network via the line c60. The line d9 from the receiving unit 450
0, the network receiving unit 450 is controlled so as to be dropped into the transfer buffer 370 through the mixer 460.

When the transfer data is dropped to the transfer buffer 370, the buffer control unit 470 updates the current buffer address of the buffer management register 190 accordingly. As a result, the current buffer address 161
When 0 becomes larger than the buffer end address 1620 (here, a value 1 MB + α less than the end of the true buffer is stored), the buffer control unit 47.
Interrupt the CPU from 0 through line c10. CPU
Secures a new page for the buffer, connects it to the secured transfer buffer 370, and expands the area of this buffer.

When the reception control unit 440 determines in [2] that the condition indicated by the comparison method is satisfied and also in [3] that the transfer destination area 340 exists in the main memory 50. First, the operation of writing the transfer data to the transfer destination area 340 on the main memory 50 is activated. That is, in this case, the transfer destination area 340 is swapped into the main memory 50, and the transfer destination process is finished using the transfer destination area 340. In this case, the transfer data is immediately transferred. Write to the transfer destination area 340. This write operation also applies to the transfer data transfer buffer 3 described above.
The same operation as that of storing in 70 is performed. However, as the process number used for the address conversion, the number of the transfer destination process added to the transferred packet is used. Further, unlike writing to the transfer buffer 370, only the data in which the control information of the transferred packet is cut off is written to the transfer destination area 340.

[6] Update of transfer destination flag (2) After the completion of writing to the main memory 50, the transfer destination flag 320 is updated. This is realized by the same operation as the update of the transfer source flag in <2.2.2>. It should be noted that the updated value is 01 indicating the reception completion buffered state or the reception completion write completed state, as shown in FIG.
It is 10 or 0100.

[7] Return of reply packet In order to notify the sender node 10 of the completion of writing to the main memory 50, a reply packet (FIG. 8) is returned. This is because the reception control unit 440 sends information necessary for reply packet generation to the transmission control register 70 by a line d65,
It is executed by activating the line c35. The activation procedure needs to follow the same procedure as that performed by the CPU 20 in <2.2.2>.

[8] Transfer of Transfer Data from Transfer Buffer 370 to Transfer Destination Area 340 In the above operation [5], transfer data is transferred to the transfer destination area 34.
When it is possible to write 0, the operation of the receiving node is completed. However, when the transfer data is stored in the transfer buffer 370, it is necessary to extract this transfer data from the appropriate timing transfer buffer 370 to the transfer destination area 340. Since the transfer buffer exists in the OS management area, the OS can only pull it out. The trigger for withdrawing differs depending on the circumstances when the transfer data is transferred to the transfer buffer 370. Each trigger will be described below.

(1) When the transfer destination area 340 does not exist in the main memory 50 at the time of receiving the transfer data In this case, the transfer destination process sends a data read instruction to the page including the transfer destination area 340. (For example,
When the load command) is issued, the transfer data is moved from the transfer buffer 370 to the transfer destination area 340. That is, in this case, when the CPU 20 executes this instruction, a page fault occurs in the process of converting the virtual address designated by this instruction into a real address. In the processing of this page fault, the transfer data is moved from the transfer buffer 370 to the transfer destination area 340.

The processing shifts to the OS triggered by this page fault. In this embodiment, a routine for executing page fault processing is provided in the OS as shown in FIG. That is, after preparing an empty page corresponding to a page fault, all the transfer buffers 370 are inspected as to whether or not the data of the page requested by the above instruction exists in the transfer buffer 370 (step 4020). The data accumulated in the transfer buffer 370 remains in the form including the packet header information. The OS can use this information to search which page the stored data is for.

The data of this page is transferred to the transfer buffer 370.
If it does not exist at all, it is swapped in from the external storage device 60 (step 4030). If the entire data of the requested page exists in the transfer buffer 370, the swap-in is not performed and the transfer buffer 37
The data of this page in 0 is memory copied to the transfer destination area 340 (4060 in the figure). Regarding the extracted data, by setting the data corresponding to the valid bit 1710 of the packet header to 0 (invalid), the data can be ignored by software in the subsequent searches. If possible, the memory copy may not be performed and only the virtual address may be replaced. If a part of the requested page exists in the transfer buffer 370, the data of the page is swapped in from the external storage device 60 (step 4040), and then,
A part of the data existing in the transfer buffer 370 is memory copied (step 4050). After the above processing is completed, the address conversion table 140 is updated so that the M bit 2630 (FIG. 18) relating to the page in the address conversion table 140 is set to 1 (step 4070). As described above, the transfer data can be retrieved from the transfer buffer 370 at the time of page fault.

In the conventional normal SEND / RECEIVE type data transfer, when the RECEIVE instruction is executed,
That is, only when the user process explicitly requests the OS to pull out data, the data is pulled out from the reception buffer, whereas in the present embodiment using the PUT type data transfer, the user process on the receiving side stores the data in the main memory. A read operation of transfer data can be started only by issuing a data read command (for example, a load command) to. This is because in the system that employs the PUT type data transfer, the receiving side process can access the transfer destination area without knowing whether or not the transfer data is written in the transfer destination area. In addition, conventional normal SEND
In the / RECEIVE type data transfer, when a user process requests the OS to retrieve transfer data, the OS
Searches the receive buffer to detect whether the requested transfer data exists in the transfer buffer, and if the data exists in the transfer buffer, the page to which the transfer data should be transferred is stored in the main memory. Detect if it exists However, in the present embodiment using PUT type data transfer,
When a user process issues a data read command and a page fault occurs, it is determined whether the page to which the data requested by this command belongs exists in the transfer buffer. Also in this respect, the procedure for using data transfer in the present embodiment is different from the procedure for using transfer data in the conventional normal SEND / RECEIVE type data transfer.

(2) When the transfer data is received, the semaphore 2
010 does not satisfy the condition specified by the transfer command. This case occurs when the user process on the receiving side is using the transfer destination area 340 at the time of receiving the transfer data. In this case, after the receiving side process finishes using the transfer destination area 340 and then uses the transfer data to be stored in the transfer destination area 340, the receiving side user process sets the transfer destination flag 320. Search and transfer buffer 3
It is determined whether data has already been transferred to 70. When it is detected that the transfer data has been transferred to the transfer buffer 370, a system call requesting that the transfer data be drawn from the transfer buffer 370 to the transfer destination area 340 is issued. The user process can specify the transfer destination flag 320 and read it in order to determine the reception state of the transfer data. The OS executes this withdrawal in response to this system call. The processing at this time is as described above using the specific program example shown in FIG. In that program,
The system call was a GETBUF instruction. An instruction for reading the transfer destination flag is generated from the IF statement included in the instruction sequence 3150A or 3150B.

If the transfer destination area does not exist in the main memory when the transfer data is received and the semaphore 2010 does not satisfy the above condition, either the system call instruction or the above memory access instruction is executed. When this is done, the transfer data is moved. When the system call instruction is issued, it is necessary to perform the page-in operation.

(3) In addition to the above case, the transfer data can be moved. For example, if the user process has the semaphore value 2010 of the transfer control flag 330.
When the user wants to transfer the transfer data accumulated in the transfer buffer 370 due to the old semaphore value to the transfer destination area 340 all at once, the user process issues a system call to move the transfer data. The OS does it.

[9] Return of Reply Packet Corresponding to Completion of Incoming Call When the above [8] is executed, the same operation as the transmitting operation of <2.2.2> is performed, and the software processing of the OS is performed. Returns the reply packet. Unlike when transmitting a data transfer packet, by writing "01" in the transfer activation field 1020 of the transmission control register 70, a reply packet (confirmation of arrival) can be transmitted. Although it should not be necessary, if "10" is written, the buffering confirmation packet can be transmitted by software.

As described above, the program illustrated in FIG. 20 is executed as described above. As a result, as described above, the PUT instruction can be executed without making the transfer destination area resident in the main memory. Further, synchronization between the user program on the transmitting side and the user program on the receiving side can be easily performed.

<3. Modifications> The present invention is not limited to the above embodiments, and can cover various modifications. One of them is shown below. Note that, as will be described next, the hardware configuration can be realized without changing. In the following, only parts different from the previous embodiment will be shown.

<3.1> Basic Idea of Modification The value of the transfer control flag is controlled by the user program.
Therefore, it is essential for the user to see whether the transfer data is buffered in the transfer buffer 370 due to the value of the transfer control flag. However, the transfer destination area 340
Whether or not exists in the main memory cannot be usually managed by the user and depends on the main memory management of the OS. That is, it is not necessary to show to the user that the transfer destination area 340 has been buffered because it does not exist in the main memory by the transfer source flag and the transfer destination flag.

<3.2> Basic point of modification In the specific use example of the transfer control flag shown in <1.4>, (1) the comparison value and the transfer control flag designated by the PUT instruction 3120A (FIG. 22). According to the result of comparison with b, P
In both the case where the received data is written in the transfer buffer 370 because the comparison condition specified by the UT instruction is not satisfied, and (2) the case where the transfer destination area 340 is swapped out, the transfer buffer is set in the transfer destination flag rb. The value 0011 indicating that writing to 370 or 0110 (FIG. 13) indicating that writing to the buffer has been completed is set.

The case where this is changed and the value 0011 representing the writing to the transfer buffer 370 and 0110 representing the completion of writing to the buffer are set in the transfer destination flag rb is limited to the first case. When the comparison value designated by the PUT instruction satisfies the above condition, but the transfer destination area is not in the main memory, the same processing as in the first case is performed. In the case of the second case, the value 0001 indicating that writing is being performed to the transfer destination area 340 or 0100 that indicates the completion of writing to the transfer destination area 340 is written in the transfer destination flag rb.

In the second case, the value of the transfer source flag sa is also a value from the value 0110 (FIG. 12) indicating that the received data has been buffered in the transfer buffer 370, indicating the completion of writing to the transfer destination area 340. Change to 1010.

The change of the set value of the transfer source flag is realized by changing the value of the reply packet returned from the receiving node. That is, this modification is realized by changing only the operation of the receiving side node. The above is exactly the same for the PUT instruction 3120B (FIG. 22).

<3.3> Change in Program Operation of Receiving Node When the program shown in FIG. 22 is executed in this modification, the value of the transfer control flag b or a is P, respectively.
When the received data is written in the transfer buffer 370 because the transfer destination area 340 has been swapped out even though the conditions specified by the UT commands 3120A and 3120B are satisfied, the wait command 3140A that waits for reception, After 3140B is established, the value of the transfer control flag b or a is 0100 = RFIN,
GETBUF in the IF statement of 3150A or 3150B
The instruction does not fire. However, 3160A or 316
Since a page fault occurs when reading the received data with the READ statement of 0B, the received data is read from the transfer buffer 370 by the OS control triggered by this.

Moving the data in the transfer buffer to the transfer destination area is not limited to the case where the data read instruction is executed, and is the same when the instruction for writing the data in the transfer destination area is executed. This also applies to the embodiments described above.

Specifically, the operation of the receiving side node is changed for the following four points.

A) In the transfer destination flag update (1) shown in <2.2.3> [4], the received data is written in the transfer buffer 370 because the transfer destination area 340 was swapped out. Also in this case, the transfer destination flag ra or rb has a value of 0 indicating that the transfer destination area 340 is being written.
Write 001.

B) <2.2.3> In the update (2) of the transfer destination flag shown in [6], the received data was written in the transfer buffer 370 because the transfer destination area 340 was swapped out. Also in this case, the value 0100 indicating the completion of writing to the transfer destination area 340 is written in the transfer destination flag ra or rb.

C) <2.2.3> In returning the reply packet shown in [7], even when the received data is written in the transfer buffer 370 because the transfer destination area 340 was swapped out, The reply packet is provided with a value 01 indicating the confirmation of the incoming call.

D) Regarding the return of the reply packet indicating the completion of the incoming call shown in <2.2.3> [8], GETBU
When the F function forcibly extracts the received data from the transfer buffer 370, the OS returns a reply packet, but when the READ command is used as a trigger, there is no need to return it.

[0140]

As is apparent from the above embodiments and modifications, according to the present invention, PUT type data transfer can be performed without fixing the reception data area to the main memory.

Also, PUT type data transfer can be executed with a small overhead for synchronization between processors.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram of a parallel computer according to the present invention.

FIG. 2 is a network adapter 3 used in the apparatus of FIG.
The schematic block diagram of 0.

FIG. 3 is a diagram showing information in a transmission control register 70 in the network adapter 30.

FIG. 4 is a diagram showing information in a reception control register 170 in the network adapter 30.

5 is a diagram showing information in a table management register 80 in the network adapter 30. FIG.

6 is a diagram showing information in a buffer management register in the network adapter 30. FIG.

FIG. 7 is a diagram showing information in a data transmission packet for the interprocessor network 1000.

FIG. 8 is a diagram showing information in a reply packet for the interprocessor network 1000.

9 is a diagram showing information in a transfer control flag 330 provided in the main memory 50. FIG.

10 is a comparison method 1090 in the transmission control register 70. FIG.
The figure for explaining the value and meaning of.

FIG. 11 is a diagram for explaining values and meanings of a packet type 1700.

FIG. 12 is a diagram for explaining the value and meaning of a transfer source flag 110 provided in the main memory 50.

FIG. 13 is a diagram for explaining the value and meaning of a transfer destination flag 320 provided in the main memory 50.

14 is a diagram for explaining a real address space used in the device of FIG.

FIG. 15 is a diagram for explaining a mapping method from a virtual address space to a real address space, which is used in the device of FIG.

16 is a diagram for explaining an address conversion method used in the device of FIG.

FIG. 17 is a diagram for explaining a method of generating an address for accessing the address conversion table 140.

FIG. 18 is a diagram showing information prepared in the address conversion table 140.

19 is a diagram for explaining the meaning of the information shown in FIG.

20 is a data transfer instruction PUT used in the apparatus of FIG. 1;
FIG.

FIG. 21 is a diagram showing an example of a program using a conventional PUT instruction.

22 is a diagram showing an example of a program used in the apparatus of FIG.

23 is a flowchart of page fault processing executed by the device of FIG.

Claims (27)

[Claims] 1. In a computer system having a plurality of nodes interconnected by a data transfer network, each node having at least one processor and a main memory, data and a transfer destination area for storing the data. The virtual address and the virtual address are sent from the transfer source node to the transfer destination node, and when the transfer destination node receives the data, the transfer destination area to which the received virtual address is allocated is the main memory of the transfer destination node. When the data is received, the data is stored in the transfer destination area to which the virtual address is assigned, and when the transfer destination area is not in the main memory of the transfer destination node when the data is received, the data is transferred. The data is stored in the buffer area for temporary holding provided in the destination node, and after the data is stored in the buffer area, the data is transferred to the transfer destination. When a predetermined command is subsequently issued by the user program running on the node, the data stored in the buffer area is transferred to the transfer destination area, and the transfer is performed by transferring the transfer destination area to the main memory. Data transfer method that is executed after being assigned to. 2. The data transfer method according to claim 1, wherein the predetermined instruction is a memory access instruction requesting access to the transfer destination area. 3. The transfer step executes page fault processing in response to a page fault generated as a result of executing the access instruction when the transfer destination area does not exist in the main memory, and the page fault processing is executed. 3. The data transfer method according to claim 2, further comprising the step of allocating the data and moving the data from the buffer area to the transfer destination area allocated in the main memory. 4. The data transfer method according to claim 1, wherein the predetermined instruction is a predetermined system call to an operating system controlling the transfer destination node. 5. The transfer step is executed by the operating system controlling the transfer destination node by the OS, and the steps other than the transfer step are connected to the processor of the transfer destination node, the main memory, and the network. The data transfer method according to claim 1, which is executed by a circuit. 6. The method further comprising the step of storing a transfer destination flag in the main memory of the transfer destination node, the transfer destination flag indicating a reception state of the data at the transfer destination node, and the transfer destination flag. The data transfer method according to claim 1, wherein said data has first and second values respectively when said data is stored in said buffer area and transfer destination area. 7. In a computer system having a plurality of nodes interconnected by a data transfer network, each node having at least one processor and a main memory, data and a transfer destination area in which the data is to be stored. Sent from the transfer source node to the transfer destination node, the transfer destination node stores the semaphore value specified by the user program being executed there, and is sent from the transfer source node. Whether the comparison value and the value of the semaphore stored in the transfer destination node satisfy a predetermined condition for allowing the data to be stored in the transfer destination area to which the virtual address is allocated. When the determination step determines that the condition is not satisfied, the temporary holding provided in the transfer destination node is determined. A data transfer method for storing the above data in a buffer area for data and transferring the above data from the above buffer area to the above destination area in response to a transfer request issued thereafter by the above program. 8. The data transfer method according to claim 7, wherein the semaphore is held at a position accessible by a user program, and the address of the position is sent together with the data from the transfer source node. 9. The transfer destination node further comprises a step of storing a transfer destination flag in an area accessible by a user program in the main memory of the transfer destination node, wherein the transfer destination flag is a reception state of the data in the transfer destination node. 8. The data transfer method according to claim 7, wherein said transfer destination flag has first and second values when said data is stored in said buffer area and said transfer destination area, respectively. 10. The data transfer method according to claim 9, wherein the address at the position where the transfer destination flag is held is sent together with the data from the transfer source node. 11. The transfer destination node notifies the transfer source node of reception status information indicating the progress of reception of the data at the transfer destination node, and the transfer source node responds to the reception status information notified from the transfer destination node. And further storing the transfer source flag in an area accessible by the user program in the main memory of the transfer source node, wherein the transfer source flag is such that the data is stored in the buffer and the transfer destination area. Sometimes the first and second
10. The data transfer method according to claim 9, which has a value of. 12. The semaphore is held in a main memory of the transfer destination node at a position accessible by a user program, and the address of the position where the semaphore is held and the transfer destination flag are held. The address of the position and the address of the position in the main memory of the transfer source node in which the transfer source flag is held are sent from the transfer source node together with the data, and the step of notifying is performed by the transfer source node. 12. The data transfer method according to claim 11, further comprising the step of notifying the transfer source node of the address at the position where the transfer source flag notified from is held. 13. The transfer destination node notifies the transfer source node of reception state information indicating the progress of reception of the data at the transfer destination node, and the transfer source node responds to the reception state information notified from the transfer destination node. And further storing the transfer source flag in an area accessible by the user program in the main memory of the transfer source node, wherein the transfer source flag stores the data in the buffer and the transfer destination area. When the first and second
10. The data transfer method according to claim 9, which has a value of. 14. The transfer destination node detects whether or not the transfer destination area assigned with the virtual address exists in the main memory of the transfer destination node when the data is received. The method further comprises a step of storing the data in the buffer area when it is detected by the detection step that the data does not exist in the main memory of the transfer destination node or when the determination step determines that the condition is not satisfied. In the step of storing the data in the transfer destination area, the determination step determines that the transfer destination area exists in the main memory of the transfer destination node at the time of receiving the data and the condition is satisfied. The data transfer method according to claim 7, which is executed when the data transfer is performed. 15. The method further comprises the step of storing a transfer destination flag in an area accessible by a user program in the main memory of the transfer destination node, wherein the transfer destination flag is a reception state of the data in the transfer destination node. 15. The data transfer method according to claim 14, further comprising the step of: 16. The data transfer method according to claim 15, wherein said transfer destination flag has a first value and a second value when said data is stored in said buffer and said transfer destination area, respectively. 17. The transfer destination flag is determined depending on whether the transfer destination area exists in the main memory of the transfer destination node when the data is stored in the buffer because the condition is not satisfied. Has a first value, the transfer destination flag indicates that the data is stored in the transfer destination area because the transfer destination area exists in the main memory of the transfer destination node, and the condition is satisfied. 16. The data transfer method according to claim 15, wherein the data transfer method has a second value when stored or when the condition is satisfied but the transfer destination area does not exist in the main memory of the transfer destination node. 18. The data transfer method according to claim 15, wherein the address at the position where the transfer destination flag is held is sent from the transfer source node together with the data. 19. The transfer destination node notifies the transfer source node of reception status information indicating the progress of reception of the data at the transfer destination node, and the reception status information indicates whether the data is stored in the buffer area or not. At least indicating whether or not the data is stored in the destination area, and the transfer source node transfers the data in an area accessible by the user program in the main memory of the transfer source node in response to the reception status information notified from the transfer destination node. The data transfer method according to claim 15, further comprising the step of storing the original flag. 20. The data transfer method according to claim 19, wherein said transfer source flag has a first value and a second value, respectively, when said data is stored in said buffer and said transfer destination area. 21. The transfer source flag indicates whether or not the transfer destination area exists in the main memory of the transfer destination node when the data is stored in the buffer because the condition is not satisfied. The transfer source flag has a first value, and the transfer source flag has the transfer destination area in the main memory of the transfer destination node, and the data is transferred to the transfer destination area because the condition is satisfied. 20. The data transfer method according to claim 19, wherein the data transfer method has a second value when the data is stored in the storage device or when the condition is satisfied but the transfer destination area does not exist in the main memory of the transfer destination node. 22. The semaphore is stored at a position accessible by a user program in the main memory of the transfer destination node, and the address of the position where the semaphore is held and the address of the position where the transfer destination flag is held. And the address of the position in the main memory of the transfer source node in which the transfer source flag is held is sent together with the data from the transfer source node, and the step of notifying is notified from the transfer source node. 20. The step of notifying the transfer source node of the address of the position where the transfer source flag is held is set.
Data transfer method described. 23. A plurality of nodes, and a data transfer network for connecting the plurality of nodes to each other, wherein each node has at least one processor and a main memory connected to the processor. The main memory, the processor, and a network adapter connected to the network. The network adapter stores data and a virtual address of a transfer destination area in which the data is to be stored, via the network. To the processor, the value, and a transmitter connected to the work, and the data transferred from the one node and the virtual address of the transfer destination area to be received via the network, to the network. The connected receiver and the virtual address received by the receiver are stored in the virtual address. To the real address of the allocated real page, connected to the main memory of each node, the address translation unit of each node and the main memory, and the real page is When present in the main memory, in response to the real address obtained by the address translation unit, the data received by the receiving unit is written in the real page, and the real page is stored in the main memory of each node. And a control circuit for storing the received data in a predetermined buffer area in the main memory of each node when the data is not present inside the processor, and the processor is assigned the virtual address when receiving the data. In response to a request issued by the program executing in the real page when the real page does not exist, the real page in the main storage To the virtual address and the computer system is programmed to transfer the data from the buffer area to the real page after the allocation. 24. The request is a memory access request, and the processor assigns a virtual address to the virtual address during execution of a page fault process for processing a page fault generated by the address translation unit at the time of executing the memory access request. To allocate real pages,
24. The computer system according to claim 23, which is further programmed to transfer the data from the buffer area to the real page allocated with the virtual address. 25. The request is a system call to the operating system of each node, the processor further allocating a real page to the virtual address while processing the system call by the operating system. 24. The computer system of claim 23, wherein the computer system is programmed to transfer the data from the buffer area to the real page assigned the virtual address. 26. A plurality of nodes, and a data transfer network for connecting the plurality of nodes to each other, wherein each node has at least one processor and a main memory connected to the processor. It has the main memory, the processor, and a network adapter connected to the network, and the network adapter is designated by a program running on each node,
Sending the data, the virtual address of the transfer destination area where the data is to be stored, and the comparison value for the data to one of the plurality of nodes via the network, the transmission being connected to the processor, the value, and the work. Section, a receiving section connected to the network for receiving the data transferred from the one node and the virtual address of the transfer destination area and the comparison value for the data via the network, and received by the receiving section. An address translation unit connected to the main memory of each of the nodes, which translates the virtual address into a real address of the real page to which the virtual address is assigned; the reception unit of the node and the address translation unit. The semaphore value specified by the program running on each node connected to the main memory Then, it is determined whether the received comparison value satisfies a predetermined condition, and when the condition is satisfied, the address obtained by the address translation unit with respect to the received virtual page. Write the data received by the receiving unit in a transfer destination area having a real address, and further,
When it is determined that the condition is not satisfied, the main memory of each node has a reception control circuit for storing the buffer area in a predetermined buffer area, and the processor executes the program executed therein. A computer system further programmed to transfer the data from the buffer area to the real page in response to a request issued by the. 27. The reception control circuit stores a transfer destination flag in an area in the main memory of each of the nodes, which is accessible by the program executing in each node, and the transfer destination flag is stored in each of the above-mentioned respective nodes. 27. The computer system according to claim 26, which indicates a reception state of the data at a node, and the transfer flag has a first value and a second value when the data is stored in the buffer area and the transfer destination area, respectively. .