JP3237599B2 - Multiprocessor system and data transfer method in multiprocessor system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a multiprocessor system and a data transfer method in the multiprocessor system, and more particularly, to a multiprocessor system in which communication hardware of each node supports a data transfer function using a virtual address.

[0002]

2. Description of the Related Art Conventionally, in a multiprocessor system in which a plurality of nodes are connected via a network, when data is transferred between nodes, a method using an operating system (OS) function is generally used. Was. However, in this method, there is a problem that a communication delay time for making a system call to the OS becomes long and that data must be copied between a user space and a system space, so that a data transfer speed becomes slow. there were.

[0003] In recent years, a distributed memory type parallel machine or a cluster type multiprocessor system which supports data transfer by a virtual address is used as a function of communication between nodes. In such a system, the user process enables data transfer by communication hardware without going through the OS. However, in such a system, there is no guarantee that the buffer to store the transmitted data and the received data exists in the main storage device, and the address translation may fail and the operation of the communication hardware may stop. was there.

In order to solve this problem, Japanese Patent Laid-Open Publication No.
Japanese Patent Application Laid-Open No. 856 discloses a method in which a communication area is specified in advance and the part is registered in the OS so as not to be paged out, so that a failure in address conversion does not occur during data transfer (first method). Has been proposed.

Further, when the address translation fails, the transmission or reception of data is stopped, the operating system is called by an interrupt from communication hardware, the transmission or reception of data is temporarily stopped, and the page fault is stopped. A method (second method) of restarting the transmission or reception of data after performing the processing for (2) has also been proposed.

[0006]

However, in the first method, in order to transfer arbitrary data in the user process, it is necessary to copy the data to the communication area after all, and the data transfer speed is reduced. There was a problem of letting it. In the second method, transmission or reception of data is waited during the processing of the page fault, so that there is a problem that it takes time to complete the transfer of all data.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the conventional example, and a multiprocessor system capable of performing high-speed data transfer between a plurality of nodes and a multiprocessor system having such a multiprocessor system. It is intended to provide a data transfer method.

[0008]

To achieve the above object, a multiprocessor system according to a first aspect of the present invention is a multiprocessor system comprising a plurality of nodes connected to each other via a communication path. At least one of the plurality of nodes includes a first storage unit for providing a physical address space, a second storage unit for providing a logical address space, and a physical address for each predetermined address translation unit. First address conversion means for performing an address conversion between a logical address and a logical address; first transmission means for transmitting data existing in the logical address space to another node via the communication path; Out of the data transmitted by the transmitting means, the information relating to the address translation unit which did not exist in the physical address space provided by the first storage means. A first conversion unit information recording unit for recording the data, and an address conversion unit corresponding to the information recorded in the first conversion unit information recording unit after the data transmission by the first transmission unit is completed. Retransmission means for retransmitting the data to the other node via the communication path, the last transmission of the transmitted data
The unit contains information indicating that the transmission unit is the last one.
And at least one of said plurality of nodes
Means that the first transmitting means is the last of the transmission units
When transmitting data, the first conversion unit information record
First conversion unit information referring to information recorded in the means.
Report reference means and the first conversion unit information reference means.
As a result of the reference, one of the address translation units is
Does not exist in the physical address space provided by the storage means
The last transmission unit.
Information about the first conversion unit information.
A first conversion unit information control means for recording the recording means, before
As a result of reference by the first conversion unit information reference means,
One of the address translation units is provided by the first storage means.
Indicates that it did not exist in the physical address space.
Data of the last transmission unit by the first transmission means.
And transmission stop means for stopping transmission of data.

[0009]

[0010]

In the multiprocessor system, an address translation unit that does not exist in a physical address space, for example, a page that has been paged out to the second storage unit, of the data transmitted by the transmission unit may be information related thereto. (For example, a page number) is recorded in the first conversion information recording means, and then retransmitted by the retransmission means. For this reason, it becomes possible to issue a page-in request at the time of recording to the conversion information recording means and to continue data processing in parallel with the request (pre-paging). As a result, the page-in process and the data transfer can be overlapped, and the data transfer can be speeded up as a whole.

The retransmitting means may retransmit the address translation unit according to the information recorded in the first translation unit information recording means. Further, the first address conversion means may perform the address conversion in the address conversion unit according to the information recorded in the first conversion unit information recording means. Therefore, unnecessary address translation and data retransmission are not required, and waste can be prevented. The last transmission unit is the retransmission unit by the retransmission unit.
Even if data is sent, it is always sent last in the entire data.
Therefore, for example, the last one bit in the data
Is used as a flag indicating reception completion,
It is possible to operate correctly.

[0013]

[0014]

In order to achieve the above object, a second aspect of the present invention is provided.
Perspective multiprocessor systems communicate with each other
Network consisting of multiple nodes connected via a
Multiprocessor system, wherein the plurality of nodes
At least one of which provides a physical address space.
1 storage means and a second storage means for providing a logical address space.
Storage means and a physical address for each predetermined address conversion unit.
First address for performing address conversion to and from a logical address
Data conversion means and data existing in the logical address space.
Transmission to other nodes via the communication path
Means, and the data transmitted by the first transmitting means.
The physical address provided by the first storage means.
Address translation unit that did not exist in the address space
First conversion unit information recording means for recording information;
After the transmission of the data by the transmitting means is completed,
For the information recorded in the first conversion unit information recording means,
The corresponding address translation unit data is transmitted via the communication path.
Resending means for resending to the other node, wherein at least one of the plurality of nodes has a third storage means for providing a physical address space, and a fourth storage means for providing a logical address space. Storage means; second address translation means for performing address translation between a physical address and a logical address for each predetermined address translation unit; and receiving data transmitted from at least one of the plurality of nodes. Receiving means, and second translation unit information recording means for recording information about an address translation unit in which a physical address corresponding to a logical address at which the data received by the receiving means is to be stored does not exist in the third storage means, Wherein at least one of the plurality of nodes is a conversion unit that reads information recorded in the second conversion unit information recording means via the communication path. Information reading means, and means for causing the second address converting means to perform address conversion of data corresponding to the information read by the conversion unit information reading means and writing the converted data from the fourth storage means to the third storage means. Wherein the retransmitting means further includes data of an address conversion unit corresponding to the information read by the conversion unit information reading means, from at least one of the plurality of nodes via the communication path among the plurality of nodes. Other
It is characterized by resending to at least one .

[0016] In the multiprocessor system having the above-mentioned configuration, in the multiprocessor system, the receiving means converts the data of the address conversion unit which does not exist in the physical address space provided by the third storage means, from the received data. May be destroyed.

In this case, the physical address for storing the address translation unit received by the receiving node is the third address.
For example, among the pages in which the received data is to be stored and which are paged out to the fourth storage means, the information (for example, page number) relating to the page is stored in the second conversion information. After being recorded in the recording means, it is retransmitted by the retransmitting means. For this reason, it becomes possible to issue a page-in request at the time of recording to the conversion information recording means and to continue data processing in parallel with the request (pre-paging). As a result, the page-in process and the data transfer can be overlapped,
Data transfer can be speeded up as a whole.

In this case, the last transmission unit of the data to be transmitted also includes information indicating that the transmission unit is the last transmission unit. At least one of the second and third conversion units refers to information recorded in the second conversion unit information recording unit when the reception unit receives the last data of the transmission unit. Conversion unit information reference means of
When the result of the reference by the second conversion unit information reference means indicates that a physical address corresponding to a logical address to store any address conversion unit does not exist in the third storage means, Second conversion unit information control means for causing the second conversion unit information recording means to record information on the address conversion unit including the last transmission unit,
As a result of the reference by the second conversion unit information referring means,
One of the address conversion units is converted to a logical address to be stored.
A corresponding physical address does not exist in the third storage means.
When indicating that the last transmission unit data
And a discarding means for discarding the information .

In this case, the last transmission unit is always transmitted last in the entire data even if there is retransmission by the retransmission means. For example, the last one bit in the data is received. Even a program used as a flag indicating completion can be operated correctly.

In the multiprocessor system according to the first and second aspects, at least one of the plurality of nodes stores a logical address space with respect to a physical address space provided by the first storage means. Further provided is a fifth storage means, and a second transmission means for transmitting an address translation unit according to a request from at least one of the plurality of nodes, wherein at least one of the plurality of nodes includes After the transmission of the data by the first transmission unit is completed, whether an address translation unit corresponding to the information recorded in the first translation unit information recording unit is stored in the fifth storage unit. Determining means for determining whether or not the result of the determination by the determining means;
Transmission requesting means for making the request to cause the second transmission means to transmit the address translation unit determined to be stored in the fifth storage means via the communication path. be able to.

When an address translation unit of data to be transmitted is stored in the fifth storage means, at least one of the plurality of nodes having the fifth storage means transmits the plurality of nodes to the plurality of nodes. Since the address translation unit is transmitted to at least one of the nodes, the time required for data retransmission can be reduced. Further, it is possible to reduce a burden on at least one of the plurality of nodes on the transmission side.

According to the first and second aspects, the multi-pro
In the Sessa system, the retransmitting means includes the first transmitting means.
Corresponds to the information recorded in the conversion unit information recording means
The data is converted by the first address conversion means into an address.
And writes the data from the second storage means to the first storage means.
Means for storing the data in the first storage means.
The written data is transferred to the other node via the communication path.
Resend to the server.

In order to achieve the above object, a third aspect of the present invention is provided.
Data in a multiprocessor system according to an aspect
The transfer methods are connected to each other via a communication path,
A first storage device that provides a physical address space;
A second storage device for providing a dress space;
Address between the physical address and the logical address for each
Address conversion mechanism for performing address conversion.
In a multiprocessor system composed of
A data transfer method, comprising:
At least one exists in the logical address space of the node
Data to be transmitted to another node via the communication path
A transmitting step; and the data transmitted in the transmitting step.
The physical address provided by the first storage device.
Address translation unit that did not exist in the address space
A first conversion unit information recording step of recording information;
After the transmission of the data in the transmission step is completed,
Corresponds to the information recorded in the first conversion unit information recording step
Address conversion unit data to be transferred to the plurality of nodes.
From at least one of the other nodes via the communication path.
A retransmission step of retransmitting data to at least one of the plurality of nodes, and the other node having received the data transmitted from at least one of the plurality of nodes in the transmission step has a physical address corresponding to a logical address in which the data is to be stored. A second conversion unit information recording step of recording information relating to an address translation unit that does not exist in the first storage device , wherein the retransmitting step includes an address corresponding to the information recorded in the second translation unit information recording step. The data of the conversion unit is further retransmitted from at least one of the plurality of nodes to the other node via the communication path .
You.

[0024]

Embodiments of the present invention will be described below with reference to the accompanying drawings.

[First Embodiment] FIG. 1 is a functional block diagram showing a functional configuration of a multiprocessor system according to this embodiment. As shown, the multiprocessor system includes a plurality of nodes 3-1 to 3-n connected to each other via a network 10.

Each of the nodes 3-1 to 3-n has a similar functional configuration, and includes a CPU (not shown), a memory 13, a secondary storage device 18, and communication hardware 27. On each of the nodes 3-1 to 3-n, the process 11 and the kernel 36 operate. Nodes 3-1 to 3-3 that can be connected in this multiprocessor system
-N can be up to 256.

The memory 13 is a main storage device of each of the nodes 3-1 to 3-n and provides a physical address space. The secondary storage device 18 is configured by a magnetic disk device or the like, and provides a virtual address space to each of the nodes 3-1 to 3-n.

For example, the communication hardware 27 sets the maximum transfer size of data to 256 KB and sets the page size to 4 KB.
It has a function of converting a virtual address of data transferred by the paging method of KB into a physical address. The communication hardware 27 has the functions of an interrupt generation unit 172, a transmission unit 171, a reception unit 272, and a remote read unit 571. The communication hardware 27 further has a request area 176 for writing a data transmission / reception request from the process 11 and a confirmation area 175 for writing information indicating that requested data transmission / reception has been completed. The request area 176 and the confirmation area 175 are mapped in the virtual space.

When the address translation fails, that is, when a page fault occurs for data to be transmitted or received, the interrupt generation unit 172 requests an interrupt from the CPU and causes the kernel 36 to perform the interrupt processing.
The transmitting unit 171 reads the data written in the memory 13, divides the data into packets, and sends the packets to the network 10. The receiving unit 271 writes the data in the packet received from the network 10 to the memory 13. The remote read unit 571 provides a function of reading a reception failure page table 641 or the like of the kernel 36 to another node in response to a remote read request received from the network 10.

The kernel 36 provides basic functions such as inter-kernel communication and virtual memory management, as well as interrupt processing 3
62 and remote page access 761 functions.
The kernel 36 also has a page table 40 for performing address conversion between a virtual address space and a physical address space, and a page table 40 for recording information about a page in which a page fault has occurred at the time of data transmission and reception, respectively. Transmission failure page table 341 and reception failure page table 641
And

The function of the interrupt processing 362 is to specify the interrupt factor when an interrupt occurs in the CPU, and to execute the process if the specified interrupt factor is a failure of the address conversion in the transmission unit 171 or the reception unit 271 (page fault). Stores information on the page for which the address conversion has failed in the transmission failure page table 341 and the reception failure page table 641, respectively. In the interruption process 362, after recording information about the page for which the address conversion failed in the transmission failure page table 341 or the reception failure page table 641, a request for restarting data transmission / reception is written in the request area 176, so that the transmission unit 171 or reception The operation of the unit 271 is restarted.

The transmission failure page table 341 indicates that the transmission unit 171
When a page fault occurs in the data to be transmitted, information about the page whose address translation has failed is recorded. The reception failure page table 641 indicates that the reception unit 271
When a page fault occurs in the data received by the server, the information about the page whose address translation has failed is recorded. Since the maximum size of data that can be transferred by the communication hardware 27 is determined in the transmission failure page table 341 and the reception failure page table 641, as shown in FIG. Record

In these failed page tables, as shown in FIG. 2, the first word contains the page number PNUM of the page whose address translation has failed first. In the second and subsequent words, the value of each bit indicates whether the corresponding page has failed in address translation.
If the value of one bit is “1”, the address conversion has failed in the PNUM + 31 page. If the value of the 31st bit of the third word is “1”, PNUM + 63.
Indicates that address conversion failed for the page. Note that the initial value of PNUM is "-1".

In the multiprocessor system of this embodiment, the process 11 on the data transmission side does not simultaneously transfer data to the processes of a plurality of other nodes. Therefore, in each of the nodes 3-1 to 3-n, only one transmission failure page table 341 is prepared. On the other hand, when the process 11 is on the data receiving side, data may be simultaneously received from processes of a plurality of other nodes.
Are prepared by the number of processes on the transmitting side. In addition,
FIG. 3 shows an arrangement of the transmission failure page table 341 and the reception failure page table 641 in the virtual space.

The remote page access 761 accesses the virtual space of a process in another node connected to the node via the network 10, and accesses the kernel of the node when the accessed page is paged out. Page-in of the page.

The process 11 transmits the data 12 to a process of another node by writing a transmission request in the request area 176 of the communication hardware 27. The process 11 confirms that the transmission of the data 12 has been completed by checking the value of the confirmation area 175 of the communication hardware 27. The process 11 has a reception buffer 22 for writing data received from another node.

The process 11 also refers to the transmission failure page table 341 and the reception failure page table 641 to confirm whether or not the address translation has failed during data transmission / reception. Perform processing. Note that the process 11 includes “0” to “255” so that the process can be uniquely determined in the system.
Are assigned. Note that the data 12 and the reception buffer 22 exist in the virtual space, and may not be present on the memory 13 but may be paged out to the secondary storage device 18.

The operation of the multiprocessor system according to this embodiment will be described below. In the following description, it is assumed that the node 3-1 is a data transmission side and the node 3
-N is the data receiving side. Here, the process 11 of the node 3-1 is referred to as a transmission-side process 11, and the process of the node 3-
The n process 11 is called a receiving process 11.

First, before data transfer is started between the node 3-1 and the node 3-n, the node 3-1
The preparation process executed in each of the node and the node 3-n will be described.

When the kernel 36 generates the process 11, the page table 40, the transmission failure page table 341 and the reception failure page table 6 corresponding to the generated process 11
An area for storing 41 is secured on the memory 13 and designated so as not to be paged out. The kernel 36 further includes a transmission failure page table 341 and a reception failure page table 641.
And the virtual address SND_FTBL_ADD, respectively.
Map to R and RCV_FTBL_ADDR. Also,
The process 11 uses the map function of the kernel 36 to
The request area 176 and the confirmation area 175 are mapped in the virtual space.

Next, the general flow of data transfer from the transmitting node 3-1 to the receiving node 3-n will be described with reference to FIG.
This will be described with reference to the flowchart of FIG. The transmitting process 11 first initializes the transmission failure page table 341 of the own process. The transmitting process 11 further utilizes the transmitting function of the communication hardware 27 to execute the receiving process 1.
The part corresponding to the transmitting process 11 in the reception failure page table 1 is initialized. That is, the transmission-side process 11 writes a predetermined transmission request in the request area 176, and sets the initial value of the reception failure page table 641 to the reception-side node 3
-N is written into the corresponding area (step S11). In addition,
The calculation of the corresponding area is performed according to Equation 1.

## EQU1 ## RCV_FTBL_ADDR + 3 words × process identifier

The transmitting process 11 requests the communication hardware 27 to transmit the data 12 by writing a predetermined transmission request in the request area 176 of the communication hardware 27 (step S12).

The transmitting unit 171 of the transmitting node 3-1 starts transmitting the data 12 to the node 3-n via the network 10 by reading the transmission request written in the request area 176. The data 12 is performed in the form of a packet as described above. If the address conversion fails in the middle of the data 12, the transmitting unit 171 stops transmitting the data 12, and requests an interrupt to the CPU (step S13).

When an interrupt occurs in step S13,
The kernel 36 of the transmission-side node 3-1 performs an interrupt process, and after the interruption process, causes the transmission unit 171 to continue transmitting the data 12 (step S14). The interrupt processing will be described later in more detail.

The receiving unit 271 of the receiving node 3-n stores the data received from the network 10 in the receiving buffer 2.
Write to 2. When the address conversion fails during the data writing, the receiving unit 271 stops receiving the data and requests an interrupt to the CPU (step S15).

When an interrupt occurs in step S15,
The kernel 36 of the receiving node 3-n performs an interrupt process, and after the interrupt process, causes the transmitting unit 271 to continue transmitting the data 12 (step S16). The interrupt processing will be described later in more detail.

On the other hand, the transmission side process 11 confirms that the transmission of the requested data 12 has been completed by referring to the confirmation area 175 of the transmission side node 3-1 (step S17).

When the transmitting process 11 confirms that the transmission of the data 12 has been completed, the transmitting process 11 further transmits the data.
1 with reference to the transmission failure page table 341 and the reception failure page table 641 of the receiving node 3-n to check whether the address translation has failed (step S1).
8). Here, the failure of the address conversion in the transmission unit 171 of the transmission-side node 3-1 occurs when the transmission failure page table 341 is read by normal memory access and the value of PNUM is other than “−1”. You can see that. Receiving node 3-
For the failure of the address conversion in the receiving unit 271 of n, the reception failure page table 641 of the receiving node 3-n is read by remote reading using the remote reading unit 571, and if the value of PNUM is other than “−1”. It can be seen that it has occurred.

If it is confirmed in step S18 that the address translation has failed in either the transmitting section 171 of the transmitting node 3-1 or the receiving section 271 of the receiving node 3-n, the transmitting process 11 performs retransmission processing of the data 12 as necessary (step S19). The retransmission processing will be described later in more detail.

Next, the interrupt processing (steps S14 and S16) executed by the function in the interrupt processing 362 of the kernel 36 of the transmitting node 3-1 and the receiving node 3-n will be described in detail with reference to the flowchart of FIG. explain. When an interrupt occurs, the function of the interrupt processing 40 of the kernel 36 checks the cause of the interrupt (step S21).

If the cause of the interrupt determined in step S21 is the failure of the address conversion in the transmitting unit 171, in this example, the interrupt is generated in the transmitting node 3-1. In this case, the transmission of the data 12 is stopped, and the information on the virtual address of the page for which the address translation has failed is stored in the transmission failure page table 3 of the transmission node 3-1.
41 (step S22). At this time, the interrupt processing unit 362 of the transmitting node 3-1 can issue a page-in request for the page and cause the memory 13 to pre-page.

When the recording in the transmission failure page table 341 is completed, a predetermined transmission request for restarting the transmission of the data 12 is written in the request area 176 of the communication hardware 27, and the transmission section of the transmission node 3-1 is transmitted. 171 is the data 1 from the page following the page for which the address translation failed.
Resume the transmission of 2. However, if the page for which the address conversion has failed is the last page of the data 12 to be transmitted, the transmission processing of the data 12 ends (step S).
23).

If the cause of the interruption determined in step S21 is the failure of the address conversion in the receiving section 271, in this example, the interruption is caused by the receiving node 3-n. In this case, the reception of the data 12 is stopped, and the information on the virtual address of the page for which the address translation has failed is stored in the reception failure page table 6 of the reception side node 3-n.
41 is recorded (step S24). At this time, at this time, the interrupt processing unit 362 of the receiving node 3-n
By issuing a page-in request for the page, the memory 13 can be pre-paged.

When the recording in the reception failure page table 641 is completed, the reception unit 271 of the reception-side node 3-n discards the page for which the address conversion has failed, and receives the data 12 from the next page.

When the cause of the interrupt determined in step S21 is other than the failure of the address conversion, an interrupt process is performed according to the cause of the interrupt, though not shown in the flowchart of FIG.

Next, the retransmission processing (step S19) executed by the transmitting process 11 will be described with reference to FIGS. 6 (a) and 6 (b).
This will be described in detail with reference to the flowchart of FIG. If there is a failure in the address translation in the transmitting node 3-1:
As shown in FIG. 6A, the transmission-side process 11 refers to the transmission-side failed page table 341 of the transmission-side node 3-1 to calculate the virtual address of the page whose address translation has failed (step S31). .

Next, the transmission-side process 11 proceeds to step S
The virtual address calculated in step 31 is accessed to cause a page fault, and the kernel 36 causes the page for which the address conversion has failed to be paged into the memory 13 from the secondary storage device 18 (step S32). Then, the transmitting process 11 retransmits the page that has been paged in (step S33).

On the other hand, when the address conversion has failed at the receiving node, as shown in FIG. 6B, the transmitting process 11 performs the receiving of the receiving node 3-n called by the remote read. Referring to the failed page table 641, the virtual address of the page on which the address translation has failed in the receiving node 3-n is calculated (step S41).

Next, the transmission side process 11
6 using the function of the remote page access 761 to access the virtual address calculated in step S41 of the receiving node 3-n to cause a page fault, and the kernel 36 of the receiving node 3-n translates the address. The page which failed in the second step is paged into the memory 13 from the secondary storage device 18 (step S42). Then, the transmitting process 11 retransmits the data that could not be written to the memory 13 of the receiving node 3-n.

In the above processing, when there are a plurality of pages for which the address conversion has failed in the transmitting node 3-1 or the receiving node 3-2, the transmitting process 11 proceeds to steps S31 and S32 for all the pages. , S41, S
After performing the process of step S42, the processes of steps S33 and S43 are performed.

The kernel 36 performs a page-out process when the free space of the physical space is insufficient.
There is also a possibility that the communication hardware 27 is referring to a page to be paged out. For this reason, when the kernel 36 performs the page-out process and changes the page table 40, the operation of the communication hardware is temporarily stopped so that the address conversion is not performed by the process of the communication hardware 27. I do. And page table 4
After the content of 0 is changed, the operation of the communication hardware 27 is restarted.

Hereinafter, a specific operation example of the multiprocessor system according to this embodiment will be described with reference to FIG. In this example, the data 12 to be transferred is 4
It is assumed that four K-byte pages are included, of which the first and fourth pages exist in the memory 13, and the second and third pages are paged out to the secondary storage device 18.

When the process 11 attempts to transmit the data 12, the head address of the first page 121 is converted to a physical address 0x10000. The process 11 performs one page (0x1000 bytes) from the physical address.
Is transmitted to the transmission unit 171. At this time, in the data transmission in step S13, there is no failure in address conversion.

However, the second page 122 (0x5040)
Since (00 to 0x507FFF) is paged out, the address conversion fails and an interrupt is generated by the interrupt generation unit 172. Thus, the kernel 36 is notified that the address translation has failed in the data transmission in step S13.

In the interruption process 362, first, the page number 0x504 of this page is set to 1 in the transmission failure page table 341.
Record at word. Further, the transmission failure page table 341
Is set to “1” in the second word of the second word (step S
22). The page for which the address conversion has failed is skipped and the transmission proceeds (step S23). Similarly, since the third page is also paged out, the transmission failure page table 34
The first bit of the second word of “1” is also “1”.

After the transmission of the data 12 is completed, the transmission side process 11 checks the failed page table 341 (step S1).
8). Then, the transmitting process 11 knows that the second page and the third page have failed in the address translation on the transmitting side (step S31), and accesses these pages to cause a page fault, and the kernel 36 Then, the first page and the second page are page-in from the secondary storage device 18 to the memory 13 (step S32). Then, the transmitting process 11 retransmits the data of the failed second and fourth pages after the page-in is completed (step S33).

As described above, in the multiprocessor system of this embodiment, the transmission process 11 checks the presence or absence of a page for which the address conversion has failed by referring to the transmission failure page table 341 or the reception failure page table 641. To perform page-in and retransmission processing. For this reason, when it is determined that the transmission process 11 is not necessary, the page-in and retransmission processing do not need to be performed, so that unnecessary page-in processing and retransmission processing need not be performed.

Further, in the multiprocessor system according to the present embodiment, when the address conversion of the data to be transferred fails, only a page-in request can be issued and the data transfer can be continued in parallel. It is. As a result, the page-in process and the data transfer can be overlapped, and the data transfer can be speeded up as a whole.

[Second Embodiment] In this embodiment, the number of times of failure confirmation is changed by changing the failure page table so that failure transmission and retransmission can be performed only once after performing data transfer a plurality of times. Will be described.

The configuration of the multiprocessor system of this embodiment is almost the same as that of the first embodiment (FIG. 1). However, in the multiprocessor system of this embodiment, instead of the failed page table of the first embodiment, a failed page counter and a failed page are used as the data structure for recording the virtual address of the page for which the address translation failed. Use a table pointer.

FIG. 8 shows the structure of the failed page counter. The failed page counter includes a counter for storing the number of pages for which address translation has failed, and a write permission flag for specifying whether or not the kernel records a failed page. The counter is composed of 31 bits and is initialized to "0" by the transmission process before starting data transfer, and is incremented by "1" by the kernel 36 each time an address translation failure occurs. The write permission flag is composed of one bit. When the failed page is not recorded, that is, when it is not necessary to confirm the failed page after the transmission / reception of the data 12, the flag is set to “0”. When recording a failed page, the flag is set to “1”.

The failed page table pointer is a pointer that points to an area for holding a virtual address of a page for which address translation has failed. The transmission process 11 or the reception process 11 secures a recording area of this virtual address before data transfer, and sets the head address of the recording area in the failed page table pointer.

A data structure including such a failed page counter and a failed page table pointer is prepared for each process. However, unlike the case of the first embodiment, it is sufficient to prepare such a data structure as many as the number of process identifiers for reception. FIG. 9 shows an example of the arrangement of the above data structure in the virtual space of the process 11.

The operation of the multiprocessor according to this embodiment will be described below. The kernel 36 executes the process 1
When generating 1, the areas for the failed page counter for transmission and reception, the write permission flag, and the failed page table pointer are secured, and are mapped to the determined virtual address area. The kernel 36 specifies that the area of the failed page counter for transmission and reception, the write permission flag, and the area of the failed page table pointer are not paged out.

The generated process 11 uses the memory allocation function of the kernel 36 to secure and set the failed page table on both the transmitting side and the receiving side prior to data transfer. Since this memory area is accessed by the interrupt processing 172, the kernel 36 specifies that the memory area of the failed page table is not paged out. For example, in the case of the Mach operating system,
By using the vm_wire system call, a memory area such as a failed page counter or a failed page table is designated so as not to be paged out.

Next, the process 11 sets the start address of the memory area as a failed page table pointer for transmission and a failed page table pointer for reception. When the process 11 is a receiving process and does not transmit data, it is not necessary to set a failed page table pointer for transmission. Also, the failed page table pointer for reception is set only for the transmission process to communicate.

Before starting the transmission, the transmission side process 11 (pid = n) sets the failed page counter of the transmission side to “0”.
And the write permission flag on the transmission side is set to “1”. Further, the address (FT_BASE + 2 * (n + 1) * 4) of the failed page counter corresponding to its own process identifier in the virtual space of the receiving process is set to 0x8.
00000000.

Then, the transmission-side process 11 writes a parameter in the request area 176 of the communication hardware 27 and causes the communication hardware 27 to start transferring the data 12.

When the address conversion fails in the processing in the transmitting section 171 of the transmitting node, the interrupt generating section 172 of the communication hardware 27 of the transmitting node generates an interrupt. The function of the interrupt processing 362 of the transmitting side node, when this interrupt is accepted, the transmission process 11
From the status register (not shown) of the communication hardware 27. The function of the interrupt processing 362 refers to the write permission flag of the failed page counter for transmission in the virtual space of the transmitting process 11.

If the value of the write permission flag is “1”, the function of the interrupt processing 362 is to first increment the counter in the failed page counter,
The virtual address of the page for which the address conversion has failed is recorded in a location ((head address) + (counter) * 4) determined from the failed page table pointer and the value of the counter. If the value of the write permission flag is “0”, the interrupt processing 36
In the second function, no operation is performed on the failed page table. When returning from the interrupt processing, the function of the interrupt processing 362 designates to skip the page for which the address conversion has failed, and restarts the data transfer of the transmitting unit 171 of the communication hardware 27 of the transmitting node.

On the other hand, when the address conversion fails in the processing in the receiving unit 271 of the receiving node, the interrupt generating unit 172 of the communication hardware 27 of the receiving node generates an interrupt. Interrupt processing 362 of the receiving node
When the interrupt is accepted, the function of (1) extracts the identifier of the transmitting process 11 and the identifier of the receiving process 11 from the status register (not shown) of the communication hardware 27 of the receiving node. The function of the interrupt process 362 refers to the write enable flag of the reception failure page counter in the virtual space of the reception process 11 that corresponds to the transmission process.

If the value of the write permission flag is "1", the function of the interrupt processing 362 is to first increment the counter in the failed page counter, and further determine the location determined from the failed page table pointer and the value of the counter. For example, if the value of the failed page table pointer is RFTn_A
If the value of the DDR and the counter is k, RFTn_
The address of the page for which the address conversion has failed is written to the address of ADDR + 4 * k.

After performing the data transfer several times, the transmission process 11 checks whether or not the address conversion has failed during the data transfer. The transmission process 11 reads the transmission failure page counter in its own virtual space in order to check the failure of the address conversion in the transmission unit 171 of the transmission side node. Here, if the value of the counter in the failed page counter is larger than "0", a failure has occurred. In this case, the position of the failed page table is identified from the value of the failed page table pointer for transmission, and the failed pages are read therefrom by the counter. That is, the transmission process 11
Reads 4 * k bytes from the address of SFT_ADDR when the value of the failed page pointer is SFT_ADDR and the value of the counter is k.

The transmission process 11 next checks the remote read unit 571 to check for a failure on the receiving side.
Using the function, the address of the failed page counter for reception and the address of the failed page table pointer of the reception process 11 are simultaneously read. Since this area is not paged out, no address translation failure occurs during remote read. here,
If the value of the counter in the failed page counter is larger than "0", a failure has occurred. In this case, using the function of the remote page access 761 provided by the kernel 36, the failed page table in the receiving node is read by the counter. That is, in the transmission process 11, the value of the failed page pointer is RFTn_ADDR, and the value of the counter is k.
, RFT is performed by the remote read function.
It is assumed that 4 * k bytes are read from the address of n_ADDR.

In the multiprocessor system according to the present embodiment, the transmission process 11 must not perform data transfer beyond the previously reserved failed page table. The transmission process 11 is terminated abnormally by the function of the kernel 36 when the transmission process 11 occurs on the transmission side node, and when the reception process 11 occurs on the reception side, the reception process 11 exceeds the failed page table. In order to avoid such abnormal termination, the size of the failed page table initially secured by the user program may be increased.

As described above, in the multiprocessor system according to the present embodiment, it is possible to confirm the failure and retransmit a plurality of data transfers only once. As a result, when a small amount of data at different addresses is sent, the number of times of failure confirmation can be reduced, so that the time required for data transfer as a whole is further reduced as compared with the first embodiment. can do.

[Third Embodiment] In this embodiment, for example, the last bit of the transferred data 12 is used as a flag indicating the completion of data reception. The configuration of the multiprocessor system according to this embodiment is almost the same as that of the first embodiment, except that the following two functions are added as functions of the interrupt generation unit 172 of the communication hardware 27. I have.

That is, when the transmitting unit 171 transmits the data 12, the interrupt generating unit 172 generates an interrupt when the last packet of the data to be transmitted is transmitted. Data 1
When receiving No. 2, an interrupt is generated when the last packet of the data to be received is reached.

The operation of the multiprocessor system according to this embodiment will be described below. Kernel 36
The function of the interrupt processing 362 performs the following interrupt processing in addition to the interrupt processing described in the first embodiment. The operation other than the interrupt processing is the same as the operation in the first embodiment.

FIG. 10 is a flowchart showing the interrupt processing executed by the function of the interrupt processing 362 of the kernel 36 of the transmitting node 3-1 and the receiving node 3-n in this embodiment. When an interrupt occurs, the function of the interrupt processing 40 of the kernel 36 checks the cause of the interrupt (step S21).

If the cause of the interrupt determined in step S21 is the last packet of the data to be transmitted, in this example, it is an interrupt generated in the transmitting node 3-1. In this case, the function of the interrupt processing 362 is as follows.
1 is checked (step S51). Then, as a result of examining the transmission failure page table 341, it is determined whether or not any of the pages has failed in the address conversion in the processing of the transmission unit 171 (step S 52).

If it is determined in step S52 that the address translation has failed in any of the pages, the interrupt processing 3
The function of the transmission-side node 3-62 is that the transmission failure page table 341 regards the last page as having failed in address translation (even if there is actually no failure in address translation).
1 is recorded in the transmission failure page table 341. Then, the transmission of the last packet is stopped (step S53), and this interrupt processing ends.

On the other hand, if it is determined in step S52 that there is no failure in the address conversion for any of the pages, the interrupt processing is terminated. In this case, the last packet will be transmitted.

If the cause of the interrupt determined in step S21 is the last packet of the data to be received, in this example, the interrupt is generated in the receiving node 3-n. In this case, the function of the interrupt processing 362 is as follows.
1 is checked (step S54). Then, as a result of examining the reception failure page table 641, it is determined whether or not any page has failed in the address conversion in the processing of the reception unit 271 (step S55).

If it is determined in step S55 that any one of the pages has failed in the address conversion, the interrupt processing 3 of the receiving node 3-n is performed.
The function of No. 62 is that the receiving node 3-3 determines that the last page in the reception failure page table 641 has failed in address translation (even if there is actually no failure in address translation).
n in the transmission failure page table 641. Then, the last packet received is discarded (step S56),
This interrupt processing ends.

On the other hand, if it is determined in step S55 that no address conversion has failed in any of the pages, the interrupt processing ends. In this case, the last packet received is written to the reception buffer 22 (on the memory 13).

As described above, when the page for which the address translation has failed is retransmitted due to the addition of the interrupt processing, the last page is also retransmitted. The last packet including the indicated flag (the last packet of the last page) is received last.

As described above, in the multiprocessor system of this embodiment, the receiving node 3-n always receives the last packet last. Even a program used as a flag indicating reception completion can be operated correctly.

[Fourth Embodiment] FIG. 11 is a functional block diagram showing a functional configuration of a multiprocessor system according to this embodiment. This multiprocessor system has almost the same functional configuration as the multiprocessor system of the first embodiment (FIG. 1).
6 has a function of a remote data transmission 1062 for transmitting data existing in the secondary storage device 18 to a destination node in response to a request from the kernel of another node.

The operation of the multiprocessor system according to this embodiment will be described below. The operation of the multiprocessor system according to this embodiment is almost the same as that of the first embodiment, except that step S19 is performed.
Only the retransmission processing of Also in the retransmission processing in step S19, the processing in the case where the address conversion has failed in the receiving node 4-n is the same as that in the first embodiment (FIG. 6B).

FIG. 12 is a flowchart showing the retransmission processing executed by the transmitting process 11 when the address conversion has failed in the transmitting node 4-1. When the process starts, the transmission-side process 11 refers to the transmission-side failed page table 341 of the transmission-side node 4-1 to calculate the virtual address of the page for which the address translation has failed (step S).
31).

Next, the transmission-side process 11 proceeds to step S
The page indicated by the virtual address calculated in 31 is the remote disk (the secondary storage device 18 other than the transmitting node 4-1)
Is determined (step S34).

If it is determined in step S34 that the page is not on the remote disk, the transmitting process 11 accesses the virtual address calculated in step S31 to cause a page fault, and causes the kernel 46 to fail the address conversion. The stored page is transferred from the secondary storage device 18 to the memory 1
3 (step S32). Then, the transmitting process 11 retransmits the page that has been paged in (step S33).

When it is determined in step S34 that the page is on the remote disk, the transmitting process 11 uses the function of the remote data transmission request 1062 of the kernel 46 to send the remote data to the kernel 31 of the node. A transmission request is sent (step S35). Then, the processing in the transmission side node 4-1 ends.

Next, the processing in the node that has received the remote data transmission request (hereinafter, referred to as a remote node) will be described with reference to the flowchart in FIG. Upon receiving the remote data transmission request, the function of the remote data transmission 1061 of the remote node determines a page to be read from the secondary storage device 18 of the remote node based on the information included in the request (step S61). ).

Next, the function of the remote data transmission 1061 is as follows.
The page determined in step S61 is read from the secondary storage device 18 (step S62), and the page is transmitted to the receiving node 4-n (step S63). Finally, the function of the remote data transmission 1061 is to notify the requesting sender process 11 that the transmission has been completed to the remote node kernel 4.
The notification is made using the inter-processor communication function provided by 6 (step S64). Thus, a series of retransmission processing ends.

As described above, in the multiprocessor system of this embodiment, since data is transmitted directly from the remote node, data transfer from the remote node to the transmitting node can be reduced, and data retransmission can be performed. Time can be reduced. Further, the burden on the transmitting node can be reduced.

[Modification of Embodiment] In the above-described first embodiment, the process 11 uses the remote page access function 761 provided by the kernel in order to page in the reception buffer 22. A thread dedicated to page access may be provided, and a page may be entered using the thread.

Upon receiving the request message, the page access thread causes a page fault by directly accessing the specified virtual address of the reception buffer 22, and causes the kernel 36 to page in. This request message contains the identifier of the receiving process and a list of virtual addresses to be paged in. This request message can be sent using a communication function using a system call similar to that provided by a conventional kernel.

By using this method, a multiprocessor system similar to that of the first embodiment can be constructed even when the kernel does not have the remote page access function 761.

In the third embodiment, an interrupt is generated when the last packet is transmitted / received, and processing for preventing the last packet from being transmitted / received in the interrupt processing is performed. Send last packet /
The same can be realized when the communication hardware has a function of discarding the last packet or last page without transmitting / receiving the data instead of the interruption at the time of reception.

When such communication hardware is used, the last page is also recorded as failed in the transmission / reception failure page table in the interrupt processing when the address translation fails first. Requests the communication hardware to discard the last packet (or page) without sending / receiving. As a result, the last packet (or page) is not transmitted / received, and when the transmission process later confirms failure, the last page also appears to have failed in address translation. Thus, effects similar to those of the third embodiment can be obtained.

In the above-described first to fourth embodiments, the case where the present invention is applied to the one supporting the paging type virtual storage has been described. However, the sparseness supporting the segment type or the segment paging type virtual storage is described. The present invention can be applied to a combined type multiprocessor system.

[0114]

As described above, according to the present invention,
Data transfer between nodes can be speeded up. Further, unnecessary address conversion and retransmission processing may not be performed.

[Brief description of the drawings]

FIG. 1 is a functional block diagram illustrating a functional configuration of a multiprocessor system according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a failed page table (transmission failed page table and reception failed page table) according to the first embodiment of the present invention.

FIG. 3 is a diagram showing allocation of a failed page table in a virtual space according to the first embodiment of the present invention.

FIG. 4 is a flowchart showing an operation in the multiprocessor system according to the first embodiment of the present invention.

FIG. 5 is a flowchart illustrating an interrupt process executed by a kernel according to the first embodiment of this invention.

FIG. 6 is a flowchart illustrating a retransmission process performed by a process according to the first embodiment of the present invention;
Shows the processing when there is a failure on the transmitting side, and (b) shows the processing when there is a failure on the receiving side.

FIG. 7 is a diagram illustrating an operation example of the multiprocessor system according to the first embodiment of the present invention.

FIG. 8 is a diagram illustrating a configuration of a page counter applied to a second embodiment of the present invention.

FIG. 9 is a diagram showing allocation of a failed page table in a virtual space according to the second embodiment of the present invention.

FIG. 10 is a flowchart illustrating interrupt processing executed by a kernel according to the third embodiment of the present invention.

FIG. 11 is a functional block diagram illustrating a functional configuration of a multiprocessor system according to a fourth embodiment of the present invention.

FIG. 12 is a flowchart illustrating a retransmission process executed by a process according to the fourth embodiment of the present invention, illustrating a process performed when a failure occurs on a transmission side.

FIG. 13 is a flowchart showing a remote data transmission process executed by a process according to the fourth embodiment of the present invention.

[Description of Signs] 3-1, 3-n, 4-1, 4-n Node 10 Network 11 Process 12 Data 13 Memory 18 Secondary Storage 22 Receive Buffer 27 Communication Hardware 36, 46 Kernel 40 Page Table 171 Transmission Section 172 interrupt generation section 175 confirmation area 176 request area 271 reception section 275 page table base 341 transmission failure page table 362 interrupt processing 571 remote read section 641 reception failure page table 761 remote page access 1061 remote data transmission 1062 remote data transmission request

Continuation of the front page (56) References JP-A-57-162164 (JP, A) JP-A-7-64846 (JP, A) JP-A-7-271739 (JP, A) JP-A-9-212474 (JP, A) JP-A-8-305667 (JP, A) JP-A-7-262151 (JP, A) Hiroshi Tezuka, 4 others, "User-level zero-copy communication using pin-down cache", IPSJ research Report Vol. 97 No. 76 (97-ARC-125), Aug. 22, 1997, p. 167-172 Hiroshi Ishikawa, "Parallel Processing Technology Using Commodity Hardware", Information Processing Vol. 39, No. 8, August 15, 1998, p. 784-791 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06F 15/16-15/177 G06F 12/08-12/12 G06F 13/00 G06F 13/38-13/42 JICST file ( JOIS)

Claims (6)

(57) [Claims] A plurality of nodes connected to each other via a communication path;
Multi-processor system consisting of
hand, At least one of the plurality of nodes includes: First storage means for providing a physical address space; Second storage means for providing a logical address space; Physical address and logical address for each predetermined address translation unit
Address translation means for translating addresses between the address and the address
When, The data existing in the logical address space is transferred to the communication path.
First transmitting means for transmitting to another node via The data transmitted by the first transmitting means;
The physical address space provided by the first storage means
Records information about address translation units that did not exist in
First conversion unit information recording means, The transmission of the data by the first transmission means has been completed.
Later, it is recorded in the first conversion unit information recording means.
The data of the address conversion unit corresponding to the information is transmitted to the communication path.
Resending means for resending to the other node via
e, At least one other of the plurality of nodes includes: Third storage means for providing a physical address space; Fourth storage means for providing a logical address space; Physical address and logical address for each predetermined address translation unit
Address translation means for translating addresses between the address and the address
When, Transmitted from at least one of the plurality of nodes
Receiving means for receiving the received data; A logical address to store the data received by the receiving means.
A physical address corresponding to the resource exists in the third storage means.
The second to record information about the address translation unit not
Conversion unit information recording means, At least one of the plurality of nodes includes: Via the communication path to the second conversion unit information recording means
Read the recorded information Conversion unit information reading means to output
When, Corresponds to the information read by the conversion unit information reading means
The data to be converted to the second address converting means.
In other words, from the fourth storage means to the third storage means
Means for writing, The resending means reads the conversion unit information reading means.
Data of the address translation unit corresponding to the
The communication from at least one of the plurality of nodes.
At least another of the plurality of nodes via a communication path
Multiprocessor system characterized by resending to one
Tem. Wherein said receiving means to claim 1, characterized in that discarding data of said third address translation unit that did not exist in the physical address space storing means is provided within the received data A multiprocessor system as described. 3. The last transmission unit of the transmitted data includes information indicating that the transmission unit is the last transmission unit, and at least one of the plurality of nodes includes at least one of the plurality of nodes. A second conversion unit information reference unit that refers to information recorded in the second conversion unit information recording unit when the receiving unit receives last data of the transmission unit; When the result of the reference by the second conversion unit information reference means indicates that the physical address corresponding to the logical address to store any address conversion unit does not exist in the third storage means, a second conversion unit information control means for causing the information about the address translation unit comprising a transmission unit recorded in the second conversion unit information recording means, the result of the reference by the second conversion unit information reference unit,
One of the address conversion units is converted to a logical address to be stored.
A corresponding physical address does not exist in the third storage means.
When indicating that the last transmission unit data
3. The multiprocessor system according to claim 1, further comprising: a discarding unit that discards the data. Wherein at least one still another of said plurality of nodes, a fifth memory means for further providing the logical address space to physical address space of the first storage means is provided, said plurality of A second transmission unit for transmitting an address translation unit according to a request from at least one of the nodes, wherein at least one of the plurality of nodes transmits the data by the first transmission unit. After the processing, the determination unit determines whether an address translation unit corresponding to the information recorded in the first translation unit information recording unit is stored in the fifth storage unit, and the determination by the determination unit. As a result, the request to cause the second transmission unit to transmit the address translation unit determined to be stored in the fifth storage unit is transmitted through the communication path. Multiprocessor system according to any one of claims 1 to 3, a transmission request means, and further comprising a performing. Wherein said retransmission means, said from said second storage means data corresponding to the first conversion unit information recording means information recorded to by the address conversion in the first address translation means 5. The method according to claim 1, further comprising: means for writing to the first storage means, wherein the data written to the first storage means by the means is retransmitted to the other node via the communication path . The multiprocessor system according to claim 1. 6. A first storage device which is connected to each other via a communication path and provides a physical address space, a second storage device which provides a logical address space, and a physical address for each predetermined address conversion unit. A data transfer method in a multiprocessor system including a plurality of nodes having an address translation mechanism for performing an address translation between a logical address and a logical address, the method comprising: A transmitting step of transmitting data existing in a logical address space to another node via the communication path; and a data present in the physical address space provided by the first storage device among the data transmitted in the transmitting step No. 1 for recording information on the address translation unit not performed
After the conversion unit information recording step and the transmission of the data in the transmission step are completed,
A retransmission step of retransmitting data of an address translation unit corresponding to the information recorded in the first translation unit information recording step from at least one of the plurality of nodes to the other node via the communication path; At least one of the plurality of nodes in the transmitting step
The other node that has received the data transmitted from the
Physical address corresponding to the logical address where data should be stored
Address translation unit whose address does not exist in the first storage device
A second conversion unit information recording step for recording information on
And the retransmission step includes the second conversion unit information recording step.
Address conversion unit data corresponding to the information recorded in the
Further determine at least one of the plurality of nodes
And retransmitting the data to the other node via the communication path .