JP3077669B2 - Process Stop Method for Distributed Memory Multiprocessor System - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a process stopping method, and more particularly to a process stopping method for a distributed memory type multiprocessor system including a plurality of nodes connected to a network, each of which holds a group of threads for performing parallel processing. The present invention relates to a process stop method in the checkpoint processing performed.

[0002]

2. Description of the Related Art Japanese Unexamined Patent Publication No. Hei 8-263317 discloses a checkpoint / restart processing system for controlling the stop order of a plurality of processes related to synchronous control (exclusive control) in checkpoint processing. I have.

[0003] Japanese Patent Application Laid-Open No. 2-287858 discloses a restart system in a distributed processing system. In this restart system, whenever a communication control unit in a certain processor transmits and receives data to and from another processor group, a program that causes the communication control unit to perform such processing is saved as checkpoint data. You.

[0004]

However, the technique described in Japanese Patent Application Laid-Open No. 8-263317 is applied to a tightly-coupled multiprocessor system, not to a distributed memory multiprocessor system.
In a distributed memory type multiprocessor system, each processor has a unique (local) memory that cannot be accessed by any process of another processor group. If the checkpoint / restart processing system described in Japanese Patent Application Laid-Open No. 8-263317 is applied to this distributed memory multiprocessor system, a plurality of processes on different processor groups perform synchronous processing for checkpoint processing. May not be able to do so. This is because, when a certain process (one of the synchronous processes) in a certain processor is stopped, the processes on the other processors not sharing the memory recognize that the previous process is stopped. Is not possible. In such a situation, there is a problem that the processes on the other processors continue to wait for a response (which should not be returned) from the stopped process.

The technique described in Japanese Patent Application Laid-Open No. 2-287858 has a problem that checkpoint data cannot be saved at an arbitrary timing. Further, the frequency of saving the checkpoint data may be a factor that degrades the performance of the parallel processing.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a process for stopping checkpoint collection efficiently in a parallel processing process for transmitting / receiving data between nodes even when the number of nodes in a distributed memory type multiprocessor system is large. It is an object of the present invention to provide a method of stopping a process in a distributed memory type multiprocessor system which enables the process to be performed.

[0007] Another object of the present invention is to checkpoint a normal processing of a parallel processing process for transmitting and receiving data between nodes on a distributed memory type multiprocessor system so as not to lower the performance of the process. Another object of the present invention is to provide a method for stopping a process in a distributed memory type multiprocessor system that realizes the function of / restart.

Still another object of the present invention is to provide a distributed memory type multiprocessor system in which a checkpoint can be collected at any time for a parallel processing process of transmitting and receiving data between nodes. An object of the present invention is to provide a method for stopping a process in a memory type multiprocessor system.

[0009]

According to a first aspect of the present invention, there is provided a distributed memory type multiprocessor system in which a plurality of nodes are connected by an interconnection network, and each node has a thread to be processed in parallel. In a memory-type multiprocessor system, the node includes a management process for managing a thread to be processed in parallel,
When a checkpoint request command is input to any of the nodes, the management process of the node with the lowest node number issues a stop request to the thread in the own node, and the thread in the own node stops. After doing
A distributed memo requesting a stop request to a management process of the node having the next node number and waiting until threads of all nodes to be processed in parallel are stopped.
How to Stop Processes in Re-type Multiprocessor System
Wherein the thread transmits data between nodes.
Inter-node synchronization means for synchronizing reception processing, and stop
A key that stores information indicating whether a request has been issued.
Check point information and data transmission to other nodes.
Transmitting / receiving means for receiving, and the management process of the own node.
After notifying that the thread has stopped,
Stopping means for stopping red, and
In the data transmission and reception between nodes,
If the reception synchronization process times out,
Stop request issued based on checkpoint information
Check if a stop request has been issued.
Compare the node number of the node with the node number of the partner node
If the own node has a lower node number,
Stopping own thread by means
You.

[0010] A second aspect of the present invention is a process stop method in the first distributed memory type multiprocessor system, wherein the management process is executed in a local node based on a stop request. In-node stop request means for issuing a stop request to a thread, and in-node stop waiting means for waiting until all threads in the own node are stopped,
When a stop notification is received from all threads in the own node,
Inter-node stop request means for issuing a stop request to the management process of the node having the next node number.

According to a third aspect of the present invention, in the process stop method of the second distributed memory type multiprocessor system, the management process waits until threads of all nodes stop. It is characterized by comprising inter-node stop waiting means and inter-node stop means for notifying each node that threads of all nodes have stopped after threads of all nodes have stopped.

According to a fourth aspect of the present invention, there is provided a process stop method in the distributed memory type multiprocessor system, wherein the management process is a thread of the parallel processing process in the process stop method in the second distributed memory type multiprocessor system. A node number table describing a node number of the node, wherein the inter-node stop request means of the management process issues a stop request to a management process of a node having a next lower node number according to the node number table. Features.

According to a fifth aspect of the present invention, there is provided a process stop method in the distributed memory type multiprocessor system, wherein the management process is a thread of the parallel processing process in the process stop method in the third distributed memory type multiprocessor system. A node number table describing the node number of the node is provided, and the inter-node synchronization means of the thread compares the node number of its own node with the node number of the partner node according to the node number table.

According to a sixth aspect of the present invention, there is provided a process stop method in a distributed memory type multiprocessor system, wherein the process stop method in the third distributed memory type multiprocessor system comprises the steps of:
The inter-node communication means for setting the information of the transmission end and the information of the reception end, wherein the inter-node synchronization means sets the transmission end and the reception end information in the inter-node communication information within a predetermined time. Establish synchronization, if not set within a predetermined time, check whether a stop request has been issued based on the checkpoint information,
If a stop request has been issued, the node number of the own node and the node number of the partner node are compared, and if the node number of the own node is smaller, the own means stops the own thread. I do.

[0015]

[0016]

Embodiments of the present invention will be described below in detail with reference to the drawings.

The hardware which is the object of the present invention is a single memory and a shared memory multiprocessor in which a plurality of processors share one memory as one node. Is a distributed memory type multiprocessor system in which a plurality of nodes are connected by an interconnection network.
In addition, it is assumed that there is an inter-node communication area that can be referred to from each node for synchronization or exclusive control during data transmission / reception.

FIG. 1 shows an overall configuration of a distributed memory type multiprocessor system according to an embodiment of the present invention. In FIG. 1, the system includes a master node 10 that is a node where a master thread exists, a slave node 20 where only slave threads other than the node where the master thread exists, and an inter-node communication area 30. You.

The master node 10 includes a master thread 50a and a management process 60a for managing threads in the master node.

The slave node 20 includes a slave thread 50b and a management process 60b for managing threads in the slave node.

Master node 10 and slave node 2
0 master thread 50a and slave thread 50
b denotes a transmission unit 50-1 and a reception unit 50-2 for transmitting and receiving data between nodes, and a synchronization between the transmission unit 50-1 and the reception unit 50-2 used for transmission / reception of data. And a stopping means 50-4 for stopping the thread after notifying the management processes 60a and 60b when it is determined that the position is a safe stopping position. Point information 50-5 is included.

Master node 10 and slave node 2
0, the management processes 60a and 60b each execute a stop request by setting a checkpoint request flag for a managing thread when a stop request is issued to the own node. ,
After receiving the stop notification from the in-node stop waiting means 60-2 which waits until the thread which has issued the stop request and the threads managed by the management processes 60a and 60b, the node number is changed to the management process of the next node. An inter-node stop requesting unit 60-3 for notifying that a stop request has been issued, an inter-node stop waiting unit 60-4 for waiting until threads to be stopped in all nodes are stopped, After the thread has stopped, the inter-node stopping means 60-5 for notifying the management process of each node that all nodes have stopped, and a process based on the maximum number of used nodes specified by the user when starting the parallel processing process. And a node number table 60-6 storing the node number of the node where the thread of the parallel processing process exists. .

The inter-node communication area 30 includes a transmitting unit 50-
1 and the receiving means 50-2 transmit and receive data between nodes, and the inter-node synchronizing means 50-3 includes inter-node communication information 30-1 used for synchronizing the two.

A feature of the embodiment of the present invention is that, when a stop request is issued during data transmission / reception between the master node 10 and the slave node 20, the transmitting means 50
-1 and the inter-node synchronizing means 50-3 used by the receiving means 50-2 are configured to be able to determine in their own node whether or not the partner node has already been stopped by a stop request by a checkpoint.

Next, the operation of the embodiment of the present invention configured as described above will be described with reference to FIGS.

In the configuration shown in FIG.
However, it is assumed that there are actually a plurality of slave nodes.

At the time of starting the parallel processing process, first, the node with the lowest number that can be currently used is set as the master node 10, and the management process 60a is generated in this master node 10. The management process 60a determines all nodes to be used as slave nodes based on the maximum number of nodes used by the user, and describes the node numbers of these nodes and the master node 10 in the node number table 60-6. I do.

Next, the management process 60a in the master node 10 establishes a connection with the management process 60b in the slave node 20 described in the node number table 60-6, and sets the node number table 60-6 as a slave. It is copied to the node number table 60-6 of the management process 60b of the node 20.
Then, the master thread 50a and the slave thread 50b of the parallel processing process are managed by the management processes 60a and 60b of the master node 10 and the slave node 20.
Are generated, and the calculation proceeds while transmitting and receiving data between the nodes.

Thereafter, when a checkpoint is requested for this parallel processing by a checkpoint request command input by the user, the checkpoint processing is started.

Referring to FIG. 4, when a checkpoint request command is input to the node i by the user, the checkpoint request command is used to determine the master node 0 to which the checkpoint request command is sent. It is inputted to the inter-node stop request means 60-3 of i. Inter-node stop request means 60-3 of node i
Is the master node 0 with reference to the node number table 60-6.
A signal instructing a checkpoint request command is sent to the management process. The node i is a special node that constantly monitors a process group in the parallel processing system. Node i may or may not be distinguished from master node 0. If node i is the same as master node 0,
The checkpoint request command is input to the master node 0 by the user.

Management process 60a of master node 10
When the stop request by the check point is notified via the inter-node stop request unit 60-3, the management process 60a issues a stop request to the master thread 50a by the intra-node stop request unit 60-1, and It waits until the master thread 50a stops by the internal stop waiting means 60-2.

The master thread 50a, which has received the stop request from the intra-node stop request unit 60-1, recognizes the stop request unless data is being transmitted / received to / from another node (for example, at the end of interrupt processing). Then, the stopping means 50-
4, a stop notification is sent to the management process 60a, and the process is stopped.

Thereafter, the management process 60a of the master node 10 which has received the stop request becomes the inter-node stop request means 6
Using 0-3, it is checked from the node number table 60-6 that the node having the next lowest node number after the master node 10 is the slave node 20, and a stop request is issued to the management process 60b of the slave node 20. Then, the management process 60a of the master node 10 waits until the threads of all the nodes in the checkpoint target parallel processing process are stopped by the inter-node stop waiting means 60-4.

The management process 60b of the slave node 20, which has received the stop request from the master node 10, issues a stop request to the slave thread 50b in the slave node 20 by using the in-node stop request means 60-1. The stop waiting means 60-2 waits until the slave thread 50b stops.

At this time, if the slave thread 50b is not transmitting / receiving data to / from another node, the stopping means 50-4 recognizes the management process 60b by the stopping means 50-4 at the time of recognizing the stop request, similarly to the master thread 50a. On the other hand, a stop notification is issued and the operation is stopped.

Next, a case where the slave thread 50b is transmitting and receiving data will be described.

FIG. 2 shows a case where the master thread 50a and the slave thread 50b are transmitting and receiving data.
The slave thread 50b has already stopped receiving the stop request before starting the receiving process by the step 2, and the slave thread 50b has started the transmission process by the transmitting means 50-1 before receiving the stop request, but the master thread 50a has already stopped and has stopped. Therefore, an example of the operation when the transmission process cannot be completed is shown.

In order to synchronize the nodes during transmission and reception of data, the inter-node synchronization means 50-3 causes the thread of the partner node to execute a loop for a predetermined period of time while the thread of the partner node transmits the data to the transmission means 50-. 1 or whether the reception means 50-2 has started data transmission / reception processing is checked by repeatedly referring to the flag in the inter-node communication information 30-1.

When it is confirmed that the synchronization between the nodes has been achieved during this loop, when the transmission / reception of data by the transmitting means 50-1 or the receiving means 50-2 is completed, the stopping means 50-4 causes A stop notification is sent to the management process 60b of the slave node 20 to stop.

However, during this fixed time loop, the nodes may not be synchronized with each other, resulting in a time over.
Therefore, if the node number of the partner node (master node 10) is smaller than the node number of its own node (slave node 20) as shown in FIG. May have already been stopped by the stop request, it is checked whether or not the stop request has been issued. A stop notification is sent to the 20 management processes 60b to stop.

If the node number of the own node is smaller than the node number of the partner node, there is no possibility that the thread of the partner node has stopped. Wait for more threads to synchronize. And transmitting means 50-1
Alternatively, when the data transmission / reception processing by the receiving unit 50-2 ends, the stop unit 50-4 sends a stop notification to the management process 60b of the slave node 20 and stops.

The management process 60b of the slave thread 50b having received the stop notification by any of the above processes
Referring to the node number table 60-6, if there is a next slave thread, the inter-node stop request unit 60-3 issues a further stop request to the management process of the node with the next lowest node number. The process waits until threads of all nodes in the checkpoint target process are stopped by the inter-node stop waiting means 60-4.

When there is no next slave thread, the management process 60b of the slave node 20 sends the inter-node stop notification means 6 to the management process 50a of the master node 10.
By 0-5, it notifies that threads of all nodes have stopped. Further, the master node 20 sequentially notifies all the nodes that the threads of all the nodes have been stopped by the inter-node stop notification unit 60-5.

Thus, the management processes 60a, 60b of each master node 10 and slave node 20
The process can proceed to the creation of a restart file.

Regarding the processing at the time of restart, the above processing guarantees that no synchronization deviation occurs between nodes when the checkpoint is stopped.
For example, a checkpoint / restart processing method which does not target a distributed memory type multiprocessor system as disclosed in Japanese Patent Laid-Open No. 8-263317 can be easily realized. That is, the management process 60a of the master node 10 establishes a connection with the management process 60b of the slave node 20, and notifies the management process that a restart request has been issued. The management process in each node is described in Japanese Patent Application Laid-Open No. 8-263317 or a start processing method is applied to each node. The management process 60b of the slave node 20 determines when the start of node management to be managed ends. Thus, it can be realized by notifying the management process 60a of the master node 10.

FIG. 3 shows transmission means 50-1 and reception means 5
It is a flowchart explaining the processing content of the inter-node synchronization means 50-3 used in the data transmission / reception processing by 0-2.

In a data transmission / reception process in a normal parallel processing process in which a checkpoint request has not been issued, the following process is performed according to FIG.

In the transmitting node, the transmitting means 50-1 to the inter-node synchronizing means 50-3 set a flag indicating that the transmission data has been prepared in the inter-node communication information 30-1. With reference to the information 30-1 (step 301), it is checked whether or not the reception end flag is set by the receiving node (step 302).

If the reception end flag has not been set, since synchronization with the receiving node has not yet been achieved,
It is determined whether or not the time is over after a certain time has elapsed (step 303). If the time is not over, the process returns to the inter-node communication information 30-1 until the nodes are synchronized. 301-
The process waits by repeating the process of step 303.

In the receiving node, the receiving means 50
-2, the inter-node synchronization means 50-3 refers to the inter-node communication information 30-1 (step 301) to check whether a flag indicating that the transmission data is ready is set (step 302). ). If the flag has not been set, as with the node on the transmitting side, since synchronization with the transmitting side has not yet been established, it is determined whether or not time has passed after a certain period of time has elapsed (step 303). If the time is not over, the processing of steps 301 to 303 is repeated and waited until the synchronization is obtained by referring to the inter-node communication information 30-1 again.

Next, when a request to stop a checkpoint is issued while either the node on the transmission side or the node on the reception side repeats the processing of steps 301 to 303 while waiting for synchronization, the following occurs. Perform processing. For example, as shown in FIG.
If 0a is stopped by the checkpoint before the receiving unit 50-2 starts the receiving process, the node of the slave thread 50b on the transmitting side performs steps 301 to 303.
The time runs out while repeating the process.

Thereafter, from the management process 60b of the slave node 20 by the intra-node stop requesting means 60-1,
Checkpoint request flag is checkpoint information 5
It is checked whether 0-5 has been set (step 3
04). If the checkpoint request flag is set, the node number of the receiving node is compared with the node number of the transmitting node with reference to the node number table 60-6 (step 305). Is younger, it can be determined that the slave node 20 has already been stopped at the checkpoint.
The process is notified to the management process 60b and stopped (step 306).

In the above embodiment, the node where the master thread 10 exists is located at the lowest node number,
Since stop requests are issued in order from the node with the youngest node number,
The thread of the node to which the stop request is currently issued can know which thread of the node has already stopped and which node has not stopped. For this reason, it can be determined within the own node whether or not the stop position is appropriate.

Next, specific examples of the present invention will be described in detail.

In this embodiment, it is assumed that the master node 10 is assigned to the node number 0 and the slave node 20 is assigned to the node number 1.

FIG. 2 shows a case where the master thread 50a of the master node 10 and the slave thread 50b of the slave node 20 are going to transmit and receive data. The slave thread 50b is already stopped after receiving the stop request, and the transmission process is started by the transmission unit 50-1 before receiving the stop request. However, since the master thread 50a has already stopped, the slave thread 50b ends the transmission process. It shows an operation example in the case where it cannot be performed.

In this case, the slave thread 50b uses the inter-node synchronization means 50-3 to synchronize the nodes.
Thus, the master thread 50a repeatedly checks the flag in the inter-node communication information 30-1 to determine whether or not the receiving means 50-2 has started the data receiving process while performing a loop for a predetermined period of time.

However, the master thread 50a has already been stopped by a checkpoint stop request, and does not start receiving processing. For this reason, as shown in FIG. 3, a time-out occurs while repeating the steps 301 to 303, and the process proceeds to step 304, where the check is performed by referring to the checkpoint information 50-5 in the slave thread 50b. Check that the point request flag is set.

Next, in step 305, the master node 50a as the receiving node is assigned the node number 0, and the slave node 50b as the transmitting node is assigned the node number 1. Since the number is younger, the slave thread 50b judges that the master node 50a has already stopped, and in step 309, the stopping means 50-4 stops the slave thread 5a.
0b is notified to the management process 60b and the operation is stopped.

Although the present invention has been described with reference to the preferred embodiments and examples, the present invention is not necessarily limited to the contents of the above embodiments and examples. In the embodiment of the present invention, the parallel processing process in the case of two nodes and one slave thread has been described.
The present invention can be applied to a case having a plurality of slave nodes, and there is no limitation on the number of nodes.

The number of threads in each node can be applied to a case where both the master node and the slave node have a plurality of threads, and the number of threads is not limited. However, one management process may be provided for each node even if the number of threads increases. In this case, after receiving the stop notification from all the threads in the node by the stop unit 50-4, the management process starts the inter-node stop request unit 60-
3 notifies the management process of the next node that the stop notification has been issued.

Further, the transmission and reception of data between nodes can be applied to the case where transmission and reception are being performed between arbitrary nodes.

[0062]

As described above, according to the process stop method in the distributed memory type multiprocessor system of the present invention, the following effects can be obtained.

First, in order to perform stop processing sequentially from the youngest node, the inter-node synchronization means in the data transmission / reception processing determines whether or not the other node has already been stopped by means of communication or the like. Can be determined without checking, so that it is possible to judge whether the own node can be stopped or not in the own node. Therefore, in a distributed memory type multiprocessor system having many nodes, parallel processing having many threads Even if a stop request for a checkpoint is issued during data transmission / reception in the process, the stop can be performed with the trouble of the order at most proportional to the number of nodes, so that the stop process is performed efficiently. be able to.

Second, the checkpoint processing including the stop processing is performed only when the checkpoint request is issued. Therefore, the checkpoint processing is performed in the normal processing of the parallel processing process operating in the distributed memory type multiprocessor system. There is no reduction in performance because the stopping process does not have any effect.

Third, since a checkpoint request is issued by an external command, a checkpoint can be collected at any time for a parallel processing process operating in a distributed memory type multiprocessor system.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of a distributed memory multiprocessor system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a specific operation during data transmission and reception between nodes according to the embodiment of the present invention.

FIG. 3 is a flowchart illustrating specific processing contents of an inter-node synchronization unit according to the embodiment of the present invention.

FIG. 4 is a block diagram illustrating a relationship between a special node and a master node according to the embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 10 master node 20 slave node 30 inter-node communication area 30-1 inter-node communication information 50a master thread 50b slave thread 50-1 transmitting means 50-2 receiving means 50-3 inter-node synchronizing means 50-4 stopping means 50-5 check Point information 60a, 60b Management process 60-1 Node stop request means 60-2 Node stop waiting means 60-3 Node stop request means 60-4 Node stop waiting means 60-5 Node stop notification means 60-6 Node number table

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G06F 15/177 681 G06F 9/46 360

Claims (6)

(57) [Claims] In a distributed memory type multiprocessor system in which a plurality of nodes are connected by an interconnection network and each node has a thread to be processed in parallel, a management process in which each node manages a thread to be processed in parallel When a checkpoint request command is input to any of the nodes, the management process of the node with the smallest node number issues a stop request to the thread in the own node, and After the thread has stopped,
It issues a stop request to the management process of the node having the next node number, and waits until threads of all nodes to be processed in parallel are stopped.
Distributed memory type multi-processor
A process stop method in Ssashisutemu, the thread, Roh for synchronizing transmission and reception of data between the nodes
It stores inter- node synchronization means and information indicating whether a stop request has been issued.
Transmission / reception means for transmitting / receiving checkpoint information and data to / from another node
And the own thread stops for the management process of the own node
Stop means to stop own thread after notifying that
With the door, the node synchronization means, in the data transmission and reception between nodes, the data transmission and reception synchronization
If the process times out,
Whether a stop request has been issued based on the point information
And if a stop request has been issued,
Node number and the node number of the partner node.
If the node number is smaller,
The thread is stopped. 2. The in-node stop requesting means for the management process issuing a stop request to a thread in the own node based on a stop request, and the in-node stop waiting means for waiting until all the threads in the own node stop. And, when the stop notification is received from all the threads in the own node,
2. A process stop method in a distributed memory type multiprocessor system according to claim 1, further comprising inter-node stop request means for issuing a stop request to a management process of a node having a next node number. 3. The inter-node stop waiting means wherein the management process waits until threads of all nodes are stopped, and a node which notifies each node that the threads of all nodes have stopped after the threads of all nodes have stopped. 3. A process stopping method in a distributed memory type multiprocessor system according to claim 2, further comprising an inter-stop means. 4. The management process includes a node number table describing a node number of the node in which the thread of the parallel processing process exists, and the inter-node stop request unit of the management process uses the node number table. 3. The process stop method in the distributed memory type multiprocessor system according to claim 2, wherein a stop request is issued to a management process of a node having the next lowest node number. 5. The management process includes a node number table describing a node number of the node in which the thread of the parallel processing process exists, and the inter-node synchronization unit of the thread includes a self-node according to the node number table. 4. The process stop method in the distributed memory type multiprocessor system according to claim 3 , wherein the node number of the other node is compared with the node number of the partner node. 6. In the transmission / reception of data between nodes, there is provided inter-node communication information for setting transmission end information and reception end information, and wherein the inter-node synchronization means sets the inter-node communication information within a predetermined time. Synchronization is established when the transmission end and reception end information are set, and when it is not set within a predetermined time, it is checked whether or not a stop request has been issued based on the checkpoint information. 4. The method according to claim 3 , wherein the node number of the own node is compared with the node number of the partner node, and if the node number of the own node is smaller, the own means stops the own thread. 3. A process stop method in the distributed memory type multiprocessor system according to 1.