JP2000215182A - Cluster type parallel computer system and inter- processor barrier synchronizing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a barrier synchronizing method for a cluster type parallel computer to attain the effective participation of an optional processor without hierarchizing the intra-node and inter-node networks after discriminating them from each other. SOLUTION: The number of processors 21 which participate in the barrier synchronization is stored as the initial value in an internal register 12 of a barrier synchronization management part 10 of a barrier synchronizing device 1 which is connected to an inter-node network 3. Then the initial value is copied to a synchronizing counter 11, and the value of the counter 11 is subtracted by one for every barrier synchronization request 40 given from each processor 21 participating in the barrier synchronization. When the value of the counter 11 is set a 0, a barrier synchronization formation notification 41 is issued to every node 2. Based on the notification 41, the bit of a state register 22 of the node 2 is inverted and every intra-node processor 21 can detect the formation of the barrier synchronization by comparing the inverted bit value with the bit value set before the request 40 is issued.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a cluster-type parallel computer system including nodes having a plurality of processors, and more particularly to a barrier synchronization between processors which spans between nodes and in which a plurality of processors in the nodes participate.

[0002]

2. Description of the Related Art An example of a conventional inter-processor barrier synchronization method is described in Japanese Patent Application Laid-Open No. Hei 7-234841. Less than,
The prior art will be described with reference to the publication.

In a parallel computer system, a synchronization mechanism is indispensable for advancing the processing jointly while mutually confirming that each element processor has completed a certain processing. In response to this demand, a technique called barrier synchronization has been known in which all processors to be synchronized have reached a certain stage called a program barrier, and then advance to the next stage. I have.

In the "parallel processing computer" described in the above publication, a barrier synchronization control device connected to an inter-node connection network
14 (shown in FIG. 7), and as shown in FIG.
A synchronization control unit 32 is provided therein. Barrier synchronization control device 14
Has an AND gate 60 and a group phase register 58 for inputting the barrier synchronization confirmation signal 52 from the synchronization control unit 32 of each node P. As shown in FIG. 9, the synchronization control unit 32 of each node P includes a processor phase register 61 and a group phase register 58 in the barrier synchronization control device 14.
Judgment circuit that receives the group phase signal 56 from
Has 62.

In the initial state, the processor phase register 60 of each node P is set to "0" and the barrier synchronization control device 1
The state of the fourth group phase register 58 is “1”. In performing the synchronization, at each node P participating in the synchronization, when the program reaches the barrier, the instruction execution mechanism 30 inverts the bit of the processor phase register 61 to “1”. When bit inversion occurs, the match determination circuit 6
The result of the comparison between the value of the group phase register 58 and the value of the processor phase register 61 becomes “true”, which is sent to the AND gate 60 in the barrier synchronization control device 14. When all the signals from the participating nodes P become “true”, the output of the AND gate 60 becomes “true”, the barrier establishment signal 54 becomes “true”, and is delivered to each node P to establish barrier synchronization. .

At the same time, the barrier synchronization control device 14 inverts the bit of the group phase register 58 to "0". The inverted bit is sent to the synchronization control unit 32 of each node P,
The match determination circuit 60 detects a mismatch with the value of the processor phase register 61 which is already “1”, and sends “false” to the barrier synchronization control device 32 to wait for the next barrier synchronization. .

Since the AND gate 64 in FIG. 9 is not directly related to the present invention, its description is omitted, and the output of the coincidence determining circuit 62 is the output of the AND gate 64 as it is.

[0008]

However, in the above-described conventional technique, when a plurality of processors participating in a barrier exist in a node, it is necessary to hierarchically synchronize barrier synchronization between the inside of the node and between the nodes. There is a problem that processing becomes complicated. The reason is that when each participating processor updates the processor phase register 60, bit inversion occurs multiple times and barrier synchronization breaks down, or barrier synchronization between nodes is determined only by the state of the processor that first inverted the bit. This is because it will be established and it will either fail.

It is therefore an object of the present invention to provide a cluster-type parallel computer system and an inter-processor barrier synchronization method which can achieve high-speed inter-processor synchronization across clusters without hierarchization.

[0010]

SUMMARY OF THE INVENTION A cluster type parallel computer system according to the present invention is a cluster type parallel computer system in which nodes having at least one processor are connected by an inter-node network. Means for storing the number of processors in a node participating in barrier synchronization in the barrier synchronization means, and a count that is copied as an initial value and is decremented by one for each barrier synchronization request from each processor participating in barrier synchronization. Means, a means for issuing a barrier synchronization establishment notification to the node when the count value becomes 0, and means for simultaneously performing the copy again. Validation is confirmed by confirming that the assigned bit of the status register on the node is inverted by the notification of establishment. (A) detecting establishment of synchronization.

The present invention also provides a method for synchronizing barriers between processors in a cluster-type parallel computer system in which nodes having one or more processors are connected by an inter-node network.
A procedure for holding the number of processors participating in barrier synchronization as an initial value, a procedure for setting the initial value in the counting means, and subtracting one from the value of the counting means according to a synchronization request from the processor participating in the barrier synchronization. Procedure, a step of detecting the establishment of barrier synchronization when the value of the counting means becomes 0, and a procedure of notifying a node including one or more processors that participate in the barrier synchronization that barrier synchronization has been established, A step of setting the initial value in the counting means again with the establishment of the barrier synchronization, a step of reading the value of a state register in a node by a processor participating in the barrier synchronization, and the state register by the barrier synchronization establishment notification. The processor detects that the barrier synchronization has been established by updating the status register. And features.

In the present invention, the part corresponding to the processor phase register 61 in FIG. 9 is not updated by each processor in the node, and the part corresponding to the barrier synchronization control device 14 in FIG. It is characterized in that it is updated with. That is, the barrier synchronization request is sent directly from each processor to the barrier synchronization control device, where the synchronization is determined and the value on the node is changed. Therefore, there are a plurality of processors participating in the barrier synchronization in the node. However, the effect that the barrier synchronization can be correctly determined can be obtained.

[0013]

Next, embodiments of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 1, there is shown an embodiment of a cluster type parallel computer system according to the present invention.

In FIG. 1, one or more barrier synchronization devices 1 exist in the system, and one or more barrier synchronization management units are provided.
10, the barrier synchronization request 40 is received from the inter-node network 3, and the communication device 31 for notifying the establishment of the synchronization and the barrier synchronization management unit 10 corresponding to the barrier synchronization request 40 are identified, and the message ID selector 15 for sorting
And Further, each barrier synchronization management unit 10 includes a counter 11 for barrier synchronization and a register for holding the initial value.
12, has a barrier synchronization establishment determination unit 13, and a subtractor 16,
Barrier synchronization request 40 from each processor 21 on each node 2
, The establishment of barrier synchronization is determined, and the synchronization notification 41 is made via the communication device 31.

The node 2 is a component of the cluster-type parallel computer system, and is an arithmetic computer having one or more processors 21. Each node 2 includes a state register 22 for obtaining a state of barrier synchronization and a barrier synchronizer 1
Communication device 23 for communication with the

Next, the operation of inter-processor synchronization in the cluster type parallel computer system of FIG. 1 will be described with reference to FIG.

Prior to the barrier synchronization, each processor 21 participating in the barrier synchronization performs
21 synchronization groups are determined (step S200), and the node 2
The group configuration is given to the above OS, and a request is made to allocate one of the barrier synchronization management units 10 (step S201).

The OS that has received the request arbitrates with the OS of the other node 2 to secure an available one in the barrier synchronization management unit 10 (step S210). Next, the OS gives the barrier synchronization ID (step S211), and notifies the barrier synchronization device 1 of the number of processors in the group and the barrier synchronization ID (step S212). Lastly, the OS maps one vacant bit of the status register 22 to the barrier synchronization ID, and notifies each processor 21 of the barrier synchronization group of the assigned barrier synchronization ID and the bit position of the status register 22 ( Step S213).

When the barrier synchronizer 1 receives a notification from the OS (step S212) while waiting for a request (step S220),
The vacant barrier synchronization management unit 10 is mapped to the notified barrier synchronization ID (step S221), and an initial value is passed to the mapped barrier synchronization management unit 10 to request initialization (step S222).

When receiving the initialization request while waiting for the request (step S230), the barrier synchronization management unit 10 mapped in step S221 sets the value passed to the register 12 (step S231). The barrier synchronization management unit 10
Is set to the counter 11 (step S232), the process waits for a barrier synchronization completion notification (step S230). At this time, the barrier synchronization management unit 10 must be uniquely determined between clusters, but the status register
22 bits and the assigned barrier synchronization manager 10
Whether or not is mapped may be different for each node 2.

Each processor 21 performs a calculation process (step S20).
When 2) advances and reaches the synchronization point (step S203), the state register 22 assigned at the start of barrier synchronization
The value is read from the bit position (step S204).
Next, the processor 21 transmits a message of the barrier synchronization request 40 to the barrier synchronization device 1 through the communication device 23 (Step S205).

In the barrier synchronization device 1, when a barrier synchronization request 40 message transmitted from each processor 21 in step S205 arrives while waiting for a request (step S220), the ID selection unit 15 transmits the barrier synchronization management unit corresponding to the request. 10 is selected (step S223), and the message of the barrier synchronization request 40 is passed (step S222).

In the barrier synchronization management unit 10, when the message of the barrier synchronization request 40 is passed while waiting for the request (step S230), the subtracter 16 decrements the value of the counter 11 by one (step S233). Then, when the result value becomes 0, it means that the barrier synchronization request message has been received from all the processors 21 participating in the barrier synchronization, and the barrier synchronization is established. If it is not 0, it is in a message waiting state (step S230) again (step S234).

When the barrier synchronization has been established, the barrier synchronization management unit 10 issues a barrier synchronization completion notification 41 to each node 2 (step S235). At the same time as issuing the barrier synchronization completion notification 41, the barrier synchronization management unit 10 copies the value of the register 12 to the counter 11 again (step S232), and enters a state of waiting for a barrier synchronization request (step S230). The barrier synchronization synchronization completion notification 41 is received by the communication device 23 on each node 2 and inverts the bit of the status register 22 mapped to the barrier synchronization ID.

The processor 21 that has issued the message 40 of the barrier synchronization request 40 (S205) reads out the bit position assigned to the status register 22 as needed, but the process proceeds to step S235.
Step by the result of status register bit inversion by
When a value different from the value read in S204 is read, the establishment of barrier synchronization is detected (steps S206, S207).

As described above, in the above embodiment, since the barrier synchronization status register 22 is updated only by the barrier synchronization completion notification 41 from the barrier synchronization device 1, each participating processor 21 in the node can directly update the status register 22. Barrier synchronization request
By issuing 40 and directly referring to the status register 22, each processor 21 can independently detect the completion of barrier synchronization. Therefore, it is not necessary to layer barrier synchronization, and each processor 21 can perform barrier synchronization in the same procedure.

Next, a second embodiment of the cluster computer system according to the present invention will be described. In this embodiment, the basic configuration is the same as that of the first embodiment. However, as shown in FIG. 3, equivalent functions are realized only by software without using hardware such as the barrier synchronization device 1. .

The barrier synchronization management node 5 has a barrier synchronization ID
A counter 51 for barrier synchronization identified by
And one or more barrier synchronization management processes 50 including a synchronization establishment determination unit 53 and a communication device 23 for communicating with another node 2 via the inter-node network 3. Any number of barrier synchronization management nodes 5 may exist in the system.

Each node 2 is an arithmetic computer comprising one or more processors 21, and is generated for each communication device 23 for communicating with another node 2 via the inter-node network 3 and for each barrier synchronization ID. And a barrier state variable 25.
The barrier communication device 23 has a function of performing DMA on a memory on another node.

Next, the operation of the embodiment of FIG. 3 will be described with reference to the flowchart of FIG. Prior to the barrier synchronization, each processor 21 participating in the barrier synchronization determines a group of the processors 21 participating in the barrier synchronization (step T200), and assigns one barrier state variable 25 to the node 2 where the participating processor 21 exists. Generate and initialize with 0 (step T20
1). Then, the barrier synchronization management node 5 is notified together with the group configuration, and requests to assign the barrier synchronization management process 50 (step T202).

In the barrier synchronization management node 5, the ID identifying unit 55 waiting for synchronization (step T220) receives this request, issues a new barrier synchronization ID, generates a barrier synchronization management process 50, and generates a barrier synchronization management process. The ID is notified to each processor 21 (step T221).

When generated, the barrier synchronization management process 50 sets the value given to the internal register 52, copies it to the internal counter 51, and further sets the state variable.
57 is initialized with the same 0 as the barrier state variable 25 of each node 2 (step T230), and the request waits (step T231).

Each processor 21 performs a calculation process (step T2).
03), when the synchronization point is reached (step T204), at the start of barrier synchronization, the value of the barrier state variable 25 is read (step T205). Next, the processor 21 is a communication device.
A barrier synchronization request message 40 is transmitted to the barrier synchronization management node 5 through 23 (step T206).

In the barrier synchronization management node 5, the ID identification unit 55
When the message of the barrier synchronization request 40 transmitted in step T206 from each processor 21 arrives while waiting for a request (step T220), the barrier synchronization management process corresponding to the request
50 is selected (step T222), and a barrier synchronization request 40 message is passed to it (step T223).

In the barrier synchronization management process 50, when the message of the barrier synchronization request 40 is passed while the request is waiting (step T231), the value of the counter 51 is decremented by one (step T23).
3). When the result becomes 0, the message has arrived at the barrier synchronization management process 50 from all the processors 21 participating in the barrier synchronization, so that the barrier synchronization is established. If it is not 0, it is in a message waiting state (step T231) again (step T234).

When the barrier synchronization is established, the barrier synchronization management process 50 increments the value of the state variable 57 by 1, and sends the value to the barrier state variable 25 on each node 2 to notify the synchronization 41 of the completion of barrier synchronization. , Using the DMA function of the communication device 23 (step T235). Further, the barrier synchronization management process 50 issues a barrier synchronization completion notification 41 (step T2).
36) At the same time, the value of the register 52 is copied to the counter 51 again (step T232), and a state of waiting for a barrier synchronization request (step T230) is entered.

In step T206, a barrier synchronization request message
The processor 21 that has issued 40 reads the value of the barrier state variable 25 as appropriate, but at the time when a value different from the value read in step T204 is read due to the write result of the synchronization state variable 25 in step T235. Detects the establishment of barrier synchronization (steps T207 and T208).

As described above, in the above embodiment, the update of the barrier state variable 25 is performed except for the initialization except for the barrier synchronization management process.
Since only the synchronization notification 41 of the barrier synchronization completion from 50 is performed, each participating processor 21 in the node 2 issues a barrier synchronization request 40 directly to each, and directly refers to the barrier state variable 25, Processor 21 can independently detect the completion of barrier synchronization. Therefore, it is not necessary to layer barrier synchronization, and each processor 21 can perform barrier synchronization in the same procedure.

Next, as a third embodiment of the present invention, FIG. 5 shows an example in which the cluster system has an atomic operation function for the memory of another node. This embodiment has the same configuration as that of the second embodiment. However, since the system has an atomic operation function for the memory of another node, the synchronization management process 50 does not need to have the state variable 57. I have.

This is because it is not necessary to store the state before the change in order to change the value of the barrier state variable 25 on each node 2 participating in synchronization. Because
This is because the value of the barrier state variable 25 on each node 2 can be changed by +1 by using the atomic operation function without knowing the original value.

FIG. 6 shows the flow of processing in the third embodiment.
Prior to the barrier synchronization, each processor 21 participating in the barrier synchronization determines a group of the processors 21 participating in the barrier synchronization (step U200), and assigns one barrier state variable 25 to the node 2 where the participating processor 21 exists. It generates (step U201), notifies the barrier synchronization management node 5 together with the group configuration, and requests to assign the barrier synchronization management process 50 (step U202).

The barrier synchronization management node 5 receives the request from the ID identifying unit 55 waiting for synchronization (step U220), issues a new barrier synchronization ID, generates a barrier synchronization management process 50, and stores the barrier synchronization ID in the barrier synchronization management node 5. Each processor 21 is notified (step U221).

When generated, the barrier synchronization management process 50 sets the internal register 52 to a given initial value, copies it to the internal counter 51, and furthermore, sets the state variable 57 of each node 2 to It is initialized with the same 0 as the barrier state variable 25 (step U230), and waits for a request (step U231).

Each processor 21 performs a calculation process (step U20).
When 3) advances and reaches the synchronization point (step U204), the value of the barrier state variable 25 is read at the start of barrier synchronization (step U205), and the message of the barrier synchronization request 40 is transmitted to the barrier synchronization management node 5 through the communication device 23. Transmit (step U206).

In the barrier synchronization management node 5, the ID management unit 55
When the message of the barrier synchronization request 40 transmitted from each processor 21 in step U205 arrives while waiting for the request (step U220), the barrier synchronization management process corresponding to the request
Select 50 (step U222), barrier synchronization request message
Pass 40 (step U223).

In the barrier synchronization management process 50, when the message of the barrier synchronization request 40 is passed while waiting for the request (step U230), the value of the counter 51 is decremented by 1 (step U23).
3). When the result becomes 0, the message has arrived at the barrier synchronization management process 50 from all the processors 21 participating in the barrier synchronization, so that the barrier synchronization is established. If it is not 0, it is in a message waiting state (step U230) again (step U234).

When the barrier synchronization is established, the barrier synchronization management process 50 uses the atomic operation function for the other nodes of the system to execute the barrier state variable 25 on each node 2.
Is incremented by 1 to obtain a barrier synchronization completion notification 41. The barrier synchronization management process 50 issues the barrier synchronization completion notification 41 (step U235), and simultaneously counts the value of the register 52 as a counter.
The file is copied again to step 51 (step U232), and a state of waiting for a barrier synchronization request (step U230) is entered.

The processor 21 that has issued the barrier synchronization request message 40 appropriately reads the value of the barrier state variable 25, and the value read in step U204 according to the write result of the synchronization state variable 25 by the above-described atomic function. When a value different from the above is read, the establishment of barrier synchronization is detected (steps U207 and U208).

As described above, in the above-described embodiment, the updating of the barrier state variable 25 is performed except for the initialization except for the barrier synchronization management process.
Since it is performed only by the barrier synchronization completion notification 41 from 50, each participating processor 21 in the node issues a barrier synchronization request 40 directly to each processor 21 and directly refers to the barrier state variable 25 to Can independently detect the completion of barrier synchronization.

At this time, the synchronization state variable 25 of each node 2 does not necessarily have to be initialized to the same value.
Since the state (value) at that time is incremented by +1 by the barrier synchronization management process 50 by the atomic operation, the value immediately before the start of barrier synchronization is stored each time (step U205), and the value and the value of the barrier state variable 25 are stored. Should be compared (step U207). Therefore, it is not necessary to layer barrier synchronization, and each processor 21 can perform barrier synchronization in the same procedure.

The above-described inter-processor barrier synchronization method may be programmed, recorded on a recording medium such as a semiconductor memory or a magnetic disk, read by a computer, and executed.

[0053]

As described above, in the present invention, the value indicating the state of barrier synchronization is updated only from the part that manages the synchronization state, so that barrier synchronization across clusters is not hierarchized. It is effective if it can be performed at high speed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of a cluster-type parallel computer system according to the present invention.

FIG. 2 is a flowchart showing the operation of the first embodiment of FIG.

FIG. 3 is a block diagram showing a second embodiment of the cluster type parallel computer system of the present invention.

FIG. 4 is a flowchart showing the operation of the second embodiment of FIG. 3;

FIG. 5 is a block diagram showing a third embodiment of the cluster type parallel computer system of the present invention.

FIG. 6 is a flowchart showing the operation of the third embodiment of FIG. 5;

FIG. 7 is a block diagram of a barrier synchronization control device in a conventional parallel processing computer.

FIG. 8 is a block diagram of a node in a conventional parallel processing computer.

FIG. 9 is a block diagram of a synchronization control unit in a conventional parallel processing computer.

[Explanation of symbols]

 1 barrier synchronizer 2 node 3 inter-node network 5 barrier synchronization management node 10 barrier synchronization management unit 11 counter 12 register 13 synchronization establishment determination unit 15 ID selection unit 16 subtractor 21 CPU (processor) 22 status register 23 communication device 25 barrier status Variable 26 Atomic operation unit 29 Disk device 31 Communication device 40 Barrier synchronization request 41 Barrier synchronization completion notification 42 Initial value setting 50 Barrier synchronization management process 51 Counter 52 Register 53 Synchronization establishment judgment unit 55 ID identification unit 57 State variable 58 CPU (processor) 59 Disk unit

Claims (5)

[Claims] 1. A cluster type parallel computer system in which nodes having one or more processors are connected by an inter-node network, wherein the intra-node processor participating in barrier synchronization is added to barrier synchronization means connected to the inter-node network. Means for storing the number, counting means for copying the value as an initial value and decrementing it by one for each barrier synchronization request from each processor participating in barrier synchronization, and a barrier when the count value becomes zero. Means for issuing a synchronization establishment notification to the node, and means for simultaneously performing the copy again. Each processor in the node determines the assigned bit of the status register on the node by the barrier synchronization establishment notification. Cluster type parallel meter characterized by detecting the establishment of barrier synchronization by confirming that Machine system. 2. A state in which a state variable is provided in place of the state register, the storage means, the counting means, and the means for performing copying again are realized by software, and the software uses the value of the state variable as an initial value. 2. The cluster type parallel computer system according to claim 1, further comprising a variable, wherein when the count value becomes zero, the state variable is incremented by one and the state variable in the node is rewritten. 3. The cluster-type parallel computer system according to claim 2, wherein said node has an atomic operation function with respect to a memory of another node, and a state variable in said software is made unnecessary. 4. A method for synchronizing barriers between processors in a cluster-type parallel computer system in which nodes having one or more processors are connected by an inter-node network, comprising: a step of retaining the number of processors participating in barrier synchronization as an initial value; A step of setting the initial value in the counting means; a step of subtracting one from the value of the counting means in accordance with a synchronization request from a processor participating in the barrier synchronization; and a step in which the value of the counting means becomes zero. A step of detecting the establishment of the barrier synchronization; a step of notifying the establishment of the barrier synchronization to a node including one or more processors participating in the barrier synchronization; and setting the initial value in the counting means again with the establishment of the barrier synchronization. And a processor participating in the barrier synchronization reads a value of a status register in the node by a reader. And updating the value of the status register according to the barrier synchronization establishment notification, wherein the processor detects that barrier synchronization has been established by updating the status register. . 5. An inter-processor barrier synchronization method in a cluster-type parallel computer system in which nodes having one or more processors are connected by an inter-node network, wherein the number of processors participating in barrier synchronization is held as an initial value. A step of setting the initial value in the counting means, a step of subtracting one from the value of the counting means according to a synchronization request from a processor participating in the barrier synchronization, and a value of the counting means being zero. Detecting the establishment of the barrier synchronization at the point of time; notifying the node including one or more processors participating in the barrier synchronization of the establishment of the barrier synchronization; and A step of setting a value, and a processor participating in the barrier synchronization reads a value of a status register in the node. Forward and, recording a computer readable recording medium storing a program for executing the method on a computer and a procedure for updating the value of the status register by the barrier synchronization establishment notification.