JP2615408B2 - Parallel computer system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel computer system for performing dynamic load distribution.

[0002]

2. Description of the Related Art Conventionally, load equalization executed by each processor in a parallel computer system is described in "Load."
Dispatching Strategy on
Parallel Inference Machine
es ”(FGCS88 Proceedings, Vol. 3, pp 987
993, 1988).

In the above prior art, a plurality of lowermost processor groups are formed from a plurality of processors connected to a shared memory, respectively, and these processor groups are hierarchically connected by a network to form a hierarchical processor group. To achieve. The amount of execution waiting load for each processor is placed in the shared memory of the lowest group to which the processor belongs, and when the representative processor of each lowest group distributes the load of its own group to another group, another processor group The load amount of the own group is controlled based on the load of the other group and the load of the own group.

[0004] Inquiring of the load amount is performed at a break of the normal processing so as not to hinder the progress of the normal processing being executed by the representative processor of the group of the inquiry destination.

[0005]

In the above-mentioned prior art, since the inquiry about the load amount is made at the break of the normal processing at the inquiry destination, the load distribution control waits half the time of the normal processing on average. .

Further, in this prior art, since a message for inquiring about the load from the representative processor of the load distribution source group to another group is transferred, it takes a long time to transfer this message between the two groups. In particular, the transfer time increases as the layers to which these groups belong differ.

An object of the present invention is to solve the above two problems and to provide a parallel computer system for performing efficient dynamic load distribution.

[0008]

[Means for Solving the Problems] In order to solve the above-mentioned purpose
The parallel computers were connected to shared memory
Multiple lower processor groups consisting of multiple processors
And a network for interconnecting the plurality of processor groups.
Network.

[0009] The networks each have one
Are provided corresponding to the processor groups of
Related to processors belonging to the specified processor group.
Multiple communication control processes to relay message transfer
And each of the plurality of processor groups
One of a number of higher-level processor groups
With multiple lower network nodes provided corresponding to
And the corresponding one of the upper processor
Connected to some lower-level processor groups belonging to
Connected to a group of connected communication control processors
And the plurality of lower network nodes interconnected
With at least one upper network node
Shall be.

Further , each lower network node
Some duplicates included in the upper processor group corresponding to
Provided for each of the number of lower processor groups
A plurality of type 1 load registers, and a plurality of
Group of communication control processors connected to
Type 1 net that relays messages to and from Sessa
A work control circuit, and the upper network node
Corresponds to each of the plurality of lower-level processor groups.
A plurality of second-type load registers provided as
Relays messages to lower-level network nodes
And a second type network circuit.

Also, each processor is connected to it.
Negative for the processor in shared memory
Processor completes subprogram processing in load storage area
Program to store the processor load every time
Shall be rammed.

Further, each time the communication control processor completes the message relay program processing, the communication control processor reads a plurality of load storage areas from the plurality of load storage areas in the shared memory provided for the corresponding lower processor group. The load of each processor belonging to the processor group is read, the load of the processor group is calculated by adding the read loads, and the calculated load is stored in a lower network node connected to the communication control processor. Means for writing to a type 1 load register which is a shared register provided for the lower processor group, and a load held in the type 1 load register provided for the corresponding lower processor group. A method for comparing with a distribution threshold provided for the lower processor group If, when the load has been held in the first type load register is more than the distribution threshold value, the host processor grayed the lower processor group to which the corresponding belongs
Means for selecting another lower processor group that belongs to the loop and has the smallest load value, and is held in a first type load register provided corresponding to the selected other lower processor group. And read the load
Means for determining whether to distribute a part of the load of the corresponding lower processor group to the other lower processor group based on the read loads,
If the load value held in the first type load register corresponding to the other one processor group is smaller than the load value of the first type load register corresponding to the lower processor group, the lower processor group will It is assumed that a means for distributing the load to another lower processor group is provided.

Also, the first in each lower network node
Each time the first type of network control circuit completes the message relay program processing, the type of network control circuit reads the plurality of first type load registers provided in the lower level network node from the lower level network node. The load of each lower processor group belonging to the upper processor group corresponding to is read, the load of the corresponding upper processor group is calculated by adding these read loads, and the calculated load is
Means for writing to a second type load register, which is a shared register provided for the corresponding upper processor group, in the upper network node, and a second type provided for the corresponding upper processor group Means for comparing the load held in the load register with a distribution threshold provided for the higher-level processor group; and, when the load held in the second type load register is equal to or greater than the distribution threshold, Other than the corresponding upper-level processor group ,
The means for selecting one upper processor group and the loads held in the second type load registers provided corresponding to the other selected upper processor group are read out, and these read loads are read. Means for determining whether to distribute a part of the load of the corresponding upper processor group to the other upper processor group based on the second type corresponding to the other upper processor group. Means for distributing the load from the upper processor group to the other upper processor group when the load value held in the load register is smaller than the load value of the second type load register corresponding to the upper processor group. Wherein the load is the total number of subprograms waiting to be executed by the processors belonging to the processor group. And those which are.

[0014]

The present invention is provided in a network for holding the load amount of each lower or upper processor group, and the load amount is programmed.
Is set every time the system is completed, and means for determining whether to distribute the load of each processor group with reference to this is provided in the network, so while referring to the latest load value,
Load distribution can be executed without inquiring each processor about the load.

[0015]

Embodiments of the present invention will be described below.

FIG. 1 shows a plurality of processors PE0 to PE7 constituting a parallel computer system and a network connecting them.

One of the features of this embodiment is that the network divides these processors into a plurality of hierarchical processor groups. That is, the shared memory 1 has the processors PE0 to PE0 connected thereto.
Used for communication between PE1. The processors PE0 and PE1 connected to the shared memory 1 form a lowermost processor group. Similarly, the processors PE2 and PE3 connected to the shared memory 2 constitute another lowermost processor group. The processors PE4 and PE5 connected to the shared memory 3 constitute another lowermost processor group. Processor PE connected to shared memory 4
6, PE7 forms another lowermost processor group.

Hereinafter, these lowermost processor groups may be referred to as level 0 processor groups.

In the following, the group number of the group consisting of PE0 and PE1 is # 0, the group number of the group consisting of PE2 and PE3 is # 1, and the group number of the group consisting of PE4 and PE5 is # 2, PE6 and PE7. The group number of the group is # 3.

The processors in each group are connected to communication control processors CC0, CC1, CC2, and CC3 which are provided correspondingly and form a network.

The communication control processors CC0 and CC1 are connected to a network node 50 forming a network.

The communication control processors CC2 and CC3 are connected to a network node 51 constituting a network.

The network nodes 50 and 51 are connected to a higher-level network node 52.

The communication control processor CCi (i = 0, 1,
2 or 3) relays a message transferred between a processor of a level 0 processor group connected thereto and another processor belonging to a group other than the processor group. That is, these communication control processors are for a processor group of level 0.

The network node 50 is connected to the processor PE0 or PE0 connected to the communication control processor CC0.
1 and another processor PE2 or PE3 connected to the communication control processor CC1, or any one of the processors PE0 to PE3 connected to one of the communication control processors CC0 and CC1 and the network node 51.
Relays a message transferred between any of the other processors PE4 to PE7 connected to the other.

In other words, the network node 50
Constitutes a network for transferring messages between processors connected to the communication control processor CC0 or CC1 without passing through the network nodes 51 and 52. Thus, the communication control processor CC
0, the processor connected to CC1 has two levels 0
Of the processor group including the processor group of. Hereinafter, this processor group is referred to as a level 1 processor group. # 0 of this group
Let it be 1. In this embodiment, when a message is transferred to a specific processor, for example, a transfer destination processor number is attached to the message. When sending a message to a particular group, number the message with its forwarding group.

Similarly, the network node 51 forms a network for transferring a message between processors connected to the communication control processor CC2 or CC3 without passing through the network nodes 50 and 52. Thus, the processors connected to the communication control processors CC2 and CC3 form another level 1 processor group. The group number is # 23.

The network node 52 relays a message transferred between a processor connected to the network node 50 and another processor connected to the network node 51. In other words, the network node 52 communicates messages within one processor group including processors belonging to the level 1 group # 01 and processors belonging to the level 1 group # 23 connected to the network nodes 50 and 51. Relay.

As is clear from the above, in this embodiment,
Four level 0 processor groups # 0, # 1, #
2 and # 3, and two level 1 processor groups # 01 and # 23.

Another feature of this embodiment is that the load of each level group is distributed and stored in the network. That is, the network node 50 is provided with the register 10 for holding the sum of the load amounts of the level 0 group # 0 and the register 11 for holding the load amount of the level 0 group # 1.

Similarly, the network node 51 has a register 12 for holding the load amount of the level # 0 group # 2.
And a register 13 for holding the load of the group # 3 of level 0.

Similarly, the network node 52 has a register 0 for holding the load amount of the group # 01 of level 1.
1 and a register 23 for holding the load amount of the level # 1 group # 23.

The area 2 in the shared memories 1 to 4
0 to 27 are the individual processors PE 0 to 7 respectively.
This is an area for holding the load amount. Writing of the load amount to each area is performed by the corresponding processor.

Still another feature of the present embodiment is that the communication control processors CC0, CC1, CC2, and CC3 write the load amounts to the shared registers 10, 11, 12, and 13 for level 0, respectively. Referencing and distributing the load. That is, the communication control processors CC0 and CC1 both share the registers 10 and 11
See The communication control processors CC2 and CC3 both refer to the shared registers 12 and 13.

Similarly, the network control circuit CC01 provided in the network node 50 and the network control circuit CC provided in the network node 51
23 are shared registers 01 and 23 for level 1, respectively.
Write the load amount to Both network control circuits CC01 and CC02 refer to these registers 01 and 23 to distribute the load.

In this embodiment, the processors PE0 to PE7, the communication control processors CC0 to CC3, and the network control circuits CC01 and CC23 can be realized by a microcomputer or the like. In this embodiment, a system having a three-layer structure will be described. However, even when the number of layers increases, the same effect can be obtained by managing the load amount for each layer.

The load of each group is represented, for example, by the total number of commonly used sub-programs waiting to be executed by the processors belonging to the group.

The operation of this embodiment will be described below.

FIG. 2 shows each processor PEi (where i =
0, 1,, or 7) and each communication control processor C
Cj (where j = 0, 1, 2, or 3) and each network control circuit CCk (where k = 01 or 2)
The processing executed by 3) is shown. The network control circuit C03 relays the message, but does not distribute the load.
It is distinguished from CC23.

(1) Each processor PEi, each communication control processor CCj, and the network control circuit CCk
The normal program processing c1 is performed. Here, the program processing performed by the processor PEi refers to execution of a program for which the processor is scheduled, such as processing of one subprogram. The program processing performed by the communication control processor CCi or the network control circuit CCk refers to message relay processing such as reception of one message and transfer of the message.

(2) Each time each processor PEi, communication control processor CCj or network control circuit CCk completes the above-described program processing, a load amount setting processing c2 is performed.

That is, the processor PEi, for example, P
E1 sets the amount of execution waiting load held by its own processor in the corresponding load storage area, for example, area 20 above the corresponding shared memory, for example, 1.

The communication control processor CCj, for example, CC0
Represents the load amount of a plurality of processors belonging to level 0, for example, a plurality of processors belonging to # 0, for example, PE0, 1 at any predetermined time,
The load is read from the shared memory of the group, for example, one load storage area, for example, 20, 21 and added to calculate the load of the processor group.

Further, the calculated load amount is set in a shared register, eg, 10, provided corresponding to the group in a network node, eg, CC01, connected to the communication control processor CCj.

Network control circuit CCk, for example, CC
01 is a level 0 processor group connected to it at any predetermined time, for example, # 0, # 1
Of the shared register corresponding to each group, for example, 10, 1 in the network node including the network control circuit CCk, for example, 50.
1 and add them to calculate the load amount of the level 1 processor group connected to the network control circuit CCk.

Further, the calculated load amount is set in a shared register, for example, 01 provided corresponding to the group in the network node CC03 connected to the network control circuit CCk.

(3) After execution of the process c2, each communication control processor Ccj, for example, CC0 and each network control circuit CCk, for example, CC01, control the control flow shown in FIG.
According to the above, the dynamic load distribution control c3 is performed. That is, (3a) In step d1, each communication control processor CC
j, for example CC0, is a level 0 processor group, for example, # 0, connected to its communication control processor CCj.
Of the corresponding network node, for example, 50
, A shared register corresponding to the group, for example,
10 and is compared with a predetermined threshold constant T. As a result of the comparison, when the load amount of the group is equal to or larger than T, the process proceeds to step d2. If it is less than T, there is a high possibility that the load to be executed by the own processor group for which the load is to be distributed will be insufficient, so the dynamic load distribution is canceled.

Similarly, in this step d1, each network control circuit CCj, for example, CC01 is a level 1 processor group connected to its control circuit CCk,
For example, the load amount of # 01 is read from the shared register corresponding to the group, for example, 01 in the corresponding network node 52, and is compared with a predetermined threshold constant T. As a result of the comparison, when the load amount of the group is equal to or larger than T, the process proceeds to step d2. If it is less than T, the dynamic load distribution control is cancelled.

(3b) In step d2, each communication control processor CCj, eg, CC0, and each network control circuit CCk, eg, CC01, determines a group number of a candidate processor group to be load-balanced.

That is, the communication control processor CCj, for example, CC0, is connected to the communication control processor CCj, and is a level 0 processor group, for example, a level 1 processor group to which # 0 belongs, for example, another level 0 belonging to # 01. One of the processor groups is selected by a random number as a load distribution destination candidate.

In this embodiment, level 0 processor groups belonging to the same level 1 processor group are:
Since there are only two, in this selection the level 0 connected to its communication control processor CCj, for example CC0
, For example, a processor group of level 0 other than # 01, for example, # 1. For this reason, in the present embodiment, the communication control processor CC
0 selects the processor group # 1 as the distribution destination candidate, and the communication control processor CC1 selects the processor group # 0 as the distribution destination candidate. The same applies to the communication control processors CC2 and CC3.

Similarly, the network control circuit CCk, for example, CC01 is a level 1 processor group connected to the network control circuit CCk, for example, # 01.
One of the level 1 processor groups belonging to the level 2 processor group to which is belongs is selected as a load distribution destination candidate by using a random number.

In this embodiment, the level 1 processor groups belonging to the same level 2 processor group are:
Since there are only two, in this selection, a level 1 processor group connected to the network control circuit CCk, for example, CC01, for example, a level 1 processor group other than # 01, for example, # 23 is selected. . Therefore, in the present embodiment, the network control circuit CC01 selects the processor group # 23 as a distribution destination candidate, and the network control circuit 23 selects the processor group # 01 as a distribution destination candidate.

(3c) In step d3, each communication control processor CCj, eg, CC0, and each network control circuit CCk, eg, CC01, associates the determined load distribution of the candidate processor group for load distribution with the group. Read from the shared register.

For example, the communication control processor CC 0 reads out the load of the processor # 1 of level 0 as a distribution destination candidate from the shared register 11. Similarly, the communication control processor CC1 reads out the load amount of the distribution destination candidate level # 0 processor # 0 from the shared register 10.

Similarly, the network control circuit CC01
Reads the load amount of the distribution destination candidate level 1 processor # 23 from the shared register 23, and the network control circuit CC23 reads out the distribution destination candidate level 1 processor # 0.
1 is read from the shared register 01.

(3d) In step d4, each communication control processor CCj, for example, CC0 and each network control circuit CCk, for example, CC01 read the load amount of the distribution destination candidate processor group and the load amount of the distribution source processor group. Compare.

For example, the communication control processor CC0 reads the load amount of the processor # 0 of level 0 corresponding thereto from the shared register 10 and compares it with the load amount previously read in step d3. Other communication control processor CC
The same applies to 1 and the like.

The network control circuit CC01 reads the load amount of the level 1 processor group # 01 corresponding to the network control circuit CC01 from the shared register 01, and compares it with the load amount previously read in step d3.

As a result of the above comparison in each communication control processor CCj or each network control circuit CCk, the load amount of the candidate processor group for comparing the load amount of the distribution destination candidate processor group with the load amount of the distribution source processor group is determined by the distribution source. If it is smaller than the load amount of the processor group, the process proceeds to step d5. Otherwise, the communication control processor CCj or the network control circuit CCk does not perform load distribution.

(3e) In step d5, load distribution is performed.

That is, the communication control processor CCj, which is determined to perform load distribution in step d4, requests one of the processors belonging to the distribution source level 0 processor group #j connected thereto to distribute the load.

For example, the communication control processor CC0 changes the load of the level 0 group # 0 to another level 0 group # 0.
If it is determined that the communication control processor CC0 is to be distributed to the processor group # 1, the communication control processor CC0 is connected to the processor group # 0.
And selects one of the processors PE0 and PE1 belonging to the group, and requests the load distribution thereto. For example, the processor that has reached the break of the normal program processing earliest is selected. The selected processor sends, to the network, a message in which the number of the distribution destination processor group is added as the destination to the program waiting to be executed as a load, and more specifically, the communication connected to the selected processor. Transfer to control processor CC0.

Similarly, the network control circuit CCk determined to carry out load distribution in step d4, for example, C
C01 selects one of the two communication control processors CCj connected to it, for example, CC0 and CC1, and requests it to distribute the load. For example, of the communication control processors CC0 and CC1, the one with the larger load of the corresponding processor group of level 0, for example, CC0 is selected as the distribution source level 0 group. The selected communication control processor, for example, CC0 selects one of the processors belonging to the processor group # 0 of the distribution source and requests the processor to distribute the load. For example, the processor that has reached the break in the normal program processing earliest, for example, PE0
Select

The selected processor is connected to the network, more specifically, a message in which the program waiting to be executed as the load and the distribution destination processor group number added as the destination is connected to the network, more specifically, to the selected processor. Communication control processor CCi, for example, CC0
Transfer to

As described above, the dynamic load distribution control c
The process of No. 3 ends.

(4) Returning to FIG. 2, in the next step c4, the program to be distributed thus transmitted is transferred to one of the processors belonging to the distribution destination group. This transfer process is executed by a communication control processor and a network control circuit that relay this program.

For example, as described above, it is determined that the communication control processor CC0 distributes the load of the level 0 group # 0 to another level 0 group # 1,
When E0 is selected as the distribution source processor, and when a message in which a program to be distributed is added with the destination processor group number # 1 as a destination is transmitted from this processor to the communication control processor CC0, the transferred distribution is performed. The program to be executed is the communication control processor CC
0 relays via the network control circuit CC01,
The data is transferred to the communication control processor CC1 corresponding to the distribution destination processor group # 1.

The communication control processor CC1 selects one of the processors PE2 and PE3 belonging to the processor group # 1 connected thereto, and transfers this message to this. At this time, if a processor with the smallest load amount is selected from these two processors, useless distribution is further reduced.

Similarly, as described above, it is determined that the network control circuit, for example, CC01 distributes the load of level # 1 group # 01 to another level # 1 group # 23, and PE0 is selected as the distribution source processor. In this case, the message including the program to be distributed transmitted from the PE0 is transmitted to the distribution destination processor group # 23 via the communication control processor CC0 and the network control circuits CC01 and CC03 connected to the distribution source processor PE0. Corresponding network control circuit CC23
Is forwarded to The control circuit CC23 includes a group 23
, One of the two level 0 groups # 2 and # 3. For example, a group with a small load amount, for example, # 2 is selected, and this message is transferred to the communication control processor corresponding to the selected group, for example, CC02. The communication control processor CC02 selects one of the two processors PE4 and PE5 connected thereto as a distribution destination processor, and transfers this message to it. For example, one of these with a small load amount is selected.

The processor to which the load has been distributed as described above adds the load to the execution waiting load pool.

As is clear from the above description, in this embodiment, in the distribution of loads among the processor groups divided into a plurality of layers, it is determined whether or not to distribute the load for each layer. At this time, the load of each group of each layer is held in the network, and the communication control processor or the network control circuit provided in the network refers to the load and controls whether or not to perform load distribution.

As a result, it is not necessary to inquire each processor about the load amount, and it is not necessary to wait for the completion of the processing of the inquired processor at the time of inquiry. As a result, the necessity of load distribution can be determined at high speed.

[0074]

According to the present invention, the amount of load on each processor group required for the load distribution control can be referenced at high speed in the network, so that the dynamic load distribution control can be speeded up.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an embodiment of a parallel computer system according to the present invention.

FIG. 2 is a flowchart of process control by each processor, each communication control processor, and each network transfer control circuit in the system of FIG. 1;

FIG. 3 is a detailed flowchart of the dynamic load distribution control during the processing shown in FIG. 2;

[Explanation of symbols]

1 to 4... Shared memory, 11 to 13 .. processor waiting load amount storage shared register in processor group, 20 to 27 .. processor execution waiting load amount storage area.

 ──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Tokuyasu Imon 1-280 Higashi Koikebo, Kokubunji-shi, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd. (56) References JP-A-2-208731 (JP, A) -233873 (JP, A) JP-A-61-296465 (JP, A) IEICE Technical Report, 91-130! , P. 41-47 AFIPS CONF PROC NATL COMPUT CONF, 48, p. 613-622

Claims (1)

(57) [Claims] A plurality of lower processor groups each including a plurality of processors connected to a shared memory; and a network interconnecting the plurality of processor groups. Each of the networks corresponds to one processor group. A plurality of communication control processors for relaying the transfer of messages related to the processors belonging to the corresponding processor group; and a plurality of upper processor groups each including a part of each of the plurality of processor groups. A plurality of lower-level network nodes provided corresponding to
One commonly connected to a group of communication control processors connected to some of the plurality of lower processor groups belonging to the corresponding one upper processor group, and at least one interconnecting the plurality of lower network nodes. A plurality of first-type load registers provided corresponding to each of a plurality of lower-order processor groups included in the corresponding upper-order processor group; A first type network control circuit for relaying a message to and from a group of communication control processors connected to the plurality of lower processor groups; and the upper network node comprises a plurality of lower processor groups. A plurality of second-type load registers provided corresponding to each of And a second-type network circuit for relaying a message between the plurality of lower-level network nodes, wherein each processor has a load storage area defined in correspondence with the processor in a shared memory connected thereto. Each communication control processor is programmed to store the load of the processor each time the processor completes the subprogram processing. Each time the communication control processor completes the message relay program processing, the corresponding lower processor From a plurality of load storage areas in the shared memory provided for the group, read the load of each processor belonging to the processor group,
The load of the processor group is calculated by adding the read loads, and the calculated load is calculated in the lower network node connected to the communication control processor.
Means for writing to a type 1 load register which is a shared register provided corresponding to the lower processor group, and a load held in the type 1 load register provided corresponding to the corresponding lower processor group; Means for comparing a distribution threshold provided for the lower processor group with the upper processor to which the corresponding lower processor group belongs when the load held in the first type load register is equal to or greater than the distribution threshold Means for selecting another lower processor group belonging to the group and having the smallest load value, and a first type load register provided corresponding to the selected other lower processor group. And read the load of the corresponding lower processor group based on the read load. Means for determining whether a part of the load is to be distributed to the other lower processor group, and the load value held in the first type load register corresponding to the other processor group is determined by the lower processor. Means for distributing a load from the lower processor group to the other lower processor group when the load value of the first type load register corresponding to the group is smaller than the first type load register; Each time the first type network control circuit completes the message relay program processing, the network control circuit responds to the lower level network node from the plurality of first type load registers provided in the lower level network node. Read the load of each lower processor group belonging to the upper processor group The load of the corresponding upper processor group is calculated by adding the read loads of the above, and the calculated load is shared in the upper network node corresponding to the corresponding upper processor group. Means for writing to a second type load register, which is a register, a load held in a second type load register provided corresponding to the corresponding upper processor group, and a distribution threshold provided corresponding to the higher processor group And a means for comparing the same load with the smallest load value other than the corresponding upper processor group when the load held in the second type load register is equal to or more than the distribution threshold.
Means for selecting another upper processor group of the bell and the load held in the second type load register provided corresponding to the selected other upper processor group; Means for determining whether or not to distribute a part of the load of the corresponding upper processor group to the other upper processor group based on the performed load, and When the load value held in the second type load register is smaller than the load value of the second type load register corresponding to the upper processor group, the load is distributed from the upper processor group to the other upper processor group. Wherein the load is a sub-processor that is waiting to be executed by a processor belonging to a processor group. A parallel computer system characterized by the total number of grams.