JP2580525B2 - Load balancing method for parallel computers - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a plurality of processing systems.
The present invention relates to a dynamic load distribution method for a parallel computer composed of elements, and more particularly to a load distribution method suitable for uniform distribution of load in a parallel computer having a hierarchical structure in which clusters composed of main memory shared processors are connected by a network. .

[0002]

2. Description of the Related Art With respect to a dramatic improvement in computer performance, a parallel computer that allows a large number of processing elements to operate in parallel has been regarded as promising. Hereinafter, the processing element is abbreviated as PE. In a parallel computer, it is important to distribute a load process uniformly to a plurality of PEs. The performance is the highest when all the PEs share the processing, and the performance decreases as the load unevenness increases. In parallel computers, parallelization overhead, that is, reduction of communication processing accompanying distribution of processes among PEs is also an important issue. The load value (the number of processes waiting to be executed) is frequently exchanged between PEs,
If the process is always distributed to the PE with less load, each PE
However, the communication overhead is high and the overall execution efficiency is rather reduced. Therefore, there is a demand for a load distribution method that achieves load equalization while reducing the amount of communication between PEs.

In the technique disclosed in Japanese Patent Application Laid-Open No. Hei 2-208731, a PE that is trying to distribute a process checks the load value of a PE selected at random through a network, and only when the load value of a partner PE is small, does the load distribution. It is supposed to do it.

[0004]

The above prior art is effective when all the PEs have a communication path to the network. However, by adopting a cluster configuration sharing the main memory, the PEs are connected to the network. If they are not directly connected, the load values of other PEs cannot be easily checked, and efficient load distribution cannot be performed.

In recent years, with the development of highly integrated LSI technology, systems employing a tightly coupled multiprocessor (TCMP) sharing main memory have been increasing. Since the PEs can share the same address space in the TCMP, efficient processing is possible. Since the number of PEs that can be connected to TCMP is limited to about 10 due to the throughput of the shared bus, in order to configure a larger-scale system, it is necessary to further connect a TCMP cluster via a network and adopt a hierarchical system configuration. There is. At that time, in order to reduce the hardware scale of the network, a communication path from all the PEs to the network is not usually provided, and a system in which one communication processor is provided in a cluster is generally used. Only the communication processor is connected to the network and the other PEs
Has no communication path to the network. P in the cluster
In order for E to communicate, it must request communication to the communication processor via TCMP.

[0006] The conventionally proposed dynamic load balancing is based on all P
Assuming an architecture in which E is connected to a network, the following procedure is performed. First, PEs to be distribution candidates are randomly selected, and the load values of the distribution candidate PEs are read. At this time, the load value of each PE is stored in a dedicated register in the network. Thus, the load value can be quickly read. Thereafter, the load value of the distribution candidate PE is compared with the load value of the own PE. If the load value of the distribution candidate PE is smaller than the load value of the own PE, the load is distributed. If the load value of the distribution candidate PE is larger than the load value of the own PE, the load is not distributed. In this method, it is only necessary to check the load value of the PE that is a distribution candidate, and therefore, it is possible to perform load distribution without using global information such as load information of the entire system. This load distribution method is called a smart random method.

[0007] When a dynamic load distribution is to be performed between clusters in a cluster configuration by a smart random method,
The following problems arise. When the PE of each cluster attempts to distribute the load to another cluster and reads the load value of the distribution candidate cluster, the PE is not directly connected to the network, so the read of the load value to the communication processor via TCMP. Is requested. Further, after the reading of the load value from the network is completed, it is necessary to transfer the read load value from the communication processor to the requesting PE. Further, when the communication processor is performing other processing, it is not always possible to immediately respond to the load value read request from the PE. For this reason, there is a risk that the waiting time until the PEs in the cluster obtain the load value becomes extremely long. An increase in the waiting time for reading the load value not only causes a decrease in the operation rate of the PEs, but also delays the load distribution itself, which is a major obstacle to effective dynamic load distribution.

An object of the present invention is to provide a dynamic load distribution method capable of reducing a waiting time when a PE in a cluster performs load distribution in a parallel system having a cluster configuration.

[0009]

[Means for Solving the Problems] In order to solve the above-mentioned purpose
And one or more processing elements for communication
A processor having a main memory shared by the processors;
Cluster as a component, and multiple clusters
Communication of each cluster on parallel computers connected by
Processor periodically distributes the load of the cluster
Randomly select a candidate cluster to be distributed
The raster number is stored in the shared memory, and the distribution candidate cluster is stored.
Compare the load value of the
When the load value of the raster is smaller than the load value of the local cluster
Is a flag indicating whether the load can be distributed.
The lag is stored in the shared memory, and the negative
Negative if the load value is greater than or equal to the load value of the own cluster
A flag indicating whether or not the load is distributed is stored in the shared memory, and
Processing elements in a cluster
When performing decentralization, the processing element
The stored flag is read from the shared memory, and the read
If the flag that has appeared allows load distribution,
Distribute load to distribution candidate clusters with stored numbers
However, if the flag indicates that the load distribution is not
Distribute the load to the raster.

Further, each PE is provided with means for reading the distribution candidate cluster number determined by the communication processor and the flag indicating whether or not distribution is possible. When the PEs in the cluster attempt to distribute the load, if the flag indicating whether the load distribution is possible is permitted, the load is distributed to the distribution candidate cluster determined by the communication processor, and if not, the process of the own cluster is performed. Connect the load to the pool.

[0011]

According to the present invention, when a processor in a cluster performs load distribution, a communication processor determines in advance a cluster that is a load distribution candidate and checks whether or not load distribution is possible. By simply reading the number of the distribution candidate cluster and the flag indicating whether or not the load can be distributed, the load can be quickly distributed.

FIG. 1 is a block diagram of a parallel system according to the present invention. In the figure, 110 and 120 are PEs, and 150 is a communication processor. The communication processor has a distribution candidate cluster number 157 determined at random. Further, it has a distribution availability flag 156 as a result of comparing the load value 154 of the distribution candidate cluster read by the load value reading mechanism 153 with the load value 161 of the own cluster. The set of the distribution candidate cluster number 157 and the distribution availability flag 156 is referred to as load distribution information 170. CPU 110, 120 of each PE
Creates a process, the process management mechanism 11
2, 122 examines the load balancing information 170 and, if the load distribution is possible, distributes the process to the distribution candidate clusters.
If load distribution is not possible, add the process to the process pool of the own cluster. As a result, when each processor attempts to perform the load distribution, the load can be immediately distributed by merely reading the load distribution information 170, and the waiting time for the load distribution can be eliminated.

The distribution candidate cluster number 157 is periodically updated by a timer interrupt 151. Therefore, the load value 154 of the distribution candidate cluster and the distribution availability flag 156 are also updated at intervals of the timer. Therefore, if the interval between the timer interrupts is set sufficiently shorter than the average execution time of the process, the latest load distribution information can be always reflected, and accurate load distribution can be performed.

[0014]

1 to 4 show one embodiment of the present invention. FIG.
FIG. 2 is a configuration diagram of a parallel computer having a dynamic load distribution mechanism composed of n clusters × mPE. FIG. 2 shows each P in the cluster.
It is a flowchart of the operation of E. 3 and 4 show operation flows of the communication processors in the cluster. FIG. 3 shows a process distribution process, and FIG. 4 shows a load distribution information update process.

In FIG. 1, 100 and 200 are clusters,
500 is a network. Only the inside of the cluster 0 (100) will be described in detail. Other clusters have exactly the same configuration. Inside the cluster, 110 and 120 are PEs (for calculation), and 150 is a processor for communication. In the PE, CPUs 111 and 121 execute processes,
Reference numerals 112 and 122 denote process management mechanisms. 160 is the load value of the cluster process pool 161 (the number of processes waiting to be executed). Among the communication processors, 151 is a timer interrupt mechanism, 152 is a random number generation mechanism, 1
57 is a distribution candidate cluster number randomly determined by the random number generation mechanism, 153 is a mechanism for reading the load value of the distribution candidate cluster, 154 is the load value of the distribution candidate cluster, and 155 is the load value of the own cluster and the distribution candidate. A comparator 156 for comparing the load values of the clusters is a flag indicating whether or not the load distribution determined as a result of the comparison can be performed. The distribution candidate cluster number 157 and the distribution availability flag 156 are collectively referred to as load distribution information 170. In the network, reference numeral 502 denotes a router for communicating processes and the like, and reference numeral 501 denotes a load value register for storing a load value of each cluster.

The present invention is characterized in that the communication processor has a function of periodically updating the load balancing information 170 and a function of reading the load balancing information by each PE in the cluster.

First, the configuration of the entire system will be described. In the system, the PE (11) that executes the program
0, 120) gather to form the cluster 100. The clusters 100 and 200 are connected by a network 500 to form the entire system. The PEs in the cluster are a tightly coupled multiprocessor system sharing the main memory. Each cluster has a communication processor 150, and only communication processors are connected to the network from the cluster. Since the other PEs do not have a communication path to the network, the PEs must communicate via the communication processor 150 when communicating.

Next, a process distribution method (load distribution method) will be described. Since the PEs in the cluster share the main storage, it is possible to share the process pool 160. Therefore, the load distribution among the PEs in the cluster can be automatically performed by managing the processes in the shared process pool 160. On the other hand, clusters are connected by a network and cannot have a shared process pool. Therefore, in order to distribute the load between clusters, it is necessary to explicitly divide the process into other clusters. The present invention is a method for efficiently distributing the load between the clusters. Specifically, the communication processor determines in advance which load should be distributed between clusters and to which cluster the load should be distributed, so that each PE in the cluster can perform load distribution. Enable to reduce waiting time. Hereinafter, the detailed operation of each component unit for realizing the above functions will be described.

FIG. 2 shows an operation flow of the PE. The PE first checks whether there is a process in the process pool 160 (step 10). If there is a process, the process is taken out of the process pool (step 11),
After updating the load value (161) (step 12), the process is executed (step 13). When the execution of the process is interrupted, it is checked whether a child process has been created (step 14). As a result, if a child process has been created, load distribution processing is required (steps 15 to 15).
19). In the load distribution processing, first, the load distribution information 170 determined by the communication processor is read (step 1).
5). Since the load balancing information is read only by accessing the shared memory, the load balancing information can be read with little waiting time. As a result of reading the load balancing information, the distribution availability flag 156 is compared (step 1).
6). When the distribution enable / disable flag is ON, that is, when the load distribution is possible, the communication processor is requested to distribute the process to another cluster (step 1).
7). On the other hand, when the distribution enable / disable flag is OFF, the process is not distributed to another cluster, the process is written back to the process pool of the own cluster (step 18), and the load value is updated (step 19). ).
Thereafter, the interrupted process is continued or a new process is taken out (step 20). Through the above processing, each PE can quickly distribute the generated process by reading the load distribution information 170 determined in advance by the communication processor.

Next, the operation of the communication processor will be described. FIG. 3 shows an operation flow of the inter-cluster process distribution processing unit 158 of the communication processor. If a process has arrived from the network (step 30), the process is connected to the process pool of the own cluster (step 32), and then the load value is updated (step 3).
3). On the other hand, if there is a process distribution process from the PE in the own cluster (step 31), the process is sent to the router 502 of the network (step 34).

In addition to the above, the communication processor updates the load distribution information 170 used by the PE. The process of updating the load balancing information is started periodically by the timer interrupt 151. FIG. 4 shows the operation of the communication processor 150 when the timer is interrupted. When the timer interrupt is received, first, the load value 161 of the own cluster is stored in the load value register 15 on the network.
1 is written (step 40). As a result, an accurate load value can be notified to other clusters. Thereafter, a cluster is randomly selected using the random number generation mechanism 152, and a distribution candidate cluster number 157 is determined (step 41). The load value of the distribution candidate cluster is stored in the load value register 50 in the network by using the load value reading mechanism 153.
1 is read out (step 42). Since the load value of each cluster is managed on the network, the load value can be read at high speed. The comparator 155 compares the load value 154 of the distribution candidate cluster with the load value 161 of the own cluster (step 44). If the load value of the distribution candidate cluster is smaller than the load value of its own cluster, the distribution possibility flag is turned on (step 45), so that the PEs in the cluster can distribute the process to the distribution candidate cluster. When the load value of the distribution candidate cluster is larger than the load value of the own cluster (including when the load value is equal), the distribution permission flag is turned off (step 46), and the process is not distributed to another cluster. By determining the load distribution information by the method described above, when the PEs in the cluster attempt to distribute the load, the behavior can be immediately determined. here,
In order to always obtain new load balancing information and avoid concentration of load on a specific cluster when the PE performs load balancing, the frequency of timer interrupts for updating the load balancing information is the average of the process. Must be much smaller than the execution time.

As described above, the communication processor 150 always supplies new load distribution information to the PEs in the cluster, and the PEs in the cluster can quickly perform dynamic load distribution using the load distribution information. .

[0023]

According to the present invention, in a parallel computer having a hierarchical configuration in which a cluster composed of TCMP PEs is connected to a network via a dedicated communication processor, the PEs in the clusters not directly connected to the network are load balanced. The waiting time when performing is significantly reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram of a parallel computer having a dynamic load balancing method according to the present invention.

FIG. 2 is a flowchart of an operation of a processing element (PE) in the cluster of FIG. 1;

FIG. 3 is a flowchart when the communication processor of FIG. 1 distributes a process.

FIG. 4 is a flowchart when the communication processor of FIG. 1 updates load information.

[Explanation of symbols]

100 to 200: cluster, 500: network, 1
10 to 120: Processing element (PE), 1
50: communication processor, 160: process pool,
161: cluster load value (number of processes waiting to be executed), 1
11 to 121 CPU, 112 to 122 process management mechanism, 151 timer interrupt mechanism, 152 random number generation mechanism, 153 load value read mechanism, 154 load value of distribution candidate cluster, 155 comparator, 156 ... Load distribution availability flag, 157... Load distribution candidate cluster number, 158
... Inter-cluster process distribution mechanism, 501, load value register, 502, router.

Claims (1)

(57) [Claims] 1. A parallel computer in which one or more processing elements, a communication processor, and a cluster having a main memory shared by them are included as components, and a plurality of clusters are connected by a network. The communication processor periodically randomly selects a candidate cluster to distribute the load of the cluster , stores the number of the distribution candidate cluster in the shared memory, and stores the load value of the distribution candidate cluster and the load value of its own cluster. When the load value of the distribution candidate cluster is smaller than the load value of its own cluster, as a flag indicating whether or not load distribution is possible, a flag indicating that distribution is possible is stored in the shared memory ,
When the load value of the distribution candidate cluster is greater than or equal to the load value of the own cluster, a flag indicating whether or not load distribution is performed is set to the shared value.
Stored in the organic memory, processing elements in the cluster, load
When performing a distributed, the processing element is the
Reading the stored flag from the shared memory;
If the read flag allows load distribution, the load is distributed to the distribution candidate clusters having the stored numbers, and if the flag indicates that load distribution is not performed, the load is distributed to the own cluster. A load balancing method for a parallel computer characterized by the following.