JPH01191252A - Load distributed control system - Google Patents

Abstract

PURPOSE:To considerably reduce the overhead by comparing loads of all processing elements with only adjacent processing elements and performing local load distribution where the load of a processing element is moved to adjacent processing elements only when this load is two or more larger than loads of adjacent processing elements. CONSTITUTION:A distribution direction determining means 5 compares the value indicated by a load quantity display means 4 in a corresponding processing element with values of load quantity display means 4 in adjacent processing elements; and only when the former is two or more larger than a minimum value of values indicated by load quantity display means 4 in all adjacent processing elements, a load sending/receiving means 6 is instructed to distribute one load to the adjacent processing element in the direction indicating the minimum value. The load sending/receiving means 6 sends one load stored in a buffer 3 or receives the sent load to store it in the buffer 3 for the purpose of distributing the load to the adjacent processing element in the direction designated by a distribution direction determining means 5. Thus, loads are distributed in parallel, and they are quickly distributed.

Description

DETAILED DESCRIPTION OF THE INVENTION (1) Technical field to which the invention pertains The present invention relates to a load distribution control method in a network-coupled parallel processor system.

(2) Conventional technology Supported by the development of LSI technology, a large number of processing elements have recently been connected in the form of networks (processing elements are interconnected by links, such as rings, arrays, and trees).

Research and development of parallel processor systems coupled to hypercubes (there are various coupling forms such as hypercubes) is becoming more active. Applications of such parallel processor systems include applications in which each processing element generates new work in the process of processing the assigned work, and the work can be processed by any processing element. In this type of application, there are many fields (often seen in AI processing such as natural language processing and knowledge processing). A major challenge is how to distribute the load so that processing elements do not go into a dormant state. This control is generally called load distribution control.

Conventional load distribution control in this type of system.

A load distribution control device is provided independently of the processing elements, and this device monitors the load status of all processing elements, and if the load on any processing element becomes heavy, the load is reduced (inactive state waiting for load). This was achieved by distributing it to processing elements (including Here, specific load distribution methods include a method in which the control device actually collects the load from processing elements with a heavy load and distributes it to processing elements with a light load;
There is a method in which the control device notifies a processing element with a heavy load of a processing element number with a light load, and when actually moving the load, the processing elements in between are relayed (using links between processing elements). .

As described above, in conventional load distribution control, a dedicated device centrally monitors the load status, collects and distributes the load, and gives instructions for distribution, so these operations are performed by each processing element. However, when the number of processing elements in the system increases, this load distribution control becomes a large overhead and system performance proportional to the number of processing elements cannot be obtained.

(3) Purpose of the Invention The purpose of the present invention is to provide a control method that solves one or more drawbacks and can achieve quick load distribution even in a system with a large number of processing elements.

(4) Structure of the invention (4-1) Features of the invention and differences from the conventional technology The present invention provides a load monitoring 1 distributed control mechanism in each processing element,
Each processing element uses this mechanism to perform local load distribution in parallel, thereby effectively realizing load distribution for the entire system.

Specifically, each processing element has the minimum load only when the internal load is two or more greater than the minimum load of all processing elements physically connected to the processing element by a link. Distribute one load to the other. That is, one of the main features of the present invention is to control load distribution locally, in parallel, quickly, and simply. This method differs from the conventional technology in the manner in which the mechanism for load distribution control is installed, the manner in which the load status is monitored B, and the specific manner in which the load is distributed.

(4-2) Embodiment FIG. 1 shows an embodiment in which the present invention is applied to a ring-coupled parallel processor system.
Only one processing element is shown.

In the figure, 1 is a processing element, 2 is a processor that processes a load or generates a load, 3 is a buffer that accumulates loads waiting to be processed (loads that cannot be processed immediately after being generated by the processor 2 are stored here), Reference numeral 4 indicates a load amount display means for constantly counting and displaying the number of loads accumulated in the buffer 3, and 5 indicates a value indicated by the load amount display means 4 within its own processing element and is physically coupled to the processing element by a link. All processing elements (
The value of the load amount display means 4 in the adjacent processing element (hereinafter referred to as an adjacent processing element) is compared, and the value indicated by the load amount display means 4 within itself is equal to the minimum value of the load amount display means 4 in all adjacent processing elements. This is a distribution direction determining means that instructs the load transmitting/receiving means 6 to distribute one load to adjacent processing elements in the indicated direction. The load transmitting/receiving means 6 transmits one load accumulated in the buffer 3 or receives the transmitted load and transfers the load to the buffer 3 in order to distribute the load to the adjacent processing elements in the direction specified by the distribution direction determining means 5. to be accommodated.

FIG. 2 is an explanatory diagram of the operation of the present invention, and the numerical value within the processing element indicated by a square represents the number of loads of that processing element (value indicated by the load amount display means 4).

PE in the figure corresponds to the processing element 1 described above.

The present invention will be described in detail below with reference to the two figures.

In the present invention, load distribution control may be performed asynchronously on all processing elements, but here, to simplify the explanation, it is assumed that all processing elements are performed simultaneously and synchronously at a constant cycle. Assume that at time t0, the load on each processing element is in the state shown in FIG. 2 i). At this time, the dispersion direction determining means 5 of the processing element PEI.

The self load is "5" and the minimum value of the load of the adjacent processing element "
O' (the load on the processing element PE) by 2 or more, the load transmitting/receiving means 6 is instructed to distribute the load by 1 to the processing element PE. Similarly, processing elements PE, ,PE
The dispersion direction determining means 5 of 8 are respectively processing elements PE, , P
Instructs to distribute the load to E7. In contrast, the processing element PEG.

Since the white meat load of PEx, PEq, PEs, PEb, PEv, and PEq is not greater than the load of the adjacent processing element by two or more, the respective distribution direction determining means 5 does not instruct the load transmitting/receiving means 6 to distribute the load.

Therefore, 1 time 1. In this case, the load state is as shown in FIG. 2 (11). Next, in this state, processing element PE1. P.E.
, are respectively processing elements PE2. The PE is instructed to distribute the load to the PE. After this load distribution, the load state is as shown in iii) in the same figure, and in this case, from processing element PE4 to processing element P
Load distribution to E3 is performed.

Finally, the load state is approximately averaged as shown in iv) of the same figure.

As explained above, in the tree method, all processing elements perform local load distribution among their adjacent processing elements in parallel, so the overall system load is lower than in the conventional centralized, sequential control method. Distributed processing becomes much faster. Furthermore, this effect becomes larger as the number of processing elements in the system increases. In the above explanation, a ring-coupled parallel processor system was used as an example for the sake of simplicity, but it goes without saying that the present invention can also be applied to other network-type parallel processors such as arrays, hypercubes, trees, etc. . Also, in FIG. 1, all means are expressed as if they were realized by hardware.

Even if these functions/procedures are programmed and the processor 2 is made to execute them, the effects of the present invention are not diminished. Further, here, it is assumed that specific load movement and notification of load values to adjacent processing elements utilize links between processing elements, and no dedicated signal lines are provided.

FIG. 3 shows another embodiment of the present invention. 1. 4゜5
is the same as in FIG. 1, 7 is a small load detection means for detecting that the value indicated by the load amount display means 4 is less than a predetermined value (A), 8
Similarly, the large load detection means detects that the value indicated by the means is greater than or equal to the predetermined value "(A)+maximum distance of the network 1(L)+14".

Here, the maximum distance (L) of the network is the maximum value of the distance between any two processing elements (the number of processing elements therebetween+1) in a network type parallel processor system. For example, when the number of processing elements is N, in a ring type, when N is an even number, L=N/2. When the number is odd, L= (N
-4)/2, L=5 for the torus-coupled two-dimensional array type, and L-1og,N for the hypercube type.

Also, 9.10 is -1red OR line, 11 is Wire
dOR line 9. Take the logical product of 10 and 1 both are active
The dispersion direction determining means 5 is activated only when
This is the activation means for load distribution control to

In general, in parallel processing systems, no matter how uneven the load on all processing elements is, if the load is large, load balancing has little effect, because the purpose of load balancing is to reduce the number of idle processing elements. This is because if all processing elements have a predetermined load, there is no need for load distribution at that point. The embodiment shown in Fig. 3 focuses on this point, and attempts to perform load distribution only when the load of each processing element becomes less than a predetermined value A (in the case of Fig. 1, the embodiment shown in Fig. 2 Even in state iv), all operations other than actual load movement are performed at regular intervals). Furthermore, in this method, load distribution is not performed unless the load difference is 2 or more, so in the worst case, load distribution is performed when the difference between the maximum load and minimum load of all processing elements becomes less than the maximum distance (L) of the network. For example, in iv) of Figure 2, the difference between the maximum load and minimum load is 2.
is in equilibrium (in this example).

Since L=5, in the worst case, even if this difference is 5, load balancing may not be performed. For example, the loads of processing elements PR, ~PE are 0, 1, 2, 3, and 4.5 degrees, respectively.4, 3. 2.
when it becomes 1). Therefore, in order to implement load distribution, another condition is required, namely, that the difference between the maximum load and the minimum load of all processing elements is equal to or larger than L+1.

The embodiment of FIG. 3 meets these two conditions, viz.

The minimum load on the system is less than or equal to a predetermined value (A).

And only when the maximum load is (L+A+1) or more.

The activation means 11 activates the dispersion direction determining means 5.
It performs load balancing. Specifically, the load distribution control in this embodiment is such that the small load detection means 7 of any of the processing elements among all the processing elements detects a small load of A or less,
And the large load detection means 8 of any processing element is (L+A
When a large load of +1) or more is detected, -1re respectively.
d The OR lines 9 and 10 become active, and the 1-dispersion direction determining means 5 becomes active via the activation means 11. Therefore, in the embodiment shown in FIG. 1, load balancing is carried out only when it is necessary and effective, so that the overhead of load balancing control can be reduced more than in the embodiment shown in FIG.

Note that when attempting to realize the 2-bokuto style using software, the load distribution process is made into an interrupt processing routine, and when the activation means 11 becomes active, an interrupt is generated to the processor 2, and the load distribution process is performed on this. height,
At other times, the processor 2 may be operated to perform only general processing.

(5) As described in detail, according to the present invention, a load monitoring system is provided with a distributed control mechanism for all processing elements constituting a parallel processor system, and all processing elements are connected to their adjacent processing elements. Just compare the size of the load.

Local load balancing is performed by moving the load to adjacent processing elements only when the load on one element is large, so load balancing is performed in parallel for the entire system, which speeds up the process. There are advantages. That is, compared to conventional centralized and sequential load distribution control, this method has the advantage of significantly reducing overhead and achieving high system performance commensurate with the number of processing elements.

[Brief explanation of the drawing]

Fig. 1 shows an example in which the present invention is applied to a ring-coupled parallel processor system, Fig. 2 is an explanatory diagram of the operation of the embodiment shown in Fig. 1, and Fig. 3 shows load distribution control only during necessary and effective periods. This is another embodiment configured to perform the following. ■=Processing element 2: Processor 3: Buffer 4: Load amount display means 5: Distribution direction determining means 6: Load transmitting/receiving means 7: Small load detecting means 8: Large load detecting means 9, 10: Wired OR line 11: Starting means Patent applicant Nippon Telegraph and Telephone Corporation

Claims (2)

[Claims] (1) In a parallel processor system in which a large number of processing elements are connected in a network, load amount display means for constantly displaying the number of loads waiting to be processed in each processing element as a coefficient; The value indicated by the load amount display means is compared with the value indicated by the load amount display means in at least all adjacent processing elements that are physically coupled to the processing element, and the load within the own processing element is compared to any of the adjacent processing elements. A distribution direction determining means for determining whether to distribute to a processing element and a load transmitting/receiving means for transmitting the load within the own processing element to an adjacent processing element or receiving the load from them are provided, and each processing element is provided with a load amount within the processing element. Only when the value indicated by the display means is larger than the minimum value of the load amount display means in all adjacent processing elements by a predetermined value or more, one load is applied to the adjacent processing element in the direction showing the minimum value. A load distribution control method characterized by distributing. (2) Each processing element in claim 1 is provided with a small load detection means for detecting that the value of the internal load amount display means has become less than a predetermined value (A) and a predetermined value "(A) + network A large load detection means is provided to detect when the load of any one of all the processing elements becomes equal to or less than the predetermined value (A), and when the load of any one of all the processing elements becomes equal to or less than the predetermined value (A), and all the processing elements Only when the load of any of the processing elements becomes (A+L+1) or more,
A load distribution control method characterized by performing the control according to claim 1.