JPH09274608A - Inter-processor load distribution control method for multiprocessor system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a load distribution control method with which the throughput reduction of a system is avoided by distributing a local CPU load among plural processors in optimum manner. SOLUTION: A processor managing means 11 resident at each calculation processor 2 reports the entire CPU use rate of the present calculation processor 2 and the CPU use rate of each process to a system managing means 5 operated by a system managing processor 1. The system managing means 5, where the report is received, manages load information inside the respective calculation processors 2 through a CPU use rate managing table 10 and when such a load state exceeds the range of upper limit value, lower limit value and overload value defined by system parameters, while using a processor load managing function 7 and a processor selecting function 8, the process is dynamically moved to any processor where the CPU load is made optimum.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a load distribution control method among processors in a multiprocessor system in which a plurality of processors are coupled via a communication line, and particularly to a process managed by an overloaded processor to another processor. The present invention relates to a load distribution control method among processors in a multiprocessor system in which a processor load is optimally distributed by moving.

[0002]

2. Description of the Related Art A multiprocessor system in which a plurality of (for example, tens to tens of thousands) processors are connected to each other in order to process a large amount of data at high speed is known.
However, in such a multiprocessor system, unless processes are efficiently allocated to a large number of processors connected in parallel to operate, the load is biased among the processors and the multiprocessor system cannot be used efficiently. In a conventional multiprocessor system, a program executer explicitly designates a processor for starting a process, or a plurality of processors are divided into groups, and a program executer executes a process in a processor group. It was realized by specifying.

Further, as a conventional scheduling technique relating to the distribution of processor load, Japanese Patent Laid-Open No. 342057/1992.
Japanese Patent Laid-Open Publication No. 1-214961. In the former, the load of the overloaded processor is taken over by another lightly loaded processor,
The thing to equalize the load of each processor is described. In the latter, the workload and the communication volume are statically estimated, and this estimated workload is the sum of the communication volume and the processing volume for requesting processing. When the size is larger than that, it is described that the load is optimally distributed by dynamically allocating it to a free processor. All of these are techniques related to process scheduling in a tightly coupled multiprocessor system that shares main memory.

[0004]

In the above prior art,
It is necessary to explicitly specify a processor and a processor group for scheduling the process in advance, and if the specification of the processor and the processor group for scheduling the process is biased, the local throughput is lowered in the processor unit. If a local decrease in throughput occurs on a processor-by-processor basis, a synchronization delay occurs with the processes of other processors executing in parallel, and the overall throughput of the parallel computer system decreases. It was

Further, in the one disclosed in Japanese Patent Laid-Open No. 342057/1992, the load of each processor is equalized by substituting the load of a processor having an overload with another processor having a light load. However,
When the loads on the processors are equalized, there is a problem that when a new load with a large memory usage occurs, no processor can execute the load. Further, the above-mentioned Japanese Patent Laid-Open No. 1-214961 relates to scheduling in units of instructions in a tightly coupled multiprocessor system sharing a main memory, and particularly, how to efficiently schedule instruction words on an instruction cache. It also relates to how to control a group of instructions to be read into the instruction cache, and does not consider scheduling the running process to optimize the load distribution among the processors. An object of the present invention is to solve the above problems, to centrally manage the CPU load of each processor that changes momentarily during system operation within the system, and to optimize the distribution of the processor loads that make up the system. An object of the present invention is to provide a load distribution control method between processors in a multiprocessor system capable of securing a processor capable of executing a new load of a large memory usage.

[0006]

According to the present invention, in order to achieve the above object, one system management processor (1) and a plurality of calculation processors (2) are connected via a communication means (network 4). A method for controlling load distribution between processors in a multiprocessor system, wherein the system management processor (1) centrally manages the CPU load of each calculation processor (2) and the upper limit of the CPU usage rate of the calculation processor (2). A value and a lower limit value are set in advance, and a calculation processor in which the CPU usage rate exceeds the upper limit value or is less than the lower limit value continues for a certain period is set as a migration source processor, and is managed by the system management processor (1). Each calculation processor (2)
In the case where one of the processes (15) managed by the migration source processor is selected as a migration process with reference to the CPU load (CPU usage rate management table 10 in FIG. 4) in step S4 and the migration process is accepted. CP
A calculation processor that satisfies the condition that the U usage rate is equal to or less than the upper limit value of the CPU usage rate and is as close as possible to the upper limit value is selected as a migration destination processor, and the selected migration process is transferred from the migration source processor to the migration destination processor. The feature is that they are moved.

In addition, in order to select a process that is easy to move, the selection of the move process is performed in the order of increasing the actual memory amount to be used. Further, in order that the CPU usage rate when the migration process is accepted is equal to or less than the upper limit value of the CPU usage rate and is as close as possible to the upper limit value (this allows a processor with a low CPU usage rate to be left for the subsequent load). The destination processor is selected in descending order of CPU utilization (see flowcharts in FIGS. 5 and 6).

[0008]

BEST MODE FOR CARRYING OUT THE INVENTION According to the present invention, one system management processor that manages the entire system and n calculation processors that respectively execute and manage a plurality of processes are connected via a high-speed network mechanism. , It is premised on a loosely coupled multiprocessor system capable of communicating between all the processors thus connected. The system management processor has a control program (hereinafter referred to as system management means) that operates on the system management processor, and manages the entire system by this. Further, each calculation processor has a control program (hereinafter, referred to as processor management means) that operates on the calculation processor, and thereby manages a plurality of processes that operate on the calculation processor.

A process number that is unique in the system is given to each process running on each computing processor. Each computing processor itself is also given a unique processor number in the system. Which process is executed by which calculation processor is managed by the system management means by a correspondence table between process numbers and processor numbers (hereinafter referred to as process management table). Then, when communicating between the processes, the process management table is searched to obtain the calculation processor in which the process of the transmission destination is executed, and the communication is performed to the processor.

In order to implement the present invention, the processor management means of each calculation processor includes a CPU of the processor.
A "CPU usage rate reporting function" for monitoring the usage rate and notifying the CPU usage rate to the system management means of the system management processor and a "process migration function" for migrating a process to another processor are provided. There is. Further, the system management means of the system management processor includes the CPs.
A "CPU usage rate management table" for managing the usage rate of U, a "processor load management function" for accumulating the CPU usage rate notified from the processor management means of each calculation processor in the CPU usage rate management table, and a CPU A "processor selection function" is provided for selecting the optimum calculation processor for use.

The CPU usage rate is notified by the CPU usage rate reporting function of the processor management means of each calculation processor,
If a computing processor whose CPU usage rate exceeds the upper and lower limits of the CPU usage rate of the system parameter (the system parameter is stored in the storage means in the system management processor) is detected, the processor of the system management means The selection function is used to select another calculation processor that has the optimum CPU usage rate in the calculation processor based on the CPU usage rate and resource usage of each process in the calculation processor, and the process being executed by that calculation processor. Moves itself to continue the process on the destination compute processor. This makes it possible to avoid a decrease in throughput of the parallel computer system due to a local CPU load of the calculation processor.

Next, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a system block diagram of an embodiment of the present invention. In the figure, 1 is a system management processor (1) for managing the entire system, 2 is a calculation processor (a plurality of) for executing and managing a plurality of processes,
These processors can be identified by a processor number 3 uniquely defined in the system (in the example of FIG. 1, the system management processor 1 is given a processor number 0, and the three calculation processors 2 shown in the figure are provided.
The processor numbers 1 to 3 are assigned to the above). Further, these processors are connected via a high-speed network 4, which is a communication means, and all the connected processors can communicate with each other and exchange information. The connection between processors is not limited as long as they are connected by hardware, and the bus type
Either star type is acceptable.

The system management means 5 of the system management processor 1 is provided with a process management function 6, a processor load management function 7 and a processor selection function 8. The process management function 6 is a function for referring to / updating and managing a process management table 9 in which generated process numbers correspond to processor numbers, and this function is used when communication is performed between a plurality of processes. This makes it possible to communicate using only the process number without being aware of the processor in which the process of the communication partner exists.

The processor load management function 7 is a function for storing and managing the CPU usage rate of the calculation processor 2 reported from the processor management means 11 of each calculation processor 2 in the CPU usage rate management table 10. The processor selection function 8 is a function for referring to the CPU usage rate management table 10 to select an optimum calculation processor and responding to the requesting calculation processor.

The processor management means 11 of each calculation processor 2 is provided with a process generation / termination function 12, a process migration function 13, and a CPU usage rate reporting function 14. The process generation / termination function 12 is a function for managing the process information of the own calculation processor. The process moving function 13 is a function for moving the process 15 being executed by the own calculation processor to another calculation processor 2.

The CPU usage rate reporting function 14 is a processor load management function provided in the system management means 5 of the system management processor 1 for calculating the CPU usage rate obtained by sampling each process executed and managed by the calculation processor 2. This is a function for reporting to 7. FIG. 2 is a diagram showing the system parameter 18, which is usually stored in a storage means (such as a magnetic disk) in the system management processor 1 and is referred to by both the system management means 5 and the processor management means 11. . System parameter 1
8 is, for example, as shown in FIG.
It consists of the CPU utilization lower limit value, sampling interval value, overload value, and the like. The system management unit 5, the processor management unit 11, and the above functions are all configured by software (program).

FIG. 3 shows an example of the structure of the process management table 9 used by the system management means 5. As shown in the figure, the process management table 9 is composed of a process number column 21, a processor number column 22 in which the process exists, and a process state (operating state) column 23 indicating the operating state of the process. .

FIG. 4 shows C referred to by the system management means 5.
3 is a diagram showing a configuration example of a PU usage rate management table 10. It manages the CPU usage rate of each calculation processor.
The CPU usage rate management table 10 manages the CPU usage rate of each calculation processor, and the process number column 3
1, a CPU usage rate column 32 for each process, an actual memory usage amount column 33 used in a process, a processor number column 34, a CPU usage rate column 35 for each processor,
It is composed of a record including an overload value column 36 indicating the elapsed time of the overload state in the processor. For a processor having no process, a processor number column 34, a CPU usage rate column 35 for each processor, and an overload value column 36 that indicates the elapsed time of an overload state in the processor.
It consists of records with only

Next, the operation of this embodiment will be described with reference to the drawings. An operation of moving one process managed by a calculation processor to another calculation processor when the CPU usage rate of the calculation processor becomes high will be described with reference to FIG. The CPU usage rate reporting function 14 in the processor management means 11 of each computing processor 2 samples the CPU usage status of each process 15 in the computing processor 2 at the sampling interval value defined by the system parameter 18, and for each process unit. The CPU usage rate and the CPU usage rate of the calculation processor 2 are calculated (for example, the CPU in the case of being used n times out of m times of sampling).
The usage rate is set to (n / m) × 100%) and is reported to the processor load management function 7 in the system management means 5 of the system management processor 1.

The processor load management function 7 stores the contents reported from each computing processor 2 in the CPU usage rate column 32 for each process and the CPU usage rate column 35 for each processor in the CPU usage rate management table 10. At this time,
For those exceeding the “CPU usage rate upper limit value” and below the “CPU usage rate lower limit value” defined in the system parameter 18, the CPU usage rate management table 10
The value in the overload value column 36 is incremented by one. This overload value column 3
The processor whose value of 6 exceeds the value of the "overload value" defined in the system parameter 18 is set as the migration source processor, and this migration source processor is notified and the process and migration destination to be migrated to the processor selection function 8 Request the selection of the calculation processor that becomes.

The processor selection function 8 indicates that the processor in the overloaded state (“processor CPU usage rate <lower limit value of CPU usage rate or processor CPU usage rate> upper limit value of CPU usage rate” is an overload of the system parameter 18). Of the processes managed by the processor that continues to exceed the allowable value range), select the process that uses a small amount of real memory (that is, the process that is easy to move), and
When this selected process is adopted, the destination processor that satisfies the condition of "lower limit of CPU usage rate ≤ processor CPU usage rate ≤ upper limit value of CPU usage rate" and is as close as possible to the upper limit of CPU usage rate is selected. It is a function to do. This function balances the load on the working processor near the upper CPU utilization limit by moving the process that uses less real memory from the overloaded processor to another processor. Be prepared for the next load.

5 and 6 show the processor selection function 8 when the processor load management function 7 requests the selection of the process to be moved and the calculation processor to be the movement destination.
6 is a flowchart illustrating an operation of selecting a migration process and a migration destination calculation processor performed by the above. Hereinafter, FIGS.
The flowcharts of FIGS. 5 and 6 will be described with reference to FIG. The processor in the overload state detected by the processor load management function 7 is set as the migration source processor, and the processor number (processor number 1 in the example of FIG. 7 described later) is input (step 41). The processor selection function 8 is C
Input the PU usage rate management table 10 to processor number 3
4 and the actual memory usage 33 are used as keys to select S, which is one of the sort functions provided in large-scale general-purpose computers.
ORT is performed to create a move source selection queue (step 4)
2). That is, a record having the input processor number in the processor number column 34 is first selected, and SORT is performed in ascending order with the selected record using the actual memory usage column 33 as a key to create a migration source selection queue.
The migration source selection queue created as a result of this is a queue in which the processes managed by the overloaded processor of the migration source are arranged in the order in which the actual memory usage is small (that is, in the order in which they are easy to move).

Next, the processor selection function 8 performs SORT in descending order from the CPU usage rate management table 10 using the CPU usage rate column 35 for each processor as a key to create a destination selection queue (step 43). As a result, in the destination selection queue, the processors are arranged in descending order of CPU usage (that is, as the destination processor, CPU
Select a processor with a high usage rate, balance the CPU usage rate close to the upper limit, and keep the processor with a low CPU usage as a destination processor to prepare for the next load). .

Next, the processor selection function 8 selects the source process and the destination processor using the source selection queue created in step 42 and the destination selection queue created in step 43. This selection procedure will be described in detail below. First, i = 1 is set (step 44). Next, the i-th record is taken out from the transfer source selection queue and temporarily set as the original record (step 45). The queue pointer of the destination selection queue is reset (j = 1) to indicate the head of the destination selection queue (step 46). Next, the j-th record is taken out from the destination selection queue and temporarily set as the destination record (step 47).

Next, it is judged whether or not "the process CPU usage rate of the original record + the processor CPU usage rate of the destination record.ltoreq.the CPU usage rate upper limit value of the system parameter 18" is satisfied (step 48). Is step 54
If not, go to step 49. Step 4
The condition 8 is a condition in which the processor of the preceding record is not overloaded when the process of the original record is moved to the processor of the preceding record.

If the condition of step 48 is not satisfied,
In step 49, j is updated (j = j + 1), and in step 50, it is judged whether or not the updated j exceeds the size of the destination selection queue. If not, the process returns to step 47 to select the destination. The next record (that is, the j-th record) is taken out from the queue, is set as the previous record, and the above-described processing from step 48 is repeated.

When j exceeds the size of the destination selection queue in step 50, the process proceeds to step 51 and i is updated (i = i + 1). Next, it is determined whether or not the updated i exceeds the size of the source selection queue. If not, the process returns to step 45, and the next record (that is, the i-th record) is extracted from the source selection queue. The original record is set, and the processing from step 46 onward is repeated. If i exceeds the size of the migration source selection queue in step 52, it is assumed that there is no migration source record that meets the conditions, and the migration destination processor is not selected (step 53).

If the condition of "process CPU usage rate of original record + processor CPU usage rate of destination record≤CPU usage rate upper limit value of system parameter 18" is satisfied in step 48, the process proceeds to step 54 (FIG. 6). ,
The source record is the source record and the destination record is the destination record. In the next step 55, the source processor, source process, and destination processor are specified. That is, the process number of the migration source record and the processor number in the overloaded state received from the processor load management function 7 identify the migration source process and the migration source processor, and the processor number of the migration destination record identifies the migration destination processor. The migration source process thus identified is set as the process 15 to be migrated, the migration source processor is designated as the migration source calculation processor 2, and the migration destination processor is designated as the migration destination calculation processor 2 and is reported to the process migration function of the calculation processor 2. To do.

FIG. 7 is a conceptual diagram for explaining the operation of the process moving function 13 to move the process being executed by its own processor to another processor. In FIG. 7, the system management processor 1 is described as “processor number 0”, the overloaded calculation processor 2 that is the migration source processor is “processor number 1”, and the calculation processor 2 that is the migration destination processor is “processor number 2”. To do. When the process migration function 13 of the calculation processor 2 having the processor number 1 which is the migration source processor starts the process migration process (process a), it starts the process migration to the system management means 5 of the system management processor 1. Is reported (processing b). System management means 5 that received this report
Sets the process status column 23 (see FIG. 3) of the process to be moved (the process selected in step 55 of FIG. 6) in the process management table 9 by the process management function 6 to “pending state” (process c), The process migration function 13 of the migration source calculation processor 2 having the processor number 1 is informed of the completion of the process transition to the pending state (process d).

In response to this response, the process migration function 13 of the migration source calculation processor 2 having the processor number 1
Processes the process 15 to be moved to the quiescent state (processing e), and then sends the process information 17 to the process migration function 13 of the migration destination calculation processor 2 having the processor number 2.
The real memory area information currently used by the moving process 15 is transferred (process f).

The process migration function 13 of the migration-destination computing processor 2 having the processor number 2 restores the transferred process information 17 and the real memory area information of the process 15 to be migrated to the state before the migration (process g). . Here, “restore to the state before the movement” means that the real memory area of the transferred process 15 and the process information 1 that is also transferred.
7 to restore the relationship between the virtual space and the real storage device, and replace the information unique to each calculation processor stored in the process information 17 with the information of the migration destination calculation processor,
It includes operations that enable it to operate as a process. The process migration function 13 of the migration destination calculation processor 2 then releases the process quiescent state (processing h) and then reports the completion of the process migration to the system management means 5 of the system management processor 1. Yes (process i).

Upon receiving the report of the completion of the process migration, the system management means 5 causes the process management function 6 to release the process status column 23 of the process management table 9 from the "pending state" (process j), and the processor number column. The processor number 22 is changed to the processor number 2 which is the move destination (process k), and the completion of the release of the process pending state is returned to the process move function 13 of the move destination calculation processor (processor number 2) ( Treatment l).

In response to this reply, the process migration function 13 of the migration destination computing processor 2 having the processor number 2
Reports the completion of the process migration to the process migration function 13 of the migration source computer having the processor number 1 (process m). In response to this report, the process migration function 13 of the migration source computer having the processor number 1 erases the process information 17 and the like (process n), and releases the real memory used by the migrated process 15 ( Process o). However, if the request cannot be accepted for each request to each processor due to some reason, the process movement is stopped and the subsequent processing is continued on the processor.

According to the present embodiment, by moving the process of the processor having a high CPU load to the processor having a low CPU load, it is possible to avoid the decrease in the throughput of the parallel computer system due to the local CPU load. Also, at that time, select a process that uses as little real memory as possible, and set the destination processor as close to the upper limit of CPU usage as possible without overloading when the process is accepted. Since a processor with a high CPU usage rate is balanced near the upper limit of the CPU usage rate, a processor with a low CPU usage rate should not be the destination processor as much as possible. Thus, the low load state can be maintained to prepare for the subsequent load. Although the loosely coupled multiprocessor system (parallel computer system) has been described as an example in the above embodiment, the present technology can be applied to a tightly coupled multiprocessor system.

[0035]

According to the present invention, even if the processor load is biased during the operation of the multiprocessor system and the system efficiency is reduced, the operator load can be optimally distributed without the intervention of the operator. System throughput can be improved.

[Brief description of drawings]

FIG. 1 shows a block diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing system parameters in the present invention.

FIG. 3 is a diagram showing an example of configuration information of a process management table 9 of this embodiment.

FIG. 4 is a diagram showing an example of configuration information of a CPU usage rate management table 10 of each calculation processor of the present embodiment.

FIG. 5 is a flow chart (No. 1) for explaining the operation of the processor selection function 8 of the present embodiment.

FIG. 6 is a flowchart (part 2) for explaining the operation of the processor selection function 8 of the present embodiment.

FIG. 7 is a diagram for explaining the operation of the process migration function 13 of this embodiment for migrating a process being executed by its own processor to another processor.

[Explanation of symbols]

1: system management processor, 2: calculation processor,
3: processor number, 5: system management means, 6: process management function, 7: processor load management function, 8: processor selection function, 9: process management table, 10: C
PU usage rate management table, 11: processor management means,
12: Process creation / termination function, 13: Process migration function, 14: CPU usage rate reporting function, 15: Process, 1
6: process number, 17: process information, 18: system parameter, 21: process number, 22: processor number, 23: process state, 31: process number, 3
2: CPU usage rate per process, 33: Actual memory usage per process, 34: Processor number, 35: CPU usage rate per processor, 36: Overload value

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Kazuya Higuchi, Kazuya Higuchi, 5030 Totsuka-cho, Totsuka-ku, Yokohama, Kanagawa Prefecture Software Development Division, Hitachi, Ltd. (72) Hiroyuki Sakuraba, Totsuka-cho, Totsuka-ku, Yokohama, Kanagawa 5030 Incorporated company Hitachi Ltd. Software Development Division (72) Inventor Ho Sato 6-81 Onoue-cho, Naka-ku, Yokohama-shi Kanagawa Hitachi Software Engineering Co., Ltd. (72) Inventor Kenichi Takahashi Onoue, Naka-ku, Yokohama-shi, Kanagawa 6-81, Machi Hitachi Software Engineering Co., Ltd. In-house (72) Inventor Toshiro Hashiya 5030 Totsuka-cho, Totsuka-ku, Yokohama-shi, Kanagawa Within the Hitachi, Ltd. Software Development Division

Claims (3)

[Claims] 1. A load distribution control method between processors in a multiprocessor system in which one system management processor and a plurality of calculation processors are connected via communication means, wherein the CPU load in each calculation processor is controlled by the system management processor. In addition to centrally managing it, the upper and lower limits of the CPU usage rate of the calculation processor are preset, and the state where the CPU usage rate exceeds the upper limit value or is less than the lower limit value continues for a certain period. The calculation processor is used as a migration source processor, one of the processes managed by the migration source processor is selected as a migration process by referring to the CPU load of each computation processor managed by the system management processor, and CPU utilization when accepting move process A computing processor that satisfies the condition of being less than or equal to the upper limit value of the CPU usage rate and as close as possible to the upper limit value is selected as a migration destination processor, and the selected migration process is migrated from the migration source processor to the migration destination processor. A load distribution control method between processors in a multiprocessor system characterized by the above. 2. The load distribution control method between processors in a multiprocessor system according to claim 1, wherein the selection of the migration process is performed in an ascending order of the amount of real memory used. 3. The destination processor is selected by a CPU.
The method according to claim 1, wherein the steps are performed in descending order of usage rate.
Alternatively, a load distribution control method between processors in the multiprocessor system according to the second aspect.