JP2012173753A - Job scheduling system, job scheduling method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a job scheduling system improving operating ratio of a system.SOLUTION: A job scheduling system of this invention comprises: a scheduler map storage part to store a scheduler map; a job targeted for assignment selection part to select a job to be assigned in the scheduler map by backfill type scheduling; a space information retrieval part to retrieve a space area of the scheduler map; a passing possibility distinction part to distinguish a passing possibility job not to delay a job execution start time of a delay prohibition job even if the passing possibility job overtakes the delay prohibition job; an execution estimated time determination part to determine an execution estimated time of the passing possibility job; and an execution plan calculation node selection part to select an execution plan calculation node or the passing possibility job.

Description

  The present invention relates to a job scheduling system.

  In a clustering system composed of a plurality of computing nodes, a job scheduler using backfill type scheduling (hereinafter referred to as “backfill scheduler”) is used. The backfill scheduler uses a two-dimensional map (hereinafter referred to as a “scheduler map”) centered on the calculation resources and time of the system to allocate the resources required by the job while managing the scheduled execution time of the job ( Hereinafter, the scheduler is referred to as “assign”. A job scheduling system based on a scheduler map is used mainly in the field of HPC (High Performance Computing).

  In a conventional job scheduling system using a scheduler map, a large-scale job that uses a large amount of the system's computational resources continues to be overtaken by a small-scale job that uses a small amount of the system's computational resources. There was a problem that was not. FIG. 1 is a diagram for explaining a problem of a job scheduling system based on a conventional scheduler map. In the scheduler map 101 in FIG. 1, the vertical axis represents the number of calculation nodes, and the horizontal axis represents time. In the scheduler map 101, a job to be executed by the calculation nodes 1 to 4 can be reserved for a job whose processing starts during the time from the current time to the end of the map. In the example of FIG. 1, a state where there are jobs 7 to 10 as the assignment waiting job 102 is shown. In this example, execution of the large-scale job 7 is not started for a long time because the large-scale job 7 that heavily uses the computational resources of the system is overtaken by the small-scale jobs 8 and 9 that use less computational resources of the system. It is shown.

  The conventional job scheduling system based on the scheduler map 101 is provided with an overtaking prohibition function in order to solve the problem described with reference to FIG. FIG. 2 is a diagram for explaining the overtaking prohibiting function of the job scheduling system according to the conventional scheduler map 101. In general, the overtaking prohibition function is a function that prioritizes assignment of a corresponding job by prohibiting overtaking of subsequent jobs when there is a job that cannot be assigned for a certain period of time. In the example of FIG. 2, a state where there are jobs 7 to 10 as the assignment waiting job 102 is shown. In this example, since there is a job 7 that could not be assigned for a certain time, the subsequent jobs 8, 9, 10 are prohibited from being overtaken with respect to the job 7. However, when the overtaking prohibition function is enabled, a small-scale job cannot be assigned to the scheduler map 101, and an empty area increases in the calculation node, which may lower the system operation rate. In the example of FIG. 2, since the overtaking prohibition function is enabled, the free area of the computation node 3 is increased.

  Conventional clustering systems in the HPC field have been mainly operated by classifying each size (number of calculation nodes used, number of CPUs) and dividing calculation resources used for each class. In such an operation, since the class is different between a large-scale job and a small-scale job, there is no resource contention. However, in recent years, a method of sharing and managing all resources of the clustering system has become mainstream, and problems due to resource competition have occurred between large-scale jobs and small-scale jobs. Therefore, there is a need for a job scheduling system that suppresses a reduction in the operating rate of the clustering system by efficiently assigning jobs even when large-scale jobs and small-scale jobs are mixed.

  Japanese Patent Laid-Open No. 2001-22601 (Patent Document 1) is known as a document known invention in the field of the present invention. In Japanese Patent Laid-Open No. 2001-22601, as a technique related to job scheduling, a job execution control method and a parallel computer that preferentially schedule a job whose execution request is issued at the head of a plurality of jobs waiting to be executed Techniques related to the system are disclosed.

JP 2001-22601 A

  An object of the present invention is to provide a job scheduling system that improves the operating rate of a system by preceding execution of a small-scale job as much as possible without delaying the start of execution of the large-scale job.

  A job scheduling system according to the present invention includes a scheduler map storage unit that stores a scheduler map, an assignment target job selection unit that selects a job to be assigned to the scheduler map by backfill scheduling, and a space for searching for an empty area of the scheduler map. Information search unit, passable discriminating unit that discriminates a passable job that does not delay the job execution start time of a delay-prohibited job even if it overtakes a delay-prohibited job, and execution that determines the scheduled execution time of the passable job A scheduled time determination unit; and an execution schedule calculation node selection unit that selects an execution schedule calculation node of the passable job.

  The job scheduling method of the present invention is a job scheduling method implemented by a job scheduling system. The scheduler map storing unit stores the scheduler map, the assignment target job selecting unit selects a job to be assigned to the scheduler map by backfill scheduling, and the assignment target job selecting unit selects the job If the selected job is a delay-prohibited job, a step of assigning the selected job to the special assignment area, a step of searching for a free area of the scheduler map, and a passable discriminating unit, A step of determining a passable job that does not delay the job execution start time of the delay-prohibited job even if it overtakes the delay-prohibited job, and a step of determining a scheduled execution time of the passable job, The scheduled execution node selection unit will execute the job that can be overtaken. And a step of selecting a schedule compute nodes.

  According to the present invention, it is possible to provide a job scheduling system that improves the operating rate of a system by preceding execution of a small-scale job as much as possible without delaying the start of execution of the large-scale job.

FIG. 1 is a diagram for explaining a problem of a job scheduling system based on a conventional scheduler map. FIG. 2 is a diagram for explaining an overtaking prohibiting function of a job scheduling system based on a conventional scheduler map. FIG. 3 is a block diagram of the job scheduling system 100 according to the first embodiment of the present invention. FIG. 4 is a diagram for explaining the scheduler map 21 of the job scheduling system 100 according to the first embodiment of the present invention. FIG. 5 is a hardware configuration diagram of the job scheduling system 100 according to the first embodiment of the present invention. FIG. 6 is a flowchart of the job scheduling method of the job scheduling system 100 according to the first embodiment of the present invention. FIG. 7 is a block diagram of the job scheduling system 100 according to the second embodiment of the present invention. FIG. 8 is a flowchart of the job scheduling method of the job scheduling system 100 according to the second embodiment of the present invention. FIG. 9 is a diagram for explaining the scheduler map 21 of the job scheduling system 100 according to the second embodiment of the present invention.

(First embodiment)
A job scheduling system 100 according to a first embodiment of the present invention will be described below with reference to the accompanying drawings.

[Description of configuration]
First, the functional configuration of the job scheduling system 100 in this embodiment will be described. FIG. 3 is a block diagram of the job scheduling system 100 according to the first embodiment of the present invention.

  The job scheduling system 100 includes an assignment target job selection unit 10, a scheduler map storage unit 20, and a job assignment unit 30.

  The assignment target job selection unit 10 includes a delay prohibition job determination unit 11 and a special assignment area reservation unit 12, and selects a job to be assigned to the scheduler map 21 from the assignment waiting job 22. FIG. 4 is a diagram for explaining the scheduler map 21 of the job scheduling system 100 according to the first embodiment of the present invention. In the scheduler map 21 of FIG. 4, the number of calculation nodes is plotted on the vertical axis, and the time is plotted on the horizontal axis. In the scheduler map 21, a job to be executed by the calculation nodes 1 to 4 can be reserved for a job whose processing starts during the time from the current time to the end of the map. The scheduler map 21 shown in FIG. 4 is different from the scheduler map 101 shown in FIGS. 1 and 2 in that a special assign area 23 is provided. The special assignment area 23 will be described later. The delay prohibition job determination unit 11 determines whether the target job of the assignment waiting job 22 is a job in which the delay of the job execution start time is prohibited. The delay prohibition job determination unit 11 performs the processing of the special assignment area reservation unit 12 when the delay of the job execution start time of the target job is prohibited and the special assignment area 23 is not yet reserved in the scheduler map 21. call. The special assignment area securing unit 12 secures in the scheduler map 21 a special assignment area 23 for assigning a job whose delay in job execution start time is prohibited.

  The scheduler map storage unit 20 stores information on the scheduler map 21. The information of the scheduler map 21 includes a calculation resource used by the job, a job execution start time, a job execution required time, and the like.

  The job assigning unit 30 includes a free space information searching unit 31, a scheduled execution time determining unit 32, an overtaking possible determination unit 33, and a scheduled execution calculation node selecting unit 34, and assigns a job to the scheduler map 21. The free area information search unit 31 searches for free areas (locations of resources that can be assigned) in the scheduler map 21. The scheduled execution time determination unit 32 determines the scheduled execution time of the assignment waiting job 22. The overtaking possible determination unit 33 does not delay the job execution start time of the job assigned to the special assignment area even if the job assigned to the special assignment area is overtaken (hereinafter, “does not overlap with the special assignment area 23”). Job)). The overtaking possibility determination unit 33 permits a job assigned to the special assignment area 23 to be overtaken for a job that does not overlap the special assignment area 23. The execution schedule calculation node selection unit 34 reserves in the scheduler map 21 the job assignment area for which the scheduled execution time has been determined.

  Next, the hardware configuration of the job scheduling system 100 in this embodiment will be described. FIG. 5 is a hardware configuration diagram of the job scheduling system 100 according to the first embodiment of the present invention.

  The job scheduling system 100 of this embodiment is implemented in the management node 5 that manages the computation nodes 1 to 4 in the clustering system 50 as shown in FIG. Although the number of calculation nodes in the clustering system 50 in FIG. 5 is four, the number of calculation nodes is not limited to four. The job scheduling system 100 in the management node 5 is realized by the CPU 6 executing a job scheduling program 9 developed in the memory 8. The job scheduling program 9 is stored in an auxiliary storage device 7 such as an HDD (Hard Disk Drive).

[Description of operation method]
Next, a job scheduling method in the job scheduling system 100 according to the present embodiment will be described. FIG. 6 is a flowchart of the job scheduling method of the job scheduling system 100 according to the first embodiment of the present invention.

(Step S1)
The assignment target job selection unit 10 selects an assignment target job from the assignment waiting jobs 22.

(Step S2)
The delay prohibition job determination unit 11 determines whether or not the job selected in step S1 is an execution start delay prohibition job. The delay prohibition job determination unit 11 proceeds to step S3 if it is an execution start delay prohibition job, and proceeds to step S6 if it is not an execution start delay prohibition job.

  As an example of a method for determining an execution start delay prohibition job, a method of assigning a job that stays for a long time to the assignment waiting job 22 can be considered. In the example of FIG. 4, a job that uses a lot of resources, such as job 7, cannot be assigned to the scheduler map 21 and often stays as an assignment waiting job 22. For this reason, such a job is considered to be a job with a high priority, and a method of making it an execution start delay prohibited job can be considered.

(Step S3)
The delay prohibition job determination unit 11 checks whether the special assignment area 23 is secured in the scheduler map 21. The delay prohibition job determination unit 11 proceeds to step S5 when the special assignment area 23 is secured, and proceeds to step S4 when the special assignment area 23 is not secured.

(Step S4)
The special assignment area securing unit 12 secures the special assignment area 23 at the map end of the scheduler map 21. The special assignment area 23 is an area where only jobs that do not delay the job start time due to overtaking by other jobs can be assigned, and an area is unconditionally secured at the end of the scheduler map 21 (other jobs already assigned). May overlap). The special assignment area 23 does not move on the scheduler map 21 in conjunction with the passage of time, and is always present at the end of the scheduler map (for jobs overlapping on the special assignment area 23, the time passes. Move in the direction of the current time). Since it is prohibited to assign jobs other than the execution start delay prohibition job to the special assignment area 23, the special assignment area is always an empty area when time elapses, and a job for which overtaking is prohibited can be assigned. . This guarantees the execution start time of the execution start delay prohibited job.

(Step S5)
The special assignment area securing unit 12 assigns a job to be assigned, which is regarded as an execution start delay prohibited job, to the special assignment area 23, and ends the scheduling process for the assignment target job.

(Step S6)
The free area information search unit 31 searches for a free area in the scheduler map 21 to which a job to be assigned can be assigned.

(Step S7)
The overtaking possible determination unit 33 determines whether the assignment target job does not delay the execution start time of the execution start delay prohibited job. If the assignment target job can be assigned without overlapping (not overlapping) the special assignment area 23, the execution start delay prohibition job is not affected, so that the execution start delay prohibition job can be overtaken. Is determined.

(Step S8)
The scheduled execution time determination unit 32 determines the scheduled execution time of the job to be assigned.

(Step S9)
The execution schedule calculation node selection unit 34 secures an assignment area for the job to be assigned on the scheduler map 21 and determines a calculation node to be used by the job to be assigned.

  In the example of FIG. 4, when the job 9 passes the execution start delay prohibition job 7 and is assigned on the scheduler map 21 and the special assignment area 23 becomes free as time elapses, the execution start delay prohibition job 7 is An example of assignment is shown.

  According to the present embodiment, in the conventional job scheduling system in which the overtaking prohibition function is enabled, a small-scale job (job 9 in the example of FIG. 4) for which overtaking is prohibited is assigned on the scheduler map 21. Therefore, the free area on the scheduler map 21 is reduced, and the operating rate of the clustering system 50 is improved.

  In addition, after securing the special assignment area 23, jobs other than execution start delay prohibition jobs cannot be assigned to the special assignment area 23. Therefore, when time elapses, no jobs are overlapped with the special assignment area. . Therefore, the special assignment area 23 is assigned with a large-scale job (job 7 in the example of FIG. 4) forbidden to be overtaken. This guarantees the execution start time of a large-scale job whose execution start delay is prohibited.

(Second Embodiment)
A job scheduling system 100 according to a second embodiment of the present invention will be described below with reference to the accompanying drawings.

[Description of configuration]
First, the functional configuration of the job scheduling system 100 in this embodiment will be described. FIG. 7 is a block diagram of the job scheduling system 100 according to the second embodiment of the present invention. The block diagram of FIG. 7 is the same as the block diagram of FIG. 3 except that a special assignment area moving unit 35 is added to the job assignment unit 30. Therefore, descriptions other than the special assignment area moving unit 35 are omitted.

  The special assignment area moving means 35 moves the special assignment area 23 secured by the special assignment area securing unit 12. In the first embodiment of the present invention, the special assignment area 23 is fixed. However, the second embodiment is different from the first embodiment in that the special assignment area 23 can be moved.

[Description of operation method]
Next, a job scheduling method in the job scheduling system 100 according to the present embodiment will be described. FIG. 8 is a flowchart of the job scheduling method of the job scheduling system 100 according to the second embodiment of the present invention. Since the operation method from step S1 to step S7 is the same as that of the first embodiment of the present invention, the operation method after step S8 will be described.

(Step S1) to (Step S7)
The operation method is the same as that of the first embodiment of the present invention.

(Step S8)
The overtaking possible determination unit 33 calls the processing of the special assignment region moving unit 35 when determining whether a job to be assigned can be assigned to an empty region of the scheduler map 21. The special assignment area moving unit 35 moves the special assignment area 23 secured at the end of the scheduler map 21 (to a future time) so that the succeeding job can be assigned in a state where the free area of the scheduler map 21 is the smallest. Slide). FIG. 9 is a diagram for explaining the scheduler map 21 of the job scheduling system 100 according to the second embodiment of the present invention. In the example of FIG. 9, when the job 8 is assigned to the calculation node 3 and the job 9 is assigned to the calculation node 1, the scheduler map 21 has the smallest free space. Therefore, the special assignment area moving unit 35 moves the special assignment area 23 to the rear of the area where the job 9 is to be assigned. Since the special assignment area 23 gives priority to the start of execution of a large-scale job for which overtaking is prohibited, the special assignment area 23 can be moved only within a certain time (hereinafter referred to as “limit”) from the map end time. When the special assignment area 23 is moved by the special assignment area moving means 35, it moves in the current time direction as time elapses from the moved position to the end of the scheduler map 21. When the special assignment area 23 reaches the end of the scheduler map 21, the special assignment area 23 stops moving. The limit, which is the movable width of the special assignment area 23, is set as a configuration parameter, so that it is possible to control how far it can be delayed by operating the job scheduling system 100 (parameter limit is set to 0). Is set to the same operation as in the first embodiment).

(Step S9)
The scheduled execution time determination unit 32 determines the scheduled execution time of the job to be assigned.

(Step S10)
The execution schedule calculation node selection unit 34 secures an assignment area for the job to be assigned on the scheduler map 21 and determines a calculation node to be used by the job to be assigned.

  According to this embodiment, by making the special assignment area 23 movable, in the first embodiment of the present invention, a plurality of small jobs (in FIG. In the example, job 8 and job 9) can be assigned on the scheduler map 21. Therefore, it becomes possible to use the free area on the scheduler map 21 more effectively than in the first embodiment, and the operating rate of the clustering system 50 is improved.

  The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the above-described embodiments, and can be appropriately changed by those skilled in the art without departing from the scope of the invention.

1-4 computing node 5 management node 6 CPU (Central Processing Unit)
7 Auxiliary storage device 8 Memory 9 Job scheduling program 10 Assignment target job selection unit 11 Delay prohibition job determination unit 12 Special assignment area securing unit 20 Scheduler map storage unit 21 Scheduler map 22 Job waiting for assignment 30 Job assignment unit 31 Free area information search unit 32 scheduled execution time determination unit 33 overtaking possible determination unit 34 scheduled execution calculation node selection unit 35 special assignment area moving unit 50 clustering system 100 job scheduling system 101 prior art scheduler map 102 job waiting for assignment

Claims (7)

A scheduler map storage unit for storing a scheduler map, wherein the scheduler map includes a special assignment area to which a delay prohibition job in which a delay in job execution start time is prohibited is provided at the end of the scheduler map;
An assignment target job selection unit that selects a job to be assigned to the scheduler map by backfill type scheduling, and the assignment target job selection unit, when the selected job is the delay prohibition job, Assign the selected job to the special assignment area,
A free information search unit for searching for a free area of the scheduler map;
An overtaking possible determination unit that determines an overtaking possible job that does not delay the job execution start time of the delay prohibited job even after overtaking the delay prohibited job;
A scheduled execution time determination unit that determines a scheduled execution time of the overtaking job;
A job scheduling system comprising: an execution schedule calculation node selection unit that selects an execution schedule calculation node of the passable job. The job scheduling system according to claim 1, wherein the assignment target job selection unit determines whether the assignment target job is the delay-prohibited job, as the job with the longest assignment waiting time. The job scheduling system according to claim 1, wherein the position of the special assignment area can be moved on the scheduler map within a predetermined time determined by a configuration parameter. A scheduler map storage unit storing the scheduler map, wherein the scheduler map includes a special assignment area to which a delay prohibition job in which a delay in job execution start time is prohibited is allocated at the end of the scheduler map; ,
The assignment target job selection unit selects a job to be assigned to the scheduler map by backfill scheduling, and
Assigning the selected job to the special assignment area when the job selected by the assigning job selection unit is the delay prohibition job;
A step of searching for a free area of the scheduler map, a free information search unit;
A step of determining a passable job that does not delay the job execution start time of the delay-prohibited job even if the passable determination unit passes the delay-prohibited job;
A scheduled execution time determining unit determining a scheduled execution time of the passable job;
A job scheduling method comprising: a scheduled execution calculation node selection unit selecting an execution scheduled calculation node of the passable job. The step of assigning the selected job to the special assignment area includes:
The job scheduling method according to claim 4, further comprising: determining whether the job to be assigned is the delay-prohibited job as the job having the longest assignment waiting time. The job scheduling method includes:
The job scheduling method according to claim 4, further comprising a step of moving the position of the special assignment area on the scheduler map for a fixed time determined by a configuration parameter.   A program for causing a computer to execute the job scheduling method according to any one of claims 4 to 6.

Images