WO2007043142A1

WO2007043142A1 - Job management device and job management program

Info

Publication number: WO2007043142A1
Application number: PCT/JP2005/018418
Authority: WO
Inventors: Makoto Kanno
Original assignee: Fujitsu Limited
Priority date: 2005-10-05
Filing date: 2005-10-05
Publication date: 2007-04-19

Abstract

It is possible to effectively perform allocation even if the generation ratio of a sequential job and a parallel job is changed. When a job queue is a parallel job processing request, parallel job allocation means (1f) decides an empty node in accordance with the parallel number of the parallel jobs as an allocation destination. When a job queue is a successive job processing request, allocation method selection means (1g) judges whether the number of empty nodes is appropriate according to the number of empty nodes and the number of reserved nodes. If the number of empty nodes is insufficient, a node which has already executed another job is decided as an allocation destination of the job acquired by the job queue acquisition means (1d) with a higher priority. If the number of empty nodes is sufficient, a node which has not executed a job is decided as an allocation destination of the job acquired by the job queue acquisition means (1d) with a high priority.

Description

Specification

Job management apparatus and job management program

Technical field

TECHNICAL FIELD [0001] The present invention relates to a job management apparatus and job management program that allocate a queued job to one of a plurality of nodes, and more particularly to a job management apparatus and job management that allocates sequential jobs and parallel jobs to nodes. Regarding the program.

Background art

In order to distribute jobs (processing execution units) to multiple computers (nodes) in a distributed manner, it is necessary to efficiently allocate jobs. Therefore, using a server or the like, job assignment is performed based on the load state of each node (see, for example, Patent Documents 1 and 2).

[0003] However, the techniques disclosed in Patent Documents 1 and 2 do not consider job assignment when sequential jobs and parallel jobs are mixed. Therefore, in a system that executes processing of sequential jobs and parallel jobs, job execution efficiency cannot be sufficiently improved. Here, the sequential job is a sequential job executed on one node, and the parallel job is a job executed on multiple nodes in parallel.

[0004] In the process of assigning sequential jobs and parallel jobs to nodes, whether or not the ability to execute a plurality of jobs on one node differs greatly. In other words, if multiple CPUs (Central Processing Units) are installed in one node, multiple jobs can be executed simultaneously on that node. However, when multiple jobs are executed on one node, the execution performance will be lower than when the job is executed alone due to contention for resources (memory, etc.) used by the job.

[0005] Sequential jobs have a wide range of tolerances to the decline in execution performance, so multiple jobs can be executed on one node. On the other hand, in a parallel job, information is transferred between processes executed in parallel. Therefore, if multiple jobs are executed on one node, the execution performance deteriorates. If the timing for passing information to other nodes is delayed, waiting time occurs on other nodes, and processing is also delayed.

[0006] To eliminate the waiting time for acquiring information on other node forces when executing parallel jobs, It is desirable that the node execution performance is uniform. Therefore, when executing parallel jobs, only one job is executed on one node.

[0007] Therefore, when a parallel job is assigned to a node, it is necessary that the assignment destination node is an empty node (a node having the number of jobs being executed of “0”). In other words, in order to assign parallel jobs to nodes, there must be as many free nodes as the number of processes executed in parallel (the number of parallel processes). If there are not enough free nodes, the parallel job is waiting to be executed. Thus, the conditions for assigning jobs differ between sequential jobs and parallel jobs.

[0008] Conventionally, in a system that executes a job in which sequential jobs and parallel jobs are mixed, either an stuffing method or a distributed arrangement method has been adopted. In the stuffing method, in order to prioritize the execution of parallel jobs, multiple sequential jobs are allocated to one node and up to the number of processors that the node has. In the distributed placement method, sequential jobs are assigned to each node in order to avoid resource contention on one node as much as possible.

FIG. 15 is a diagram showing job assignment by the stuffing method. As shown in FIG. 15, six nodes 921 to 926 are connected via a network 910. Each node 921 to 926 is assumed to have two processors. Then, the job management apparatus 931 assigns the job to each of the nodes 921 to 926 by a packing method.

[0010] In this example, a parallel job 945 occurs after four sequential jobs 941 to 944 continue. The number of parallel jobs 945 (the number of nodes that perform parallel processing is “3”. In FIG. 15, the order in which jobs are queued is shown in the job.

[0011] In the stuffing method, jobs that can be executed by the nodes are sequentially assigned to each node in a predetermined order. Assigned to first and second sequential jobs 941, 942 force node 921. Next, the third and fourth sequential jobs 943, 944 are assigned to node 922. The fifth parallel job 945 is assigned to nodes 923-925.

FIG. 16 is a diagram showing job assignment in the distributed arrangement method. In this example, jobs similar to those in FIG. 15 are assigned to each of the nodes 921 to 926 by the job management apparatus 932 by the distributed arrangement method.

[0013] In the distributed placement method, the generated jobs are distributed so that the load on each node is equalized. Placed. Specifically, the first to fourth sequential jobs 941 to 944

Assigned to 924. At this point, there are only two free nodes, node 925 and node 926. Therefore, the parallel job 945 having the parallel processing power “3” is in a wait state until the number of free nodes becomes “3”.

Patent Document 1: Japanese Patent Laid-Open No. 8-305671

Patent Document 2: JP 2000-315199 A

Disclosure of the invention

Problems to be solved by the invention

[0014] However, in the past, since the job trend was investigated in advance, the operation policy was determined, and the job allocation method was selected, the throughput of the entire system was reduced when the trend of the generated job changed. There is a problem.

[0015] For example, when there are many parallel jobs, a stuffing method that prioritizes parallel jobs is adopted.

. However, in the stuffing method, when the number of parallel jobs decreases, the number of empty nodes increases and the operating rate of the entire system decreases. In the stuffing method, multiple sequential jobs are often executed on one node, and the execution performance of sequential jobs is also reduced.

In addition, when there are many sequential jobs, a distributed arrangement method that avoids resource contention is adopted. However, in the distributed arrangement method, when the number of parallel jobs increases, there are insufficient free nodes, and parallel jobs wait for a long time.

[0017] The present invention has been made in view of the above points, and a job management apparatus and a job management apparatus capable of performing efficient job allocation even when the generation ratio of sequential jobs and parallel jobs changes. And a job management program.

Means for solving the problem

In the present invention, in order to solve the above problems, a job management apparatus 1 as shown in FIG. 1 is provided. The job management apparatus 1 according to the present invention assigns a sequential job executed in one node 2a, 2b,... And a parallel job executed in parallel processing by a plurality of nodes to nodes. Therefore, it has the following functions.

When a job queue indicating a job processing request is input, the queue buffer la stores the job queue in the input order. Reserved node number determination means lb is stored in queue buffer la Based on the stored job queue indicating the parallel job processing request, the number of reserved nodes indicating the number of nodes to be reserved in advance for the parallel job processing is determined. The free node number calculating means lc monitors whether or not a job is executed on each of a plurality of nodes, executes the job, and calculates the number of free nodes indicating the number of nodes. Job queue acquisition means Id acquires the job queues stored in the queue buffer in the order in which they were stored. The job type determination unit le determines whether the job queue acquired by the job queue acquisition unit Id is a parallel job processing request or a sequential job processing request. When the job queue acquired by the job queue acquisition unit Id is a parallel job processing request, the parallel job allocation unit If determines an empty node corresponding to the parallel number of parallel jobs as an allocation destination. When the job queue acquired by the job queue acquisition unit Id is a sequential job processing request, the allocation method selection unit lg determines whether the number of free nodes is excessive or insufficient based on the number of free nodes and the number of reserved nodes. If the number of free nodes is insufficient, the stuffing method is selected as the job allocation method, and if the number of free nodes is sufficient, the distributed arrangement method is selected as the job allocation method. When the stuffing method is selected, the stuffing method job allocation unit lh preferentially determines a node that has already executed another job as an assignment destination of the job acquired by the job queue acquisition unit Id. When the distributed allocation method is selected, the distributed allocation method job allocation unit li preferentially determines a node that is not executing a job as an allocation destination of the job acquired by the job queue acquisition unit Id. The processing request transmission means lj transmits the processing request for the job indicated by the job queue acquired by the job queue acquisition means Id to the node determined as the allocation destination.

According to such a job management apparatus, when a job queue indicating a job processing request is input, the job queues are stored in the queue buffer la in the input order. Also, the reserved node number determining means lb determines the reserved node number indicating the number of nodes that should be reserved in advance for processing the parallel job. Also, the number of free nodes calculating means lc monitors whether or not a job is executed in each of the plurality of nodes, and calculates the number of free nodes indicating the number of obsolete nodes by executing the job. Further, job queues stored in the queue buffer are acquired in the order in which they are stored by the job queue acquisition means Id. Then, the job queue obtained by the job queue obtaining means Id is parallelized by the job type judging means le. It is determined whether it is a job processing request or a sequential job processing request. Job queue acquisition means

When the job queue acquired by Id is a parallel job processing request, the parallel job allocation unit If determines the free node corresponding to the parallel number of parallel jobs as the allocation destination. Job queue acquisition means When the job queue acquired by Id is a sequential job processing request, the allocation method selection means lg determines whether the number of free nodes is excessive or insufficient based on the number of free nodes and the number of reserved nodes. If the number of nodes is insufficient, the stuffing method is selected as the job allocation method, and if the number of free nodes is sufficient, the distributed arrangement method is selected as the job allocation method. When the filling method is selected, the node that has already executed another job is preferentially determined by the filling method job assignment unit lh as the assignment destination of the job acquired by the job queue acquisition unit Id. When the distributed placement method is selected, the V-zone that is not executing the job is preferentially determined by the distributed placement method job assignment means li as the assignment destination of the job acquired by the job queue acquisition means Id. Then, the processing request transmitting unit lj transmits the processing request for the job indicated by the job queue acquired by the job queue acquiring unit Id to the node determined as the allocation destination.

In order to solve the above problem, in a job management program that assigns to a node a sequential job executed on one node and a parallel job executed in parallel processing by a plurality of nodes, When a job queue indicating a job processing request is input, a parallel processing is performed based on a queue buffer storing the job queue in the input order and a job queue indicating a parallel job processing request stored in the queue buffer. Determines the number of reserved nodes indicating the number of nodes that should be reserved in advance for job processing. Reserved node number determination means, monitors the execution of jobs on each of the plurality of nodes, and executes jobs. An empty node number calculating means for calculating the number of empty nodes indicating the number of nodes, and the job stored in the queue buffer Job queue acquisition means for acquiring queues in the order of storage, job type determination means for determining whether the job queue acquired by the job queue acquisition means is a parallel job processing request or a sequential job processing request, and the job When the job queue acquired by the queue acquisition means is a parallel job processing request, a free node corresponding to the parallel number of parallel jobs is determined as an allocation destination. If the job queue acquired by the queue job allocation means and the job queue acquisition means is a sequential job processing request, the excess number of empty nodes is determined based on the number of empty nodes and the number of reserved nodes! If the number of empty nodes is insufficient, the stuffing method is selected as the job allocation method, and the distributed allocation method is selected as the job allocation method when the number of empty nodes is sufficient. Method selection means, when the stuffing method is selected, a stuffing method job assignment means for preferentially determining a node already executing another job as an assignment destination of the job acquired by the job queue acquisition means, When the distributed placement method is selected, jobs that have been acquired by the job queue acquisition means are given priority to nodes that are not executing jobs. Distributed allocation method job allocation means that determines the allocation destination of the job, and processing request transmission that transmits the processing request of the job indicated by the job queue acquired by the job queue acquisition means to the node determined as the allocation destination A job management program characterized in that it functions as a means is provided.

[0022] By causing a computer to execute such a job management program, the same function as that of the job management apparatus is realized by the computer.

The invention's effect

[0023] In the present invention, the number of reserved nodes is determined based on a job queue indicating a parallel job processing request stored in the queue buffer, and an excess of the number of free nodes is determined based on the number of free nodes and the number of reserved nodes. Judgment is made, and if the number of free nodes is insufficient, a sequential job is assigned to the nodes using the stuffing method. If the number of free nodes is sufficient, a sequential job is assigned to the nodes using the distributed placement method. As a result, when there are few parallel job processing requests, a distributed arrangement method that prioritizes sequential jobs is adopted, and when parallel job processing requests increase, a stuffing method that gives priority to parallel jobs is adopted. As a result, efficient job assignment can always be performed.

[0024] The above and other objects, features and advantages of the present invention are preferred as examples of the present invention, and will become apparent from the following description in conjunction with the accompanying drawings showing embodiments.

Brief Description of Drawings

FIG. 1 is a diagram showing an outline of the present embodiment. FIG. 2 is a diagram showing an example of job distribution.

FIG. 3 is a diagram showing a system configuration example according to the present embodiment.

FIG. 4 is a diagram illustrating a hardware configuration example of a job management server used in the present embodiment.

FIG. 5 is a block diagram showing functions of the job management server.

FIG. 6 is a block diagram illustrating functions of a job assignment unit.

FIG. 7 is a flowchart showing a procedure for job assignment processing.

FIG. 8 shows an example of a queued job.

FIG. 9 is a diagram showing job assignment statuses up to the third.

FIG. 10 is a diagram showing a state after completion of parallel job processing.

FIG. 11 is a diagram showing a state where up to the sixth job is assigned.

FIG. 12 is a diagram showing a state in which all jobs are assigned.

FIG. 13 is a diagram illustrating an example in which jobs are assigned only by a filling method.

FIG. 14 is a diagram showing an example in which jobs are assigned only by a distributed arrangement method.

FIG. 15 is a diagram showing job assignment by a filling method.

FIG. 16 is a diagram showing job assignment in a distributed arrangement method.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an outline of the present embodiment. The job management apparatus 1 assigns sequential jobs executed on one node and parallel jobs executed by parallel processing by a plurality of nodes to the nodes 2a, 2b,. For this purpose, the job management apparatus 1 includes a queue buffer la, a reserved node number determining unit lb, a free node number calculating unit lc, a job queue obtaining unit Id, a job type determining unit le, a parallel job allocation unit lf, and an allocation method selection unit lg. It has a stuffing method job assignment means lh, a distributed arrangement method job assignment means li, and a processing request transmission means 1 j.

When a job queue indicating a job processing request is input, the queue buffer la stores the job queue in the input order. The input job processing requests include processing requests for sequential jobs and parallel jobs. In the job queue, parallel jobs or sequential jobs Information is included. The parallel job processing request indicates the parallel number of the parallel job.

[0028] The reserved node number determination means lb determines the number of nodes to be reserved in advance for processing a parallel job based on the job queue indicating the processing request for the parallel job stored in the queue buffer la. Determine the number of reserved nodes to show. For example, referring to a job queue indicating a parallel job processing request stored in the queue buffer la, the total number of parallel jobs of each parallel job can be set as the number of reserved nodes.

[0029] Free node number calculation means lc monitors the execution of jobs on each of the plurality of nodes 2a, 2b, ···, and executes the job to calculate the number of free nodes indicating the number of nodes To do.

[0030] Job queue acquisition means Id acquires the job queues stored in the queue buffer la in the order of storage.

The job type determination unit le determines whether the job queue acquired by the job queue acquisition unit Id is a parallel job processing request or a sequential job processing request.

[0031] If the job queue acquired by the job queue acquisition unit Id is a parallel job processing request, the parallel job allocation unit If determines an empty node corresponding to the parallel number of parallel jobs as an allocation destination.

[0032] When the job queue acquired by the job queue acquisition unit Id is a sequential job processing request, the allocation method selection unit lg determines whether the number of free nodes is excessive or insufficient based on the number of free nodes and the number of reserved nodes. Judging. For example, the allocation method selection means lg determines that the number of free nodes is insufficient if the number of free nodes is equal to or less than the number of reserved nodes, and determines the number of free nodes if the number of free nodes exceeds the number of reserved nodes. Is judged to be sufficient. The allocation method selection means lg selects the stuffing method as the job allocation method if the number of free nodes is insufficient, and the distributed allocation method as the job allocation method if the number of free nodes is sufficient. Select.

[0033] When the stuffing method is selected, the stuffing method job assigning means lh executes another job and preferentially determines the node to which the job acquired by the job queue obtaining means Id is assigned as the assignment destination. . In other words, the stuffing method job allocation unit lh selects a node that can execute a job from among nodes that execute another job, and allocates it. It is a target.

[0034] When the distributed allocation method is selected, the distributed allocation method job allocation unit li executes a job and preferentially determines the source code as the allocation destination of the job acquired by the job queue acquisition unit Id. .

The processing request transmission unit lj transmits a processing request for the job indicated by the job queue acquired by the job queue acquisition unit Id to the node determined as the assignment destination. In response to this processing request, the job is processed at each node.

[0036] Such a job management apparatus 1 is a system having a configuration in which a plurality of nodes 2a, 2b, ... are connected via a network. Each node 2a, 2b,... Has a plurality of CPUs. When a job is submitted, it is queued in the queue buffer la. The job management apparatus 1 assigns the queued job to the node, and arranges and executes the job on the node. At this time, the node that allocates the job based on the number of nodes to which no job is assigned (free node), the number of free nodes in the entire system, the remaining time of running jobs, and the type of queued job. Is determined.

For example, when the number of parallel jobs is small, the number of reserved nodes can be small. Therefore, if there are not many job queues indicating parallel job processing requests in the queue buffer la, the reserved node number determining means lb sets the reserved node number to be small. As a result, if the number of free nodes is larger than the number of reserved nodes, sequential jobs can be allocated by the distributed arrangement method. In the distributed placement method, jobs are preferentially assigned to free nodes, so the execution efficiency of the entire system can be improved.

[0038] When the number of parallel jobs increases, the execution waiting time for parallel jobs becomes long unless a large number of free nodes are secured. Therefore, when the number of job queues indicating processing requests for parallel jobs increases in the queue buffer la, the reserved node number determination means lb increases the number of reserved nodes. As a result, even if the number of parallel jobs increases, it is possible to prevent the execution of waiting for execution of parallel jobs.

FIG. 2 is a diagram illustrating an example of job distribution. In this figure, jobs are assigned according to the present embodiment under the same conditions as in the conventional example shown in FIGS. In this example, six nodes 2a, 2b, 2c, 2d, 2e, 2f are connected via network 3. No Each of the nodes 2a, 2b, 2c, 2d, 2e, and 2f has two CPUs. Then, after the job queue indicating the processing requests of the sequential jobs 4a, 4b, 4c, and 4d continues in the queue buffer la of the job management device 1, the job queue indicating the processing requests of the three parallel jobs 4e is stored. It shall be assumed. The total number of parallel jobs in queue buffer la is the number of reserved nodes.

In this example, the sequential jobs 4a, 4b, 4c are distributed to the nodes 2a, 2b, 2c, respectively, by the distributed arrangement method. At this point, the number of free nodes matches the number of reserved nodes. Therefore, the next sequential job 4d is assigned to the node 2a by the filling method. The parallel job 4e is assigned harm to the nodes 2d, 2e, and 2f.

[0041] Comparing the allocation results shown in Fig. 2 with the case where only the stuffing method is applied (see Fig. 15), the number of nodes that execute multiple jobs on one node is reduced, and the execution efficiency of the entire system is reduced. It can be seen that it has improved. In addition, comparing the assignment results in Fig. 2 with the case where only the distributed arrangement method is applied (see Fig. 16), it can be seen that parallel jobs can be executed immediately, and the processing efficiency of parallel jobs is improved.

[0042] That is, if jobs are assigned only by the distributed arrangement method, the waiting time for executing parallel jobs increases (the parallel jobs are poorly swept). Also, if you assign jobs using only the stuffing method, multiple sequential jobs will be stuffed into one node even though there are empty nodes! /, Sweeping sequential jobs is bad! ,.

In the present embodiment, sequential jobs that do not hinder the execution of parallel jobs are arranged so as to be distributed to the nodes, and the job can be swept. Another factor is that the throughput of the entire system with free nodes is improved.

[0044] Next, details of the present embodiment will be described.

FIG. 3 is a diagram showing a system configuration example of the present embodiment. The job management server 100 is connected to six nodes 21 to 26 via the network 10. The nodes 21 to 26 are connected via a network 30 capable of high-speed communication. Network 30 is used to communicate information passed between parallel jobs executed on nodes 21-26.

In the system having such a configuration, the job processing requesting power of the job to be executed is first registered in the job management server 100. The registered job processing request is automatically sent at a predetermined time. There are batch job processing requests that are started dynamically and interactive job processing requests that are generated in response to user input.

[0046] Then, the job management server 100 performs job processing request distribution processing, and each job processing request is assigned to any of the nodes 21 to 26. The job management server 100 has the following hardware configuration to perform job processing request distribution processing.

FIG. 4 is a diagram illustrating a hardware configuration example of the job management server used in the present embodiment. The job management server 100 is entirely controlled by the CPU 101. A random access memory (RAM) 102, a hard disk drive (HDD) 103, a graphic processing device 104, an input interface 105, and a communication interface 106 are connected to the CPU 101 via a bus 107.

[0048] At least a part of an OS (Operating System) program application program to be executed by the CPU 101 is temporarily stored in the RAM 102. The RAM 102 stores various data necessary for processing by the CPU 101. The HDD 103 stores the OS and application programs.

A monitor 11 is connected to the graphic processing device 104. The graphic processing device 104 displays an image on the screen of the monitor 11 in accordance with a command from the CPU 101. A keyboard 12 and a mouse 13 are connected to the input counter face 105. The input interface 105 transmits a signal sent from the keyboard 12 or mouse 13 to the CPU 101 via the bus 107.

[0050] The communication interface 106 is connected to the network 10. The communication interface 106 transmits / receives data to / from other computers via the network 10.

[0051] With the hardware configuration described above, the processing functions of the present embodiment can be realized. Note that the power nodes 21 to 26 shown in FIG. 4 as an example of the hardware configuration of the job management server 100 can also be realized with the same hardware configuration. However, in this embodiment, each of the nodes 21 to 26 has two CPUs.

Next, processing functions necessary for the job management server 100 to distribute job processing requests will be described. FIG. 5 is a block diagram illustrating functions of the job management server. The job management server 100 includes a communication unit 111, a job input unit 112, a queue buffer 113, a reserved node number management unit 114, an empty node number management unit 115, an execution job remaining time management unit 116, and a job allocation unit 120. is doing.

[0053] The communication unit 111 communicates with the nodes 21 to 26, collects various management information, transmits a job processing request regarding the assigned job, and the like.

The job submission unit 112 submits a job processing request to the queue buffer 113. For example, the job submission unit 112 submits a job processing request for a batch job to the queue buffer 113 at a preset time. Further, when a job processing request is issued by an operation input from the user, the job input unit 112 inputs the job processing request of the corresponding job to the queue buffer 113.

The queue buffer 113 queues the submitted job processing requests in the order of submission.

Then, the queue buffer 113 passes the job processing requests to the job allocation unit 120 in order of the input time.

The reserved node number management unit 114 calculates the number of reserved nodes based on the parallel jobs registered in the queue buffer 113. The number of reserved nodes for parallel jobs on the queue buffer 113 is calculated as follows. The calculation process for the number of reserved nodes is divided into a parallel job selection process to be calculated and a parallel number calculation process based on the selected parallel job.

[0056] (1) Calculation target parallel job selection processing

As the method for selecting the target parallel job for calculating the number of reserved nodes, the following method is adopted.

[0057] In the first selection method, all queued parallel jobs are selected as processing targets.

In the second selection method, continuous parallel jobs are counted as processing targets by counting the head force of the queue. For example, if there is a sequential job next to a parallel job, the subsequent parallel jobs are excluded from the selection target.

[0058] In the third selection method, the total number of parallel jobs is calculated in the order of queued items. And the parallel number until the total value of the parallel number exceeds the preset maximum number Select jobs, and exclude parallel jobs after exceeding the maximum number. The maximum number is

For example, the value is half the number of nodes in the system (50 for a 100-node system)

) o

[0059] (2) Parallel number calculation processing

Calculate the total number of parallel jobs in the queue buffer 113 and use it as the number of reserved nodes (for example, if 2 parallels is 4, 4 parallels is 2 and 8 parallels is 1 then 2 X 4+ 4 X 2 + 8 X 1 =

twenty four). Or, the maximum number of parallel jobs that exist is the number of reserved nodes (for example, 4 for 2 parallels, 2 for 4 parallels, or 8 for 8 parallels).

The reserved node number management unit 114 passes the reserved node number calculated in this way to the job allocation unit 120.

The free node number management unit 115 acquires status information indicating whether or not the nodes 21 to 26 are processing jobs from the nodes 21 to 26 via the communication unit 111. Then, the free node number management unit 115 passes the number of free nodes to the job assignment unit 120.

The remaining execution job time management unit 116 calculates the remaining execution time of the job being processed for each of the nodes 21 to 26 via the communication unit 111. For example, there is a case where the maximum processing time of a job is specified in advance and the job is designed to always finish within the maximum processing time when a program for executing the job is created. In this case, the execution job remaining time management unit 116 sets a value obtained by subtracting the current execution time of each job from the maximum processing time as the remaining time of each job. The execution job remaining time management unit 116 executes the job on each of the nodes 21 to 26 and passes the remaining time of the job to the job allocation unit 120.

[0062] The job allocation unit 120 receives the job processing request queued in the queue buffer 113 based on the number of reserved nodes, the number of free nodes, and the remaining time of the execution job of each of the nodes 21 to 26. Sort to 21-26.

Hereinafter, the function of the job assignment unit 120 will be described in detail.

FIG. 6 is a block diagram illustrating functions of the job assignment unit. As shown in FIG. 6, the job assignment unit 120 includes a job queue acquisition unit 121, a job type determination unit 122, a parallel job assignment unit 123, a job assignment method selection unit 124, a stuffing method job assignment unit 125, and a distributed arrangement method job. An allocation unit 126 and a job processing request transmission unit 127; The job queue acquisition unit 121 acquires the first job queue from the queue buffer 113 at the job distribution timing. Here, the job distribution timing is when a job queue is queued in the empty queue buffer 113 or when any of the jobs being executed in the nodes 21 to 26 is completed. The job queue acquisition unit 121 passes the acquired job queue to the job type determination unit 122.

The job type determination unit 122 analyzes the content of the job processing request indicated by the job queue and determines whether the job is a sequential job or a parallel job. The job type determination unit 122 determines the number of parallel jobs if the job is a parallel job. In the case of a parallel job, the job type determination unit 122 passes the job processing request indicated by the job queue to the parallel job assignment unit 123 and notifies the parallel number. In the case of a sequential job, the job type determination unit 122 passes a job processing request indicated by the job queue to the job allocation method selection unit 124.

When receiving the job processing request from the job type determination unit 122, the parallel job allocation unit 123 selects a node corresponding to the number of parallel jobs in parallel as a free node, and determines it as an allocation destination.

[0067] When the job allocation method selection unit 124 receives the sequential job processing request from the job type determination unit 122, the job allocation method selection unit 124 determines whether the job queue stored in the queue buffer 113 or the job processing status by the nodes 21 to 26 is used. Select either the filling method or the distributed placement method as the job assignment method. Specifically, if the number of free nodes is larger than the number of reserved nodes, the job allocation method selection unit 124 selects the distributed arrangement method as the allocation method. Further, if the number of free nodes is equal to or less than the number of reserved nodes, the job allocation method selection unit 124 selects the stuffing method as the allocation method.

When the stuffing method is selected as the assignment method, the job assignment method selection unit 124 passes the acquired job processing request indicated by the job queue to the stuffing method job assignment unit 125. When the distributed allocation method is selected as the allocation method, the job allocation method selection unit 124 passes the acquired job processing request indicated by the job queue to the distributed allocation method job allocation unit 126.

[0069] Upon receiving the job processing request, the stuffing method job allocation unit 125 receives the current node number. Based on the processing status of jobs 21 to 26, the job assignment destination is determined by the stuffing method. Specifically, the stuffing method job allocation unit 125 detects a node that is executing a job and that can execute an additional job. Further, the stuffing method job allocation unit 125 acquires the remaining job execution time of the detected node from the execution job remaining time management unit 116, and determines the node having the shortest remaining time as the allocation destination. If there is no node that can execute the additional job among the nodes that are executing the job, the stuffing method job allocation unit 125 determines an empty node as an allocation destination. Then, the stuffing method job allocation unit 125 notifies the determined allocation destination to the job processing request transmission unit 127.

When receiving the job processing request, the distributed allocation method job allocation unit 126 determines a job allocation destination by the distributed allocation method based on the job processing status of the current nodes 21 to 26. Specifically, the distributed placement method job allocation unit 126 detects a free node. Then, the distributed placement method job allocation unit 126 determines one node from among the free nodes as an allocation destination. Then, the distributed allocation method job allocation unit 126 notifies the job processing request transmission unit 127 of the determined allocation destination.

The job processing request transmission unit 127 transmits a job processing request to the determined assignment destination node.

FIG. 7 is a flowchart showing a procedure of job assignment processing. In the following, the process shown in FIG. 7 will be described in order of step number.

[Step S 11] The job queue acquisition unit 121 acquires the first job queue from the queue buffer 113.

[Step S12] The job type determination unit 122 analyzes the content of the job processing request indicated by the acquired job queue, and determines whether the job is a parallel job or a sequential job. If it is a parallel job, the process proceeds to step S13. If the job is a sequential job, the process proceeds to step S14.

[Step S 13] The parallel job allocation unit 123 selects the same number of free nodes as the number of parallel jobs for which a processing request has been issued, and determines it as an allocation destination. After that, the processing power is advanced S22. [Step S 14] The job allocation method selection unit 124 obtains the number of free nodes among the nodes 21 to 26 from the free node number management unit 115.

[Step S15] The job allocation method selection unit 124 determines whether or not there is a free node among the nodes 21 to 26. If there is an empty node, the process proceeds to step S16. If there is no free node, the job assignment method selection unit 124 determines the job assignment method as the clogging method, and advances the process to step S20.

[Step S 16] The job allocation method selection unit 124 acquires the presence / absence of a parallel job and the number of reserved nodes from the reserved node number management unit 114.

[Step S 17] The job allocation method selection unit 124 determines whether or not a parallel job queue is queued in the queue buffer 113. If there is a parallel job, the process proceeds to step S18. If there is no parallel job, the job assignment method selection unit 124 selects the distributed arrangement method as the job assignment method, and advances the process to step S19.

[Step S 18] The job allocation method selection unit 124 determines whether or not the number of free nodes exceeds the number of reserved nodes. If the number of free nodes exceeds the number of reserved nodes, the job allocation method selection unit 124 selects the distributed arrangement method as the job allocation method, and advances the process to step S19. If the number of free nodes is equal to or less than the number of reserved nodes, the job allocation method selection unit 124 selects the stuffing method as the job allocation method, and the process proceeds to step S20.

[Step S 19] The distributed allocation method job allocation unit 126 allocates a job to an empty node. Thereafter, the process proceeds to step S22.

[Step S20] The filling method job allocation unit 125 acquires the remaining time of the execution job of each node from the execution job remaining time management unit 116.

[Step S21] The stuffing method job allocation unit 125 selects the node with the shortest remaining time as the job allocation destination and determines it as the allocation destination.

[Step S22] The job processing request sending unit 127 sends a job to a node determined as an assignment destination by the parallel job assignment unit 123, the stuffing method job assignment unit 125, or the distributed placement method job assignment unit 126. Send processing request. In this way, the filling method and the distributed arrangement method can be switched according to the number of reserved nodes and the number of free nodes. This improves the sweeping of jobs in the queue buffer 113 and improves the overall system throughput.

Next, a specific example of job assignment will be described.

FIG. 8 is a diagram illustrating an example of a queued job. In this example, the queue buffer

Eight job queues 41 to 48 are queued in 113. In FIG. 8, the job queues 41 to 48 indicate the queuing order by numerical values (# 1 to # 8).

[0081] The two job queues 41 and 42 from the top indicate sequential job processing requests. The next job queue 43 shows processing requests for five parallel jobs. The following four job queues 44 to 47 indicate sequential job processing requests. The last job queue 48 indicates processing requests for two parallel jobs.

In the following description, as the method for calculating the number of reserved nodes, it is assumed that the total number of parallel queued queues is used.

These jobs are processed by six nodes 21 to 26 each having two CPUs. Note that before the execution of each job shown in FIG. 8 is started, all the nodes 21 to 26 are assumed to be empty nodes.

First, the job queue 41 for sequential jobs is assigned. At this time, the number of reserved nodes is “7” (the parallel number “5” of the parallel job indicated by the job queue 43 + the parallel number “2” of the parallel job 48 indicated by the job queue 48). Since the number of free nodes is “6”, “the number of reserved nodes ≧ the number of free nodes” is satisfied, and the filling method is adopted. At this stage, since all the nodes 21 to 26 are empty nodes, the sequential job indicated by the job queue 41 is assigned to any of the nodes 21 to 26. In this example, it is assumed that the sequential job 41 is assigned to the node 21.

Next, the job queue 42 for sequential jobs is assigned. At this point, the number of reserved nodes is “7” and the number of free nodes is “5”. Therefore, “reserved node number ≧ free node number” is satisfied, and the filling method is adopted. In the case of the stuffing method, if an additional job can be assigned to a node that is already executing a job, the job is assigned to that node. Accordingly, the job queue 42 is assigned to the node 21. Next, the parallel job 43 is assigned. Since the parallel job 43 needs to be executed on each free node of the number of parallel jobs, it is assigned to the free node. Therefore, parallel job 43 is assigned to nodes 22-26.

FIG. 9 is a diagram showing the third job assignment status. In the node 21, the sequential job 41a indicated by the job queue 41 and the sequential job 42a indicated by the job queue 42 are executed. In addition, in the nodes 22 to 26, the parallel job 43a indicated by the job queue 43 is executed.

As described above, the parallel job 43a is executed, so that there is no empty node. Node 21 is also executing the same number of jobs as the CPU, so no additional jobs can be assigned. Therefore, the job queue 44 is in a waiting state until the processing of any job is completed.

Here, it is assumed that the parallel job 43a has been processed.

FIG. 10 is a diagram illustrating a state after the parallel job processing is completed. As shown in FIG. 10, when the parallel job 43a has been processed, the nodes 22 to 26 become free. Therefore, job assignment after job queue 44 is started.

The number of reserved nodes when assigning the job queue 44 for sequential jobs is “2”. Currently, since the number of free nodes is “5”, “the number of reserved nodes is equal to the number of free nodes”. Therefore, the job queues 44 to 46 for the sequential jobs are allocated by the distributed arrangement method. As a result, it is assumed that job queue 44 is assigned to node 22, job queue 45 is assigned to node 23, and job queue 46 is assigned to node 24.

FIG. 11 is a diagram showing a state in which up to the sixth job is assigned. In the node 22, a sequential job 44a indicated by the job queue 44 is executed. On the node 23, the sequential job 45a indicated by the job queue 45 is executed. In the node 24, the sequential job 46a indicated by the job queue 46 is executed. At this point, there are two free nodes.

Here, when the job queue 47 for sequential jobs is assigned, the number of reserved nodes is equal to the number of empty nodes, so the jobs are assigned using the stuffing method. That is, among the nodes 22 to 24, the job queue 47 is assigned to the node having the shortest remaining job execution time. In this example, it is assumed that job queue 47 is assigned to node 22. [0092] Finally, the job queue 48 for parallel jobs is assigned. At this point, since two free nodes are reserved, the job queue 48 is assigned to the nodes 25 and 26.

FIG. 12 shows a state in which all jobs are assigned. In the node 22, a sequential job 47a indicated by the job queue 47 is executed. On the nodes 25 and 26, the parallel job 48a indicated by the job queue 48 is executed. As shown in Fig. 12, all jobs are executed in this example. In addition, since there are no free nodes, the overall system can be operated with high efficiency.

In the following, as comparison targets, there are shown examples of a case where job assignment is performed only by the stuffing method and a case where job assignment is performed only by the distributed arrangement method.

FIG. 13 is a diagram showing an example in which jobs are assigned only by the stuffing method. When job assignment is performed using the stuffing method, the state in which the third job is assigned is the same as in FIG. Then, when each job after the fourth sequential job 44a is assigned by the stuffing method, the node 26 becomes an empty node as shown in FIG. In other words, the operating rate of the entire system is poor. However, sequential jobs are executed two by two in one node, and the execution performance is degraded due to resource competition.

Further, a case will be described in which the fourth and subsequent jobs are allocated by the distributed arrangement method from the state shown in FIG.

FIG. 14 is a diagram showing an example in which jobs are assigned only by the distributed arrangement method. In the case of the distributed placement method, if each job after the fourth sequential job 44a is assigned by the distributed placement method, when two parallel jobs 48a are assigned, there will be one free node. Therefore, the job queue 48 for parallel jobs is waiting to be executed until the number of free nodes increases. This makes sweeping parallel jobs worse. In addition, since the node 26 is left free, the operation rate of the entire system remains poor.

[0096] Comparing the job assignment results according to this embodiment shown in FIG. 12 with the job assignment results shown in FIG. 13 and FIG. 14, the end of the parallel job 43a is achieved by applying this embodiment. After that, all the remaining jobs are arranged without waiting time. However, in this embodiment, it can be seen that the availability of the system has improved due to the absence of free nodes, and the sweeping of parallel jobs has improved. In addition, multiple sequential jobs can be executed simultaneously. Only two nodes need to be executed, reducing the probability of resource contention and improving execution performance.

By the way, it is also possible to determine the job allocation method by adding a predetermined correction value ex to the number of reserved nodes in the above embodiment, and comparing the value after the addition with the number of free nodes. . Specifically, the number of reserved nodes and the number of free nodes based on the number of parallel jobs on the job queue are collected. If the number of free nodes is greater than the number of reserved nodes + α, the distributed placement method is selected. If is less than the number of reserved nodes + α, select the filling method. The correction value α represents the number of nodes reserved for the most recently submitted parallel job. For example, the power specified by the system administrator and the average number of parallel jobs submitted in the past are calculated by the job assignment unit 120, and the calculation result is used as a correction value.

In addition, the number of reserved nodes can be fixed to a preset value. For example, “number of parallel jobs that can be submitted” X “number of parallel jobs” is set as the number of reserved nodes. Here, the number of parallel jobs that can be submitted is a value arbitrarily set by the system administrator in advance. Also, the number of parallels may be a value specified by the system administrator in advance! /, Or the average number of parallels in the past! /.

[0099] Also, the number of sequential jobs being executed and the number of sequential jobs queued in the queue buffer. If the number of nodes that preferentially execute sequential jobs (the number of nodes for sequential jobs) is smaller, the job is distributed If so, you may be able to place jobs using the stuffing method. The number of sequential job nodes can be, for example, a value specified in advance by the system administrator, or the average number of sequential jobs per unit time in the past! /.

[0100] In addition, as a method of selecting a node to which jobs are sequentially assigned by the stuffing method, it may be assigned to the sword having the longest execution time of the job being executed among the assigned nodes. For example, if the maximum processing time at the job program design stage is not specified, the remaining time of the job being processed cannot be calculated. In this case, jobs are assigned sequentially to the node that has the longest execution time since job processing started.

[0101] The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the job management apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents is a computer-readable recording medium. Can be recorded on the body. Examples of the computer-readable recording medium include a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Magnetic recording devices include hard disk drives (HDD), flexible disks (FD), and magnetic tapes. Optical discs include DVD (Digital Versatile Disc), DVD—RAM (Random Access Memory), CD—ROM (Compact Disc Read Only Memory), and CD—R (Recordable) ZRW (Rewritable). Magneto-optical recording media include MO (Magneto-Optical disk).

[0102] When the program is distributed, for example, a portable recording medium such as a DVD or a CD-ROM in which the program is recorded is sold. It is also possible to store the program in a storage device of the server computer and transfer the program to other computers via the network.

[0103] A computer that executes a program stores, for example, a program recorded on a portable recording medium or a server computer-powered program in its own storage device. Then, the computer reads its own storage device power program and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. The computer can also execute processing according to the received program sequentially each time the program is transferred to the server computer.

[0104] The above merely illustrates the principle of the present invention. In addition, many variations and modifications are possible to those skilled in the art, and the invention is not limited to the precise configuration and application shown and described above, but all corresponding variations and equivalents are It is regarded as the scope of the present invention by the claims and their equivalents.

Explanation of symbols

[0105] 1 Job management device

la cuno fffa

lb Reserved node number determination means

lc Number of free nodes

Id job queue acquisition means le Job type judgment means

If parallel job assignment means lg assignment method selection means lh stuffing method job assignment means li distributed placement method job assignment means lj processing request sending means

2a, 2b, ... node

Claims

The scope of the claims

A job management device that assigns to a node a sequential job executed on one node and a parallel job executed in parallel processing by a plurality of nodes.

When a job queue indicating a job processing request is input, a queue buffer for storing the job queues in the input order;

Based on a job queue indicating a parallel job processing request stored in the queue buffer.

V, reserved node number determining means for determining the number of reserved nodes indicating the number of nodes to be reserved in advance for processing parallel jobs;

Free node number calculating means for monitoring the presence or absence of job execution in each of the plurality of nodes and calculating the number of free nodes indicating the number of nodes that should not execute the job;

Job queue acquisition means for acquiring the job queues stored in the queue buffer in the order of storage;

Job type determination means for determining whether the job queue acquired by the job queue acquisition means is a parallel job processing request or a sequential job processing request;

Parallel job assignment means for determining, as the assignment destination, an empty node corresponding to the number of parallel jobs when the job queue obtained by the job queue obtaining means is a parallel job processing request;

If the job queue acquired by the job queue acquisition means is a sequential job processing request, based on the number of free nodes and the number of reserved nodes, it is determined whether the number of free nodes is insufficient, An allocation method selection means for selecting a stuffing method as a job allocation method if the number of free nodes is insufficient, and a distributed arrangement method for selecting a job allocation method if the number of empty nodes is sufficient;

When the stuffing method is selected, a stuffing method job assigning unit that preferentially determines another node that has already executed another job as an assignment destination of the job acquired by the job queue obtaining unit;

When the distributed arrangement method is selected, a node that is not executing a job is preferentially determined as a job assignment destination acquired by the job queue acquisition means, and a distributed arrangement method job assignment unit; And a processing request transmission unit configured to transmit a processing request for the job indicated by the job queue acquired by the job queue acquisition unit to a node determined as an allocation destination.

[2] The reserved node number determining means uses the total value of the parallel numbers indicated in the parallel job processing request stored in the queue buffer as the reserved node number. The job management apparatus described.

[3] The reserved node number determination means uses the maximum value of the parallel number indicated by the parallel job processing request stored in the queue buffer as the reserved node number. The job management apparatus described.

[4] The allocation method selection means determines that the number of free nodes is insufficient if the number of free nodes is equal to or less than the number of reserved nodes, and the number of free nodes exceeds the number of reserved nodes, The job management apparatus according to claim 1, wherein it is determined that the number of empty nodes is sufficient.

[5] If the number of free nodes is equal to or less than a value obtained by adding a predetermined value to the number of reserved nodes, the allocation method selection unit determines that the number of free nodes is insufficient, and the number of free nodes is 2. The job management apparatus according to claim 1, wherein if the number of reserved nodes exceeds a value obtained by adding the predetermined value, it is determined that the number of empty nodes is sufficient.

[6] The stuffing method job allocation means determines, as an allocation destination, a sword with the shortest remaining time of a job being executed among nodes that have already executed the job. The job management apparatus described.

[7] The stuffing method job allocation means determines, as an allocation destination, the longest sword of the elapsed time of the job execution start power among the nodes that have already executed the job. The job management apparatus described.

[8] In a job management program that assigns to a node a sequential job executed on one node and a parallel job executed by parallel processing by a plurality of nodes.

Computer

When a job queue indicating a job processing request is input, the job queues are input in the input order. Queue buffer to store users,

Based on the job queue indicating the parallel job processing request stored in the queue buffer, V determines the number of reserved nodes indicating the number of nodes to be reserved in advance for processing the parallel job. Reservation node number determination means,

A number of empty nodes calculating means for monitoring the presence or absence of execution of the job in each of the plurality of nodes and calculating the number of empty nodes indicating the number of nodes that should not execute the job;

Parallel job assignment means for determining, as the assignment destination, an empty node corresponding to the parallel number of parallel jobs when the job queue obtained by the job queue obtaining means is a parallel job processing request;

If the job queue acquired by the job queue acquisition means is a sequential job processing request, based on the number of free nodes and the number of reserved nodes, it is determined whether the number of free nodes is insufficient, An allocation method selection means for selecting a stuffing method as a job allocation method if the number of free nodes is insufficient, and a distributed arrangement method for selecting a job allocation method if the number of free nodes is sufficient;

When the distributed arrangement method is selected, a node that is not executing a job is preferentially determined as an assignment destination of the job acquired by the job queue acquisition unit,

A job management program for causing a node determined as an assignment destination to function as a processing request transmission unit that transmits a processing request for a job indicated by the job queue acquired by the job queue acquisition unit.

Parallel jobs executed in parallel processing by multiple jobs with sequential jobs executed on one node In a job management method in which assignment of queue jobs to nodes is performed by a computer, when job processing requests are input, queued jobs are queued in the order in which they are input, and parallel job processing requests stored in the queue buffer Based on the job queue indicating V, the number of reserved nodes indicating the number of nodes to be reserved in advance for performing parallel job processing is determined.

Monitor the presence or absence of job execution on each of the plurality of nodes, calculate the number of free nodes indicating the number of nodes that should not execute the job,

The job queues stored in the queue buffer are acquired in the order in which they are stored, and it is determined whether the acquired job queue is a parallel job processing request power or a sequential job processing request,

When the acquired job queue is a parallel job processing request, an empty node corresponding to the parallel number of parallel jobs is determined as an assignment destination,

When the acquired job queue is a sequential job processing request, it is determined whether the number of free nodes is excessive or insufficient based on the number of free nodes and the number of reserved nodes, and if the number of free nodes is insufficient, If the stuffing method is selected as the job assignment method and the number of empty nodes is sufficient, the distributed placement method is selected as the job assignment method. If the stuffing method is selected, another job has already been executed. Node is preferentially determined as an assignment destination of the job acquired by the job queue acquisition means, and when the distributed arrangement method is selected, the node that is not executing the job is preferentially acquired the job queue. Determined as the job assignment destination obtained by

A processing request for the job indicated by the job queue acquired by the job queue acquisition unit is transmitted to the node determined as the allocation destination;

A job management method characterized by the above.