JPH09311795A - Scheduling method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically acquire a system resource quantity needed to execute individual jobs by updating the difference between a measured value of the resource consumption of a computer and an estimated value and error in the resource demand quantity of a job being executed. SOLUTION: At each resource, job information 32 in execution and job resource information 21 are totalized to find the estimated value of the resource consumption of the whole computer (step 31), and the found estimated value of the resource consumption is compared with the measured value of the consumption of the resource information 22 of the computer (step 32). When an error in the resource consumption of the whole computer is exceeded, the resource consumption is corrected according to the rate of the resource demand quantity and error (step 33), and the job resource information in execution and further the difference between the measured value of the resource consumption of the computer and the estimated value of the resource consumption and the error in the demand quantity of the job resource information 21 in execution are updated (step 34). Therefore, the system resource quantity needed to execute individual jobs can automatically be acquired.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a CPU that is repeatedly executed and a CPU load or I
The present invention relates to a scheduler that estimates a load required by a job from an O load or the like, uses the estimated value to predict a system state after the job is submitted, and determines a computer to submit the job.

[0002]

[Prior Art] "PVM Guide" (http: //csep1.phy.ornl.g
According to (ov / pvm_guide / pvm_guide.html), load balancing is a means for improving performance in a distributed computing environment. The load on each computer can be averaged by measuring the number of jobs waiting to be executed on each computer and executing a new job on the computer having the smallest number of jobs waiting to be executed.

In addition, Greoge A. Champine's "Distrib
uted Computer Systems: Impact onManagement, Design,
According to "and Analysis", there is a method of measuring the number of execution steps of a job using an emulator etc. as a method of obtaining the load given to the system by an individual job. With this method, since the job operation is known in detail, the quality / amount / time of the load on the resource can be obtained almost accurately. However, since one instruction is analyzed / executed by the emulator, it takes a very long time to measure the number of execution steps of a job. Also, if the job is developed by another company, the measurement of the number of execution steps may not be permitted in the usage contract.

As another method for obtaining the load applied to the system by an individual job, there is a method in which only the job is operated on the computer and the load on the computer is measured. With this method, the load value can be measured in a short time. However, in order to eliminate the effects other than the target job, it is necessary to prepare a computer system exclusively for measurement.

[0005]

In the load distribution schedule according to the load state of the system, a job that takes a long time may be assigned to the same computer, and as a result, the overall processing time may increase.

In the method of measuring the number of execution steps of the job in order to obtain the execution load of the individual job, the simulator is usually used to measure the number of execution steps of the job, so that it takes a very long time. . In addition, when the job is developed by another company, the measurement of the number of execution steps may not be permitted under the usage contract.

In the method of measuring the load by moving only the job on a computer in order to obtain the execution load of the individual job, it is necessary to prepare a distributed system exclusively for the measurement, which causes a problem of cost increase. .

An object of the present invention is to automatically acquire the system resource amount necessary for executing the job for an individual job, and to allocate the job to a computer based on the resource amount.

[0009]

[Means for solving the problem] Corresponding to the job name,
Estimated values such as the CPU and IO load used by the job and the time required for the job, and information with their accuracy are provided.

A value such as CPU / IO capability is measured for each distributed computer.

In the case of an unknown job, the load estimated value and the accuracy are set to predetermined default values.

The load variation of the computer after the job is executed is registered as the estimated load of the job.

If the difference between the total load estimated value of the jobs being executed during the job execution and the actually measured load of the computer is within the range of the allowable error obtained from the accuracy, the accuracy is increased.

During execution of a job, when the difference between the total load estimation value of the job being executed and the actually measured load of the computer exceeds the allowable error range obtained from the accuracy, it corresponds to the ratio of the estimated error. Then, in proportion to the size of the estimated load, the difference is distributed to correct the estimated load value and the accuracy is lowered.

By these steps, the load applied to the computer system by the individual job is automatically acquired.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below.

FIG. 1 shows an outline of a distributed computer system (1) to which the schedule system according to the present invention is applied. This distributed computer system (1) includes a management computer (11), a plurality of computing servers (12), and a network (13) connecting them. The management computer (11) may also serve as one of the calculation servers (12).

The management computer (11) uses the job resource information (21)
And the job waiting for startup requested by the computer system
Job reception queue (111) that manages (112) and scheduler that allocates the job to one computing server (12)
(113) is provided.

The computing server (12) has a plurality of resources (121) such as CPU time, memory capacity, file access frequency and network access frequency, and computer resource information (22) for holding resource capacity and usage amount. Jobs that use resources (12
3), running job information (23) for managing jobs, resource usage measuring means (122) for checking the status of resources (121) of the computing server (12), and job information acquisition means (124) .

The job information acquisition means (124) uses the resource usage measuring means (122) to calculate the resource usage of the computing server (12),
The resource (121) used by the running job (123) is estimated from the running job information (23), and the job resource information (12) storing the amount of the resource used by the job is updated.

When there is a job execution request, the execution waiting job (112) is put in the execution request reception queue (111). The scheduler (113) takes out the pending job (112) from the execution request reception queue (111), acquires the resource amount information used by the job from the job resource information (21), and calculates each server (1).
The state of the resource is acquired from the resource usage measuring means (122) of 2), and the waiting job (112) is allocated to an appropriate calculation server (12).

FIG. 2 shows the structure of each table.

There is one job resource information (21) on the distributed computer system, and the job name (211), each resource name (212) consumed when the job is executed, and the resource are used by the job. It has a resource requirement (213) and an accuracy (214) showing its certainty. Processing time from job startup to termination as resources, CPU used per unit time
Time, amount of memory occupied, file access frequency, network access frequency, etc. This table is added / changed by the job information acquisition means (124) by the method described later.

The computer resource information (22) is stored in each computing server, and for each resource that can be provided by the computer, its resource name (221) and resource capacity (22
2), a usage limit (223) that represents the usage rate at which the usage of the resource remarkably deteriorates, and a resource usage amount (224) that is a measurement value of the load on the resource for the calculation server. Items other than the resource usage amount (224) in this table are assumed to be prefilled by other means. The resource usage amount (224) is regularly updated by the resource usage amount measuring means (122).

The running job information (23) includes a job name (231) and a job ID (232) for the running job (123).
Is provided. For a newly started job, an entry for the job is added, and for a finished job, the entry for the finished job is deleted.

A job information acquisition method and a scheduling method in the distributed computer system having the above configuration will be described below.

FIG. 3 shows the details of the job information acquisition method in the job information acquisition means.

The following processing is performed periodically.

In step 31, for each resource, from the entry that matches the respective job name (231) of the running job information (23) and the job name (211) of the job resource information (21), Read resource demand (213).

When a plurality of jobs are running on the job computer that is being executed, the jobs compete with each other for their respective resources, so the job utilization rate decreases.

Estimated usage amount LAms of resource s of computer m
Uses the running job p, its resource demand estimate LRps, and the resource capacity LCms of the computer m,

[0032]

[Equation 1]

Then, the operation rate Amp of the job p on the computer m
Using the capacity LCms of resource s and the resource demand LRps of job p,

[0034]

[Equation 2]

Calculated by

Next, in step 32, computer resource information (2
Find the difference between the measured resource usage (224) in 2) and the estimated value LAms of the usage. This difference is called the actual difference for the sake of explanation below. Also, the allowable error obtained from the resource demand (213) and the accuracy (214) is calculated.

The allowable error Eps is the required resource amount LR of the job p.
Using ps and its accuracy Rps,

[0038]

(Equation 3)

It is assumed that

When the actual estimation difference is larger than the allowable error, in step 33, the actual measurement value (224) of the job and the estimated usage amount LAms are calculated.
The actual inference is distributed to the resource requirements of the job being executed so that and match. At this time, the number of jobs with large resource requirements changes, and the number of jobs with low accuracy changes more.

The required resource amount LRps of the job p is determined by its accuracy R
ps, measured value LSms on computer m, estimated value LAms, constant c
Using,

[0042]

(Equation 4)

Update with.

At step 34, the accuracy of the job resource requirement is changed from the actual inference, the job resource requirement (213) and its accuracy (214).

The accuracy Rps of the resource requirement for the resource s of the job p is calculated by using the resource usage measurement value LSms on the computer m, the resource usage estimation value LAms, the allowable error Eps, and the constant c.

[0046]

(Equation 5)

Update with.

The above processing is repeated for each resource of each calculation server (12).

Next, FIG. 4 shows details of the job activation assigning means of the scheduler.

When there is a job waiting to be executed in the job execution request queue (111), the following processing is performed.

In step 41, the scheduled job is acquired from the job execution request queue (111) and the job resource information (2
From 1), for each resource of the entry that matches the job name (211), obtain the resource requirement (213) and its accuracy (214).

In step 42, the measured resource usage (224) is acquired from the computer resource information (22), and step 41
The resource usage amount after the job is submitted is estimated by using the resource requirement amount (213) and its accuracy (214) obtained in step 2.

In step 43, the resource capacity of each computer (22
From 2) and the estimated resource usage after the job is submitted,
Obtain the estimated resource utilization and compare it to the usage limit (223).

When there is a calculation server (12) whose estimated resource usage rate is less than or equal to the usage limit (223), in step 44,
The job is assigned to the calculation server with the lowest resource usage rate.

That is, this computer m, at the time of submitting the job p, regarding the resource s, the measured value LSms before the submission, the resource requirement amount LRps of the job to be submitted, the resource capacity LCms, and the resource use limit Bms,

[0056]

(Equation 6)

The condition of is satisfied.

When there is no calculation server (12) satisfying the above conditions, in step 45, the job is not started and is held in the execution waiting queue (111), and after a certain period of time, the process is restarted from step 41.

As another scheduling method, in step 42, from the running job information (23),
Obtain a list of running jobs, find the resource requirements for each of them, and check the resource requirements (21
Estimate resource requirements after job submission by summing with 3).

That is, the computer m selected in step 44, when submitting the job p, the measured value LSms before the submission of the resource s, the resource requirement amount LRps of the submitted job,
In the resource capacity LCms and the resource usage limit Bms,

[0061]

(Equation 7)

The condition of is satisfied.

As another scheduling method, in step 44, the computer is assigned to the computer having the lowest total resource usage rate after the job is submitted among all the machines.

That is, this computer m, at the time of submitting the job p, regarding the resource s, the measured value LSms before the submission, the resource demand amount LRps of the job to be submitted, the resource capacity LCms, and the resource usage limit Bms,

[0065]

(Equation 8)

The condition of is satisfied.

As another scheduling method, in step 44, the computer is assigned to the computer having the lowest maximum resource usage rate after the job is submitted among all the machines.

That is, this computer m, at the time of submitting the job p, for the resource s, the actual measured value LSms before submission, the resource requirement amount LRps of the job to be submitted, the resource capacity LCms, and the resource use limit Bms,

[0069]

[Equation 9]

The condition of is satisfied.

While the amount of resources required by a job is uncertain, the effect of prediction is small, and it is no different from scheduling with only the current computer load.

The system resource amount required by the repeatedly executed job is automatically acquired. Once the amount of resources required by the job is known, it is possible to schedule the resource usage of the computer after the job is not concentrated. For example, jobs that apply a CPU load but no IO load and jobs that do not put a CPU load but an IO load are scheduled in combination, which reduces competition for resources between jobs on the computer. You can use the calculator efficiently.

[0073]

According to the present invention, it is possible to automatically acquire the amount of plural system resources required by an individual job. With this information,
It is possible to prevent the processing efficiency from being lowered due to the processing being concentrated on a specific system resource.

The processing time of an individual job can be estimated.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a distributed computer system according to an embodiment of the present invention.

FIG. 2 is a block diagram of a table.

FIG. 3 is a block diagram of a method for acquiring a resource requirement amount of an individual job.

FIG. 4 is a block diagram of a job activation computer allocation scheduling method.

[Explanation of symbols]

LRps ... Estimated amount of resources s required for job p per unit time, LRms ... Resources s required for job per unit time on computer m
Estimated amount of resource s, LAps ... Estimated amount of resource s used by job p per unit time, LAms ... Estimated amount of resource s used by computer m per unit time, LSms ... Calculator m per unit time Measured value of the amount of resource s used, LCm… capacity of resource s that can be used per unit time of computer m, Eps… LAps error, ERps… LAps error rate, = E
ps / LAps [%], Rps… LAps accuracy, = (LAps-Eps) / LAps, Amp… Percentage of operation of job p on computer m, Bms… Resource usage limit.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shinji Morita 5030 Totsuka-cho, Totsuka-ku, Yokohama-shi, Kanagawa Stock company Hitachi Software Development Division

Claims (6)

[Claims] 1. A computer system comprising resources provided by a computer system, means for measuring resource usage in the entire computer system, jobs being executed on the computer, and list information of jobs being executed. A job resource request amount acquisition method, comprising job resource information including a name of the job, an estimated value of the amount requested by the job for each resource of the computer, and a certainty indicating the certainty thereof. ) For the job being executed on the computer, add up the error amount of the resource request amount for the job by summing the resource amount requested by the job in the job resource information and obtaining the estimated value of the resource usage amount of the entire computer. (B) The difference between the measured value of the resource usage of the computer and the estimated value of the calculated resource usage is the error of the resource usage of the entire computer. When the value is large, (c) For the job being executed on the computer, the job management information of the job being executed should be adjusted according to the resource request amount and its error rate so that there is no difference between the measured value and the estimated value of the resource usage amount. In addition to the estimated resource requirement, (d) the difference between the measured value and the estimated value obtained in step (b),
A scheduling method characterized by updating the error in the resource demand of the job being executed. 2. The computer system according to claim 1, further comprising information on resource capacity of the computer, wherein in step (a), (a1) for the job being executed and each resource, the resource estimation request amount of the job. (A2) Calculate the job utilization rate from the total estimated resource demand, the computer resource capacity, and the estimated resource demand of each job. (A3) Calculate the job utilization rate from each job A scheduling method, which is characterized by outputting the estimated resource usage amount and the estimated processing time. 3. The scheduling method according to claim 1, further comprising a step of a scheduler detecting a start / end of a job, and executing the steps when the scheduler detects the start / end of the job. 4. When a job is submitted in a distributed computer environment, (e) resource estimated usage amount information of the job is obtained based on the resource usage amount of the job obtained in claim 1, and (f) the job The estimated resource usage of the above is added to the resource usage of each computer to obtain the resource usage after the job is submitted. (G) The ratio of the resource usage to the resource capacity is calculated for each resource, and (h) ) A scheduling method, characterized in that the job is assigned to a computer having the smallest sum. 5. Instead of step (f) of claim 1, (f ') job being executed and the estimated resource usage of the job to be loaded are read out, and the estimated resource usage of the job being executed and the job being submitted is used. A scheduling method characterized by obtaining the total amount of resources and using this as the resource usage amount. 6. Instead of step (f) of claim 1, (f ") the product of the estimated resource usage of the job and the ratio of the error range is added to the resource usage of each computer, and A scheduling method characterized in that the amount of resource usage after the input is obtained.