JP4219119B2 - How to determine job execution order - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a job execution order determination method in a computer system capable of operating a plurality of jobs in parallel, and more particularly to a job execution order determination method that is effective when shifting to a computer system having a different configuration.
[0002]
[Prior art]
Conventionally, in online processing for large-scale banking systems and online processing for managing merchandise in a distribution business, many operations have been performed in order to improve customer service. Online processing is accompanied by batch data processing (batch processing) for aggregating data handled in the business. As the amount of online processing increases and the types of online services diversify, the amount of batch processing increases and the processing content is becoming more complex.
In general, a job that performs online processing is referred to as an online job, and a job that performs batch processing is referred to as a batch job. In order to cope with these increasing tasks, a computer system having a plurality of CPUs is constructed, and batch operations tend to be performed on the system.
As a system construction technique having a plurality of CPUs, there is a system (tightly coupled multiprocessor system) in which a plurality of CPUs share one main storage device and operate with one operating system (OS).
As another system construction method, there is a system (loosely coupled multiprocessor system) in which a plurality of computers having independent CPUs and main storage devices are coupled by an inter-channel coupling device or the like.
[0003]
In the systems based on both of the construction methods described above, it is necessary to organize an efficient job schedule so that batch processing is completed in a short time. In particular, in a loosely coupled multiprocessor system, a plurality of independent OSs operate, and therefore, it is necessary to perform work while mutually coordinating between the OSs. Therefore, as conventional techniques for efficiently performing batch processing, there are those disclosed in Japanese Patent Laid-Open Nos. H5-173807 and H6-68052. These two cases relate to a job distribution method in which resource utilization is dynamically measured and scheduling is performed on a loosely coupled multiprocessor system so that the load is equalized.
That is, the one described in the above-mentioned Japanese Patent Application Laid-Open No. 5-173807 is provided with a resource load measuring means for measuring the resource load amount of the system at regular intervals, and the job execution multiplicity is dynamically changed according to the measurement result there. To change. Further, what is described in JP-A-6-68052 relates to a function of automatically providing a job to a processor having the lowest load by providing a processor usage rate monitoring unit that measures the usage rate of a processor at regular time intervals. Is.
[0004]
[Problems to be solved by the invention]
In recent years, the batch processing has become more complicated with the diversification of business and the increase in data volume, as described above. As a method for shortening the processing time of the job, means for increasing the CPU performance of the computer or highly multiplexing the entire job can be considered. However, there is almost a limit in the current technology for improving the former single CPU performance, and it is certain that high multiplexing of the latter job will become essential in the future.
However, in order to achieve high multiplexing of batch jobs without breaking the correlation between these complex batch jobs, the computer resources of the multiprocessor system must be used effectively unless job scheduling is efficient. It will not be.
The prior art described in the above two publications dynamically determines job scheduling during job execution on an already constructed system. This is to improve the efficiency of scheduling for a single job when the job is executed, and does not improve the efficiency of scheduling by measuring the entire job. Further, these conventional techniques are applied when a job is executed in a system that has already been constructed, and could not be used for prior job scheduling evaluation when changing the system configuration.
[0005]
While online jobs are time-dependent jobs, batch jobs that are batch data processing have the property that they do not depend on the start time for performing certain processing on a fixed input file. With respect to a job in which such a plurality of jobs are periodically repeated, it is possible to predict the subsequent job execution based on the past job execution history. For a job having such a property, the overall processing time can be shortened by performing a job scheduling plan before it is executed. Also, by performing the job scheduling plan as described above at the time of system design, it is possible to make a plan for file allocation to the disk device etc., and to perform job scheduling evaluation in advance when changing the system configuration Can do.
[0006]
  The present invention is aimed at improving the efficiency of batch job scheduling and has the following three objects.
  The first object of the present invention is to improve efficiency based on past execution history before executing batch processing.TargetIt is an object of the present invention to provide a job execution order determination method capable of supporting the creation of a simple job schedule.
  A second object of the present invention is a job execution order determination method capable of estimating the effect before a change based on a past execution history when there is a change in system operation such as job multiplicity. Is to provide.
  The third object of the present invention is to reduce the frequency of access to disks connected to other computers in the system and to reduce the overhead related to data transfer when distributing jobs on a loosely coupled multiprocessor system. Therefore, the computer that executes each job is based on past history information so that the file accessed by the distributed job can be placed on a disk connected to one computer in the system. It is an object of the present invention to provide a job execution order determination method that can be determined as follows.
[0007]
[Means for Solving the Problems]
  In order to achieve the above object, the present invention provides a computer.The processor provided inIn one systemWhen multiple jobs run in parallel,A method for determining an execution order between jobs that need to refer to a predetermined file,The processorOutsideFromFor each job, the history information of the file accessed during job execution is displayed.ReceiveHistory informationIs stored in a memory, the history information is read from the memory, and the information is written in a corresponding job information cell having a table structure included in the history information. The parameter information including the CPU performance ratio is read into the memory, the parameter information is read from the memory, and the parameter information is analyzed to create simulation condition definition information as a simulation condition. The job information cells are sequentially read from the list of job information cells, and it is determined whether there is a job information cell connected to the space generated based on the simulation condition definition information, and is generated based on the simulation condition definition information. It is determined that there is a job information cell connected to the If the file accessed by the job in the cell information cell is accessed by another job, obtain the job scheduling condition that starts the execution of the job after the other job ends. A time at which execution can be started is calculated, whether or not there is a vacancy at the corresponding time in the designated space, and if there is a vacancy, the memory is accessed and the pointer is written to thereby store the job information cell in the space. After the connection of the job information cell is completed, the data in the job information cell such as the job execution time is rewritten to determine which job the job information cell is executed after.Characterized by decidingYes.
[0008]
[Action]
According to the present invention, it is possible to determine the job execution start time based on the current job execution by determining the execution start condition of each job from the collected access history information and determining the execution time of each job from the execution history information. . Then, by performing a simulation based on the first condition that the job multiplicity in the computer system to be simulated is the maximum number of jobs that can be operated in parallel and the second condition that the execution start condition is followed. Virtual job execution that does not depend on the job scheduler can be reproduced. From this simulation result, it is possible to present a job schedule proposal that can reduce the processing time.
[0009]
When a change in system operation such as job multiplicity increase / decrease is given as a parameter, the job schedule result when the operation change is made can be provided by adding the parameter to the execution start condition. As a result, it is possible to reconstruct the system after obtaining an estimate for solving the problem in the current system.
Further, by performing job grouping based on a file to be accessed in advance before simulation, a job schedule result under a new system condition given as a parameter can be provided. As a result, it is possible to provide a job schedule proposal on a system having a configuration different from the current system such as a loosely coupled multiprocessor system.
As described above, the static evaluation using the execution history enables the present invention to obtain a quantitative evaluation regarding the job schedule without using an actual machine, that is, without placing a burden on the target system.
[0010]
【Example】
The present embodiment is intended for a computer system including one or more CPUs and one or more computers having disks accessed from the CPUs. When the computer system is composed of a plurality of computers, each computer has means for communicating with each other.
In the above computer system, a plurality of jobs are executed in parallel. Here, there is an upper limit to the number of jobs executed in parallel, and the upper limit is called job multiplicity. A job is composed of one or more job steps. Here, the job step corresponds to one of the programs executed in the job. A file in which operations such as read, write, and reference are performed is referred to as an access file in this case.
A batch job can be logically executed if a file to be referenced is generated. Therefore, when a file accessed by a certain job is accessed by another job, it is necessary to start execution after the other job ends. The job that should be started after execution of the job is called a scheduling condition.
[0011]
  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
  FIG. 1 shows a basic system configuration of an embodiment of the present invention. In this embodiment, the computer 101 is composed of a CPU 102 and a memory 103. On the memory 103, a job scheduling analysis unit 104 that performs job scheduling analysis by inputting parameter information 114 and a job execution history 115 described later, a system operation history collection unit 110 that collects system operation history, and a job schedule program 111 In addition, the job schedule program 111 and the job activation unit 112 and registrationBookThe creation unit 113 is configured. It is assumed that the processing in each of these units is executed by the CPU 102.
[0012]
The job registered in the computer 101 is registered in the job registration list 117 by the registration list creation unit 113 in the job schedule program 111. The job activation unit 112 activates a job and puts it into an execution state based on the job registry 117 created by the above procedure. In order to start up, it is necessary that a job control statement 118 that describes which program is to be executed and which file to access is prepared for each job. The job activation unit 112 activates the job after confirming whether resources such as a file necessary for executing the job are secured based on the job control statement 118.
[0013]
The job enters the execution state according to the above procedure. The system operation history collection unit 110 collects the execution status and creates a job execution history 115. A configuration example of the job execution history 115 is shown in FIG.
The execution history data to be collected includes a job start record 115-1, a job step start record 115-2, a file close record 115-3 for the access file, a job step end record 115-4, and a job end as shown in FIG. For example, record 115-5. Each of these records is generated by an event such as job execution start or file close, and the record type that indicates what event caused the record is included at the beginning or inside of those records. Yes. In this embodiment, a description regarding a record type is 01 at the job start record, 02 at the job step start record, 03 at the file close record, 04 at the job step end record, and 05 at the job end record. Shall.
[0014]
  Each record that is the job execution history data shown in FIG. 2 will be described in more detail.
  The job start record 115-1 has a job name, a job start date, and a job start time in addition to the record type 01. The job step start record 115-2 has a record type02In addition, the job step name, job step start date, job step start time, job name that issued this record, and the file close record 115-3 related to the access file include the file name, File close date, file close time, job step name that issued this record, job name that issued this record, and job step end record 115-4 include job step name, job step in addition to record type 04 It has an end date, job step end time, and the name of the job that issued this record. The job end record 115-5 has a job name, job end date, job end time, and CPU usage time in addition to the record type 05. is doing.
[0015]
The structure of the parameter information 114 that is input to the job scheduling analysis unit 104 together with the job execution history 115 is shown in FIG. The parameter information 114 includes the number of spaces (maximum job execution multiplicity) and CPU performance ratio (CPU performance in the new system) other than the job execution history 115 shown in detail in FIG. All the information such as the performance improvement ratio and output result designation are included. FIG. 3 shows a case where the number of spaces is 30, the CPU performance ratio is 1 (in the case of CPUs having the same performance), the grouping is ON (when the grouping function is enabled), and so on.
[0016]
Next, details of the job scheduling analysis unit 104, which is a feature of the present invention, will be described.
As shown in FIG. 1, the job scheduling analysis unit 104 is necessary for the simulation from the job data analysis unit 600 that analyzes the execution history data 115 (see FIG. 2) and the parameter information 114 (see FIG. 3). The parameter analysis unit 700 that creates simulation condition definition information (114-1 in FIG. 5), the job grouping unit 800 that performs job grouping on the data created by the job data analysis unit 600, and jobs based on these input data It is comprised by the simulation part 900 which simulates the execution condition of these. In this embodiment, the job scheduling analysis unit 104 and the system operation history collection unit 110 are provided in the same system, but may be provided in different systems.
[0017]
The result analyzed by the simulation unit 900 is output from the simulation unit 900 as the simulation result 116. An example of the simulation result 116 is shown in FIG. As the output information, as shown in the figure, a temporal transition 116-1 of job execution multiplicity after simulation, a list 116-2 of execution jobs, and the like can be cited. In the execution job list 116-2 in FIG. 4, the execution jobs in space 1 are JOB1, JOB5, and JOB3, and the execution jobs in space 2 are JOB2 and JOB8. For example, JOB1 is executed from 7:00 to 7:45. The execution time is 45 minutes. It is also possible to include resources used for each job in the execution job list.
[0018]
Next, a detailed explanatory diagram of the job scheduling analysis unit 104 is shown in FIG.
The job data analysis unit 600 creates an information cell 200 that is an information unit used for simulation in the present invention from the job execution history 115.
The information cell 200 is roughly classified into a job information cell 201, a job step information cell 202, and a file information cell 203 as shown in the information cell configuration example of FIG. 6 according to the type of history included in the job execution history 115. Is done. The information cell 200 has a table structure, and the entire information cell has a list structure when an element (pointer) in the information cell points to a related information cell. The job information cell 201 indicates a list of job step information cells 202 constituting the job. The job step information cell 202 indicates the file information cell 203 of the file accessed in the job step. These form one set and represent the overall information of the job.
Since the job execution history 115 includes a plurality of jobs to be analyzed, the job information cell 201 itself has a list structure, and is configured to sequentially point to the next job information cell 201 by a pointer. Yes.
[0019]
Next, a more detailed table structure of each information cell will be described with reference to FIGS. In each information cell, information included in the execution history is stored as necessary together with information for forming a list structure of information cells.
FIG. 7 shows the table structure of the job information cell 201. In the figure, a pointer 301 points to the next element in the list of job information cells, and a pointer 302 points to the head of the list of information cells of job steps constituting the job. Reference numeral 303 denotes a job name for identifying a job, 304 denotes a job start time, and 305 denotes an area for storing a job end time. These are used when the simulation unit 900 obtains an execution condition. Reference numeral 306 denotes an area for storing the total CPU usage time used by the job.
[0020]
FIG. 8 shows the table structure of the job step information cell 202. In the figure, a pointer 401 points to the next element (job step information cell) in the list of job step information cells, and a pointer 402 points to the top of the list of information cells of the file accessed in the job step. Is. Reference numeral 403 denotes a job step name for identifying the job step, and 404 denotes an area for storing a program name executed in the job step. FIG. 9 shows the table structure of the file information cell 203. In the figure, a pointer 501 is for pointing to the next element (file information cell) in the list of file information cells. Reference numeral 502 denotes an area for identifying a file and storing a file name used when the simulation unit 900 obtains an execution condition.
[0021]
Next, the processing flow of the job data analysis unit 600 will be described in detail with reference to the flowchart of FIG.
First, the top data is read from the job execution history 115, and the processing in steps 602 to 606 is continued until the data in the job execution history 115 becomes empty (step 601).
In the information cell list until the record is read, if there is no information cell that should store the contents of the record, such as a job start record, a new information cell is created to store the contents There is a need to. It is determined whether or not the data of the read job execution history 115 is a record type for which these new information cells are to be created (step 602).
[0022]
If the determination in step 602 is true, it is determined from the contents of the record whether the record is a job, a job step, or a file, and an information cell area of the corresponding format is secured. (Step 603) Stores data in the area and sets information for forming a list structure such as setting a pointer from the list for the new information cell (Step 604).
If the determination in step 602 is false, the data is stored in the corresponding information cell in the previously created information cell (step 605).
Thereafter, the process returns to the determination step 601 for the processing of the next data in the job execution history 115 (step 606).
[0023]
The parameter analysis unit 700 analyzes the parameter information 114 as shown in FIG. 3 and creates simulation condition definition information 114-1 as a simulation condition.
Next, the processing of the parameter analysis unit 700 will be described in detail using the flowchart of FIG.
First, parameters are read from the parameter information 114, and the processing shown in steps 702 to 703 is continued until the parameters become empty (step 701).
Which parameter is the read parameter information is determined, and is reflected in the simulation condition definition information 114-1 according to the parameter type (step 702). For example, the multiplicity at which the job is executed is read as the number of spaces from the parameter information 114, and set in the corresponding item in the simulation condition definition information 114-1.
In order to continue the processing, the process returns to the determination step 701 for the next parameter information 114 (step 703).
[0024]
The job grouping unit 800 is used when obtaining a result of job grouping focusing on the locality of file access in consideration of a loosely coupled multiprocessor system different from the current system. By the processing of the job grouping unit 800, the list of the job information cell 201 is divided into a list of information cells of jobs with overlapping files to be accessed and grouped. By performing simulation by the simulation unit 900 in units of job groups created by the job grouping unit 800, it is possible to obtain scheduling analysis results in job groups in which duplication of files to be accessed is eliminated or reduced.
[0025]
Next, the processing of the job grouping unit 800 will be described in detail with reference to the flowchart of FIG.
First, referring to the table data that specifies whether or not to perform job grouping in the simulation condition definition information 114-1, when the grouping is specified, the following steps 802 to step 802 are performed. The processing of 804 is performed, and the processing is terminated without performing any other processing (step 801).
[0026]
If it is specified in step 801 that grouping is to be performed, the job information cell 201 is read first, and the following steps 802 to 804 are continued until the list of job information cells 201 becomes empty. (Step 802).
Jobs that access the same file as the job represented by the job information cell read in step 802 are sequentially searched from the list of job information cells, and these jobs are regarded as the same group and grouped. Also, the job accessing the same file as the job made in the same group here is also made the same group (step 803).
For the next job information cell (step 804), the process returns to step 802 and the same processing is performed again.
[0027]
Next, the job execution simulation procedure in the simulation unit 900 will be described in detail with reference to the flowcharts of FIGS.
FIG. 14 schematically shows the list switching of the job information cell 201 performed in FIG. In FIG. 14, 201-j (j = 1 to j) represents job information cells having different contents. An example of the table structure of the space 1001 in FIG. 14 is shown in FIG.
In the table structure of the space 1001 shown in FIG. 15, the pointer 1101 is for pointing to the next space 1001. A pointer 1102 is used to point to the top of a list of job information cells connected to the space 1001.
[0028]
Next, the operation of the simulation unit 900 will be described in detail along the flowchart of FIG.
First, information is read from the simulation condition definition information 114-1. The following steps 902 to 903 are performed until the simulation condition definition information 114-1 becomes empty (step 901). When the simulation condition definition information 114-1 becomes empty, the process proceeds to step 904.
[0029]
In step 902, processing corresponding to the loaded simulation condition is performed. For example, if it is a parameter of the number of spaces that represents the maximum number of job execution multiplicity, that number of spaces 1001 is secured in the memory area. If it is the CPU performance ratio, the CPU usage time 306 in the job information cell 201 is converted according to the ratio, and at the same time, the job end time 305 is rewritten (see FIG. 7).
[0030]
In order to sequentially extract the job information cells 201 from the head of the list 1 of job information cells to be connected, it is determined whether there is a job information cell 201 (step 904). The following steps 905 to 911 are performed until the data are taken out from the head in order and become empty.
First, a scheduling condition such as file delivery of the above job with another job is obtained (step 905).
Next, a time at which execution can be started is calculated based on the scheduling condition obtained in step 905 (step 906).
The contents of the job information cell list connected to the space 1001 are searched at the time obtained in step 906, and it is determined whether or not there is a vacancy at the time when the execution can be started (step 907).
[0031]
If the determination in step 907 is true (executable), a job information cell is connected to the empty space 1001. When the job information cell is connected, the job step information cell indicated by the job information cell is also connected to the space 1001 (step 908).
If the determination in step 907 is false (unexecutable), the job information cell is connected to the space 1001 that can be executed earliest after the execution startable time obtained in step 906 (step 909). .
Thereafter, after the connection of the above-described job information cell is completed, data in the job information cell such as job execution time is rewritten (step 910), and the process returns to the determination step 904 for the next job information cell.
Note that a time during which the job is not executed may occur during processing in the simulation unit 900. By expressing these times by job information cells indicating that they are vacant, idle times in the job information cell list of the space 1001 can be expressed.
[0032]
Next, how the job information cell is specifically changed in steps 904 to 911 in the simulation unit 900 will be described with reference to FIG.
First, in decision step 904, a list of unprocessed job information cells to be connected is referred to. In the case of the example shown in FIG. 14, since there are 201-1, 201-2,..., 201-i as unprocessed job information cells, the determination step 904 is YES. , 201-i, 201-i are taken out in order, and the processing from step 905 to step 911 is performed on each of them.
[0033]
First, when processing the head job information cell 201-1 in the list, in step 905 to step 908 or step 909, the job information cell 201- in the list of job information cells representing the result of the simulation on the memory is displayed. Determine which job 1 should be executed after. In the case of the example in FIG. 14, it is executed after the job information cell 201-4. In that case, in step 908 or step 909, the job information cell 201-1 is switched after 201-4. At the same time, the job information cell 201-1 is deleted from the unprocessed job information cell list 200.
As described above, in this embodiment, the job execution status is represented as a list of information cells on the memory, and simulation is performed using the parameter information and the job execution history.
[0034]
【The invention's effect】
According to the present invention, it is possible to obtain a job scheduling plan that can shorten the job processing time in the current system.
Further, the job execution status due to the system configuration change can be grasped based on the job operating status of the current system before the system configuration change.
Furthermore, since the simulation is performed based on the job execution history, it is possible to perform a static evaluation without subjecting the target system to the burden of dynamic job monitoring.
[Brief description of the drawings]
FIG. 1 is an example of a computer system provided with a job scheduling analysis unit 104 for realizing a job scheduling analysis method of the present invention.
FIG. 2 is an explanatory diagram of a job execution history 115 serving as input information of the present invention.
FIG. 3 is an explanatory diagram of parameter information 114 serving as input information according to the present invention.
FIG. 4 is an explanatory diagram of a simulation result 116 serving as output information of the present invention.
FIG. 5 is a specific configuration example of a job scheduling analysis unit 104 according to the present invention.
FIG. 6 is an explanatory diagram showing a configuration of an information cell that is a job and data related to the job used in the job scheduling analysis method of the present invention.
FIG. 7 is an explanatory diagram showing a table structure of a job information cell for storing job data.
FIG. 8 is an explanatory diagram showing a table structure of a job step information cell for storing job step data.
FIG. 9 is an explanatory diagram showing a table structure of a file information cell for storing data of an access file.
10 is a flowchart illustrating processing of a job data analysis unit 600. FIG.
FIG. 11 is a flowchart illustrating processing of a parameter analysis unit 700.
FIG. 12 is a flowchart illustrating processing of a job grouping unit.
FIG. 13 is a flowchart illustrating processing of a simulation unit 900 for scheduling.
FIG. 14 is an explanatory diagram showing a data structure for representing job execution in simulation.
FIG. 15 is an explanatory diagram showing a table structure of a space 1001 used in the simulation.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 101 ... Computer, 102 ... CPU, 103 ... Memory, 104 ... Job scheduling analysis part, 600 ... Job data analysis part, 700 ... Parameter analysis part, 800 ... Job grouping part, 900 ... Simulation part, 110 ... System operation history collection part , 111... Job schedule program, 112... Job starter, 113... Register creation unit, 114 .. Parameter information, 115 .. Job execution history, 116.

Claims (5)

A method in which a processor provided in a computer determines an execution order between jobs that need to refer to a predetermined file when a plurality of jobs operate in parallel in a certain system,
The processor receives history information of a file accessed during job execution from the outside for each job, stores the history information in a memory, reads the history information from the memory, and includes a table included in the history information. Write information to the corresponding job information cell with structure,
The processor reads the parameter information including the number of spaces required for external simulation and the CPU performance ratio into the memory, reads the parameter information from the memory, analyzes the parameter information, and serves as a simulation condition. Create definition information,
The processor sequentially reads job information cells from the list of job information cells in the memory, determines whether there is a job information cell connected to a space generated based on the simulation condition definition information, and the simulation condition When it is determined that there is a job information cell connected to the space generated based on the definition information, and the file accessed by the job in the job information cell is accessed by another job, the other job Find the job scheduling condition that starts the job after completion , calculate the time when execution can start from the obtained condition, determine whether there is a vacancy at the corresponding time in the specified space, If there is, the job information cell is connected to the space by accessing the memory and writing a pointer, A job execution order determination method characterized by rewriting data in a job information cell such as a job execution time after connection of the job information cell and determining which job the job information cell is executed after .   2. The job execution order determination method according to claim 1, wherein the processor acquires information related to the execution multiplicity of the job of the system in addition to the file history information.   3. The job execution order determination method according to claim 1, wherein the simulation of job execution includes a step of outputting the simulation result. Based on the job execution history, jobs using the same file are grouped into one group,
4. The job execution order determination method according to claim 1, wherein simulation is performed for jobs included in the grouped group.   The job execution order determination method according to any one of claims 1 to 4, wherein the system is a system different from a system in which the job is executed.

Images